...

Introduction

Usually, Jenkins master controller and Jenkins agent are located in the same network, therefore it is quite easy to adopt Jenkins agents using SSH. This time we will talk about the situation when Jenkins master controller and bunch of other DevOps tools are running in "network A" and Jenkins agents are running in "network B". Quite often connections from A to B are rejected, however, connections from B to A are accepted. There might be various reasons, in general, network B has elevated security expectations. The typical setup is Jenkins master controller located in public cloud and Jenkins agent(s) located in a private customer network. Private customer network might be needed in such because some sort of higher data privacy requirements must be matched or some specialised hardware is present in customer premises and it cannot be accessible directly from the public internet.

...

The recommended solution is mastercontroller/agent setup, when the master controller runs in a public environment and agent is running on a server located in a private network. The agent is also called "on-premise executor" (OPE). Jobs are then performing jobs/tasks on such agent in internal/private networks. Jenkins Agent has an active connection from the internal network to internet accessible Jenkins Master via recommended JNLP port tcp/9000 and keeps listening to builds/jobs. Port can be changed, but we will keep talking about 9000. NO direct or NAT network connection is required from the internet to internal network. It is a secure and simple solution.

• Network
  • firewall opening for port tcp/9000 from relevant source agent(s) IP(s) in internal network towards internet in general (destination 0.0.0.0)
    • network layer (using TDS portal network functionality)
    • operating system layer (firewalld if needed)
• Jenkins mastercontroller running in public
  
  • Decide which port you are going to use as we use fixed setup in our case. We are choosing 9000 in this guide.
  • Make sure to have port 9000 opened:
    • on network/firewall layer - Firewall

    • in OS level, for example firewalld:

      Code Block
      firewall-cmd --add-port=9000/tcp --permanent firewall-cmd --reload

      
      

  • Listening
  • running in public
  • listening on JNLP port tcp/9000
    • Go to https://jenkins.xxx.tds.customerx.com/configureSecurity (remember to use correct URL of your Jenkins mastercontroller)
    • Set "TCP port for inbound agents" to Fixed:9000
    • Open "advanced" and choose "Inbound TCP Agent Protocol/4 (TLS encryption)" (deselect others if not relevant)
  • node added according to the following steps
    • Go to https://jenkins.xxx.tds.customerx.com/computer/new (remember to use correct URL of your Jenkins mastercontroller)
    • Set "Node name" to relevant name useful for you
    • Choose "Permanent"
    • Set "Remote root directory" to "/home/jenkinsopejenkins-agent"
    • Set "Launch method" to "Launch agent by connecting it to the mastercontroller" previously called "Launch agent via Java Web Start"
    • Click "Save"
  • now go to newly created node and copy secret/token for connecting agent
    • Go to https://jenkins.xxx.tds.customerx.com/computer/XXX (remember to use correct URL of your Jenkins master controller and replace XXX with the name of your node)

    • You will see something like:

      Code Block

      Run from agent command line: java -jar agent.jar -jnlpUrl https://jenkins.xxx.tds.customerx.com/computer/test/slave-agent.jnlp -secret 8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762 -workDir "/home/jenkinsopejenkins-agent" Run from agent command line, with the secret stored in a file: echo 8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762 > secret-file java -jar agent.jar -jnlpUrl https://jenkins.xxx.tds.customerx.com/computer/test/slave-agent.jnlp -secret @secret-file -workDir "/home/jenkinsopejenkins-agent"

      Please copy only the secret, which is for example in this case "8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762"

• running in public
• Jenkins agent node (slave) - or so-called "on-premise executor"
  • running on a server in the internal network(s)
  • agent service(s) with service auto-start to assure automatic re-connect to Jenkins master controller at any time even after server reboot
    • Install dependencies

      • CentOS

        Code Block
        yum install java-1.8.0-openjdk-devel git -y # you can install also other dependencies that will be required for your jobs

        
        

      • Ubuntu

        Code Block
        apt-get update; apt-get install openjdk-8-jdk git -y # you can install also other dependencies that will be required for your jobs

        
        

    • Installing agent

      • Prepare a folder for config

        Code Block
        mkdir -p /data/configs mkdir -p /opt/jenkins-agent

        
        

      • Create service file /tmp/jenkinsopefile /opt/jenkins-agent/jenkins-agent.service

        Code Block
        title jenkinsope.service
        [Unit] Description=Jenkins Agent - On Premise Executor Wants=network.target After=network.target [Service] # EnvironmentFile cannnot be used on Debian/Ubuntu anymore - Reference: https://github.com/varnishcache/pkg-varnish-cache/issues/24 # So we are using drop-in config /etc/systemd/system/jenkinsopejenkins-agent.service.d/local.conf ExecStart=/usr/bin/java -Xms${JAVA_MEMORY} -Xmx${JAVA_MEMORY} -jar /usropt/binjenkins-agent/agent.jar -jnlpUrl ${MASTERCONTROLLER_URL}/computer/${NODE_NAME}/slavejenkins-agent.jnlp -secret ${SECRET} -workDir "${WORK_DIR}" User=jenkinsopejenkins-agent Restart=always RestartSec=10 StartLimitInterval=0 [Install] WantedBy=multi-user.target

        
        

      • Create config file /data/configs/jenkinsopejenkins-agent.conf

        Code Block
        JAVA_MEMORY=512m MASTERCONTROLLER_URL=https://jenkins.xxx.tds.customerx.com NODE_NAME=XXX SECRET=8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762 WORK_DIR=/home/jenkinsopejenkins-agent

        
        

      • Create script /opt/tmp/jenkinsopejenkins-agent/jenkins-agent-install.sh

        Code Block
        #!/bin/bash AGENT_APP_HOME=/opt/jenkins-agent SERVICE_USER=jenkins-agent MAINCONFIG=/data/configs/jenkins-agent.conf OLD_CONF=/data/configs/jenkinsope.conf if [ -f $OLD_CONF ];then mv -f $OLD_CONF $MAINCONFIG;fi source $MAINCONFIG OLD_USER=jenkinsope OLD_HOME=/home/$OLD_USER if [ -d $OLD_HOME ];then sed -i 's#^WORK_DIR.*#WORK_DIR=/home/jenkins-agent#g' $MAINCONFIG source $MAINCONFIG systemctl stop jenkinsope mv $OLD_HOME $WORK_DIR userdel -rf $OLD_USER rm -rf /etc/systemd/system/jenkins-agent.service.d rm -f /usr/lib/systemd/system/jenkinsope.service fi
        Code Block
        useradd -m -s /bin/bash jenkinsope$SERVICE_USER 2> /dev/null mkdir -p /home/jenkinsope$WORK_DIR/.ssh chmod 700 /home/jenkinsope/$WORK_DIR/.ssh touch /home/jenkinsope$WORK_DIR/.ssh/config chmod 600 /home/jenkinsope$WORK_DIR/.ssh/* chown jenkinsope:jenkinsope$SERVICE_USER:$SERVICE_USER -R /home/jenkinsope/.ssh source /data/configs/jenkinsope.conf wget$WORK_DIR curl -s ${MASTERCONTROLLER_URL}/jnlpJars/agent.jar -O /usr/bino $AGENT_APP_HOME/agent.jar chmod 644 /usr/bin/$AGENT_APP_HOME/agent.jar install -D -m 644 /tmp/jenkinsope$AGENT_APP_HOME/jenkins-agent.service /usr/lib/systemd/system/jenkinsopejenkins-agent.service mkdir -p /etc/systemd/system/jenkinsopejenkins-agent.service.d echo "[Service]" > /etc/systemd/system/jenkinsopejenkins-agent.service.d/local.conf sed 's#^#Environment=#g' /data/configs/jenkinsope.conf$MAINCONFIG >> /etc/systemd/system/jenkinsopejenkins-agent.service.d/local.conf systemctl daemon-reload systemctl restart jenkinsopejenkins-agent systemctl enable jenkinsopejenkins-agent systemctl status jenkinsope jenkins-agent echo "0 4 * * * root $AGENT_APP_HOME/jenkins-agent-install" > /etc/cron.d/jenkins-agent-update

        
        

      • Run install script

        Code Block
        chmod +x /opt/tmp/jenkinsopejenkins-agent/jenkins-agent-install.sh /opt/tmp/jenkinsopejenkins-agent/jenkins-agent-install.sh

        
        

    • Uninstalling agent (for cleanup purposes or if you messed up something)

      • Create script /tmp/jenkinsope-uninstall.shscript /opt/jenkins-agent/jenkins-agent-uninstall

        Code Block
        systemctl disable jenkinsopejenkins-agent systemctl stop jenkinsopejenkins-agent rm -f /usr/lib/systemd/system/jenkinsopejenkins-agent.service rm -rf /etc/systemd/system/jenkinsopejenkins-agent.service.d systemctl daemon-reload userdel -r jenkinsopejenkins-agent rm -rf /home/jenkinsopejenkins-agent

        
        

      • Run install script

        Code Block
        chmod +x /opt/tmp/jenkinsopejenkins-agent/jenkins-agent-uninstall.sh /opt/tmp/jenkinsopejenkins-agent/jenkins-agent-uninstall.sh

        
        

Inspired by:

• service itself - https://github.com/jenkinsci/systemd-slave-installer-module/blob/master/src/main/resources/org/jenkinsci/modules/systemd_slave_installer/jenkins-slave.service
• service parameters/options - https://gist.github.com/dragolabs/05dfe1c0899221ce51204dbfe7feecbb
• way of service installing - https://gist.github.com/michaelneale/9635744