Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Decide which port you are going to use as we use fixed setup in our case. We are choosing 9000 in this guide.
  • Make sure to have TCP port 9000 opened:
    • on network/firewall layer - Firewall
      • firewall opening for port 9000/tcp from relevant source agent(s) IP(s) or network(s)

        Note

        We do not recommend too wide opening for 0.0.0.0 when you have Jenkins deployed in public cloud accessible from internet as it could be unnecessarily yet another potential target of attacks.


    • in OS level, for example firewalld:

      Code Block
      firewall-cmd --add-port=9000/tcp --permanent
      firewall-cmd --reload


      Info

      This generic 9000/tcp opening is fine if you are controlling access of each and every source trusted IP or IP pool(s) via portal firewall settings. Otherwise we recommend narrowing down the scope to your trusted source IP or IP pool(s).


  • Listening on JNLP port tcp/9000
  • node added according to the following steps
    • Go to https://jenkins.xxx.tds.customerx.com/computer/new (remember to use correct URL of your Jenkins controller)
    • Set "Node name" to relevant name useful for you, we will use "node01" for simplicity of this example. Here are some recommended examples for inspiration - short name (node01), FQDN (node01.xxx.tds.customerx.com). 
    • Choose "Permanent"
    • Set "Remote root directory" to "/homedata/jenkins-agent"
    • Set "Launch method" to "Launch agent by connecting it to the controller" (previously called "Launch agent via Java Web Start")
    • Click "Save"
  • now go to newly created node and copy secret/token for connecting agent
    • Go to https://jenkins.xxx.tds.customerx.com/computer/XXX (remember to use correct URL of your Jenkins controller and replace XXX with the name of your node)
    • You will see something like :in very first block called "Run from agent command line: (Unix)":

      Code Block
      curl -sO 
      Code Block
      Run from agent command line:
      
      java -jar agent.jar -jnlpUrl https://jenkins.xxx.tds.customerx.com/computerjnlpJars/test/slave-agent.jnlp -secret 8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762 -workDir "/home/jenkins-agent"
      Run from agent command line, with the secret stored in a file:
      
      echo 8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762 > secret-file
      agent.jar
      java -jar agent.jar -jnlpUrl https://jenkins.xxx.tds.customerx.com/computer/testnode01/slavejenkins-agent.jnlp -secret @secret-file8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762 -workDir "/homedata/jenkins-agent"

      Please copy only the secret, which is for example in this case "8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762"

...

  • This agent can be running on a server next to your Jenkins controller/master or in the internal network(s)
  • agent service(s) with service auto-start to assure automatic re-connect to Jenkins controller at any time even after server reboot
    • Install dependencies
      • CentOS 7

        Code Block
        yum install java-11-openjdk-devel git -y
        # you can install also other dependencies that will be required for your jobs


      • CentOS 8/9

        Code Block
        yum install java-17-openjdk-devel git -y
        # you can install also other dependencies that will be required for your jobs


      • Ubuntu

        Code Block
        apt-get update; apt-get install openjdk-11-jdk git -y
        # you can install also other dependencies that will be required for your jobs


    • Installing agent
      • Prepare a folder for config

        Code Block
        mkdir -p /data/configs
        mkdir -p /opt/jenkins-agent


      • Create service file /opt/jenkins-agent/jenkins-agent.service

        Code Block
        titlejenkinsope.service
        [Unit]
        Description=Jenkins Agent - On Premise Executor
        Wants=network.target
        After=network.target
        
        [Service]
        # EnvironmentFile cannnot be used on Debian/Ubuntu anymore - Reference: https://github.com/varnishcache/pkg-varnish-cache/issues/24
        # So we are using drop-in config /etc/systemd/system/jenkins-agent.service.d/local.conf
        ExecStart=/usr/bin/java -Xms${JAVA_MEMORY} -Xmx${JAVA_MEMORY} -jar /opt/jenkins-agent/agent.jar -jnlpUrl ${CONTROLLER_URL}/computer/${NODE_NAME}/jenkins-agent.jnlp -secret ${SECRET} -workDir "${WORK_DIR}"
        User=jenkins-agent
        Restart=always
        RestartSec=10
        StartLimitInterval=0
        
        [Install]
        WantedBy=multi-user.target


      • Create config file /data/configs/jenkins-agent.conf

        Code Block
        JAVA_MEMORY=512m
        CONTROLLER_URL=https://jenkins.xxx.tds.customerx.com
        NODE_NAME=XXX
        SECRET=8b2911d98400bad5d45635b812b5f2e8e7c1d216bbbae9422a3ba57c691bf762
        WORK_DIR=/homedata/jenkins-agent


      • Create script /opt/jenkins-agent/jenkins-agent-install

        Code Block
        #!/bin/bash
        AGENT_APP_HOME=/opt/jenkins-agent
        SERVICE_USER=jenkins-agent
        MAINCONFIG=/data/configs/jenkins-agent.conf
        OLD_CONF=/data/configs/jenkinsope.conf
        if [ -f $OLD_CONF ];then mv -f $OLD_CONF $MAINCONFIG;fi
        source $MAINCONFIG
        OLD_USER=jenkinsope
        OLD_HOME=/home/$OLD_USER
        if [ -d $OLD_HOME ];then
          sed -i 's#^WORK_DIR.*#WORK_DIR=/home/jenkins-agent#g' $MAINCONFIG
          sed -i 's#^MASTER_URL#CONTROLLER_URL#g' $MAINCONFIG
          source $MAINCONFIG
          # Stopping old service
          systemctl stop jenkinsope
          sleep 10
          mv $OLD_HOME $WORK_DIR
          userdel -rf $OLD_USER
          rm -rf /etc/systemd/system/jenkins-agent.service.d
          rm -f /usr/lib/systemd/system/jenkinsope.service
        fi
        useradd -m -s /bin/bash $SERVICE_USER 2> /dev/null
        mkdir -p $WORK_DIR/.ssh
        chmod 700 $WORK_DIR/.ssh
        touch $WORK_DIR/.ssh/config
        chmod 600 $WORK_DIR/.ssh/*
        echo "Changing ownership of home/work folder ($WORK_DIR) and its content (can take long time with many files)..."
        chown $SERVICE_USER:$SERVICE_USER -R $WORK_DIR
        curl -s ${CONTROLLER_URL}/jnlpJars/agent.jar -o $AGENT_APP_HOME/agent.jar
        chmod 644 $AGENT_APP_HOME/agent.jar
        install -D -m 644 $AGENT_APP_HOME/jenkins-agent.service /usr/lib/systemd/system/jenkins-agent.service
        mkdir -p /etc/systemd/system/jenkins-agent.service.d
        echo "[Service]" > /etc/systemd/system/jenkins-agent.service.d/local.conf
        sed 's#^#Environment=#g' $MAINCONFIG >> /etc/systemd/system/jenkins-agent.service.d/local.conf
        systemctl daemon-reload
        systemctl restart jenkins-agent
        systemctl enable jenkins-agent
        systemctl status jenkins-agent
        echo "0 4 * * * root $AGENT_APP_HOME/jenkins-agent-install" > /etc/cron.d/jenkins-agent-updateTODO


      • Run install script

        Code Block
        chmod +x /opt/jenkins-agent/jenkins-agent-install
        /opt/jenkins-agent/jenkins-agent-install


    • Uninstalling agent (for cleanup purposes or if you messed up something)
      • Create script /opt/jenkins-agent/jenkins-agent-uninstall

        Code Block
        systemctl disable jenkins-agent
        systemctl stop jenkins-agent
        rm -f /usr/lib/systemd/system/jenkins-agent.service
        rm -rf /etc/systemd/system/jenkins-agent.service.d
        systemctl daemon-reload
        userdel -r jenkins-agent
        rm -rf /homedata/jenkins-agent


      • Run install script

        Code Block
        chmod +x /opt/jenkins-agent/jenkins-agent-uninstall
        /opt/jenkins-agent/jenkins-agent-uninstall


...

...

Troubleshooting

Java Runtime class files recognition errors

Jenkins agent

...

installing procedure tries to be smart enough to detect and use correct Java (OpenJDK 11 or 17), however in some cases it might fail.

Symptoms

  • service jenkins-agent is constantly failing

    Code Block
    [root@node01 ~]# systemctl status

Steps:

  • Run script on Jenkins agent:

    Code Block
    systemctl stop jenkins-agent
    systemctl disable jenkins-agent
    
    mkdir -p /data
    if [ ! -d /data/jenkins-agent ] && [ -d /home/jenkins-agent ];then mv /home/jenkins-agent /data/jenkins-agent; else echo "Agent data are either already migrated or not present at all, please check it properly!"; sleep 10;done
    
    id -u jenkins-agent
    id -g jenkins-agent
    
    usermod -u 900 jenkins-agent
    groupmod -g 900 jenkins-agent
    
    id -u jenkins-agent
    id -g jenkins-agent
    
    chown jenkins-agent:jenkins-agent -R /data/jenkins-agent
    It applies following changes
    • stop jenkins agent
    • move data to new location
    • move /op/jenkins-agent into new location
    • update jenkins agent user ID
    • change ownership of jenkins agent files to jenkins agent to reflect new user ID

    update registered agents in Jenkins to use new path

    Info

    We do this step manually in this example. It can be automated if you have many agents or if you want to apply it on all agents at once.

    Then continue with standard agent setup according to 
    .service - Jenkins Agent - On Premise Executor
       Loaded: loaded (/usr/lib/systemd/system/jenkins-agent.service; enabled; vendor preset: disabled)
      Drop-In: /etc/systemd/system/jenkins-agent.service.d
               └─local.conf
       Active: activating (auto-restart) (Result: exit-code) since Sun 2023-10-08 13:40:53 UTC; 1s ago
      Process: 9960 ExecStart=/usr/bin/java -Xms${JAVA_MEMORY} -Xmx${JAVA_MEMORY} -jar /opt/jenkins-agent/agent.jar -jnlpUrl ${CONTROLLER_URL}/computer/${NODE_NAME}/jenkins-agent.jnlp -secret ${SECRET} -workDir ${WORK_DIR} (code=exited, status=1/FAILURE)
     Main PID: 9960 (code=exited, status=1/FAILURE)
    
    Oct 08 13:40:53 loadgen systemd[1]: Unit jenkins-agent.service entered failed state.
    Oct 08 13:40:53 loadgen systemd[1]: jenkins-agent.service failed.


  • following errors can be spotted in journal

    Code Block
    journalctl -fu jenkins-agent
    
    Oct 08 13:51:57 loadgen systemd[1]: jenkins-agent.service holdoff time over, scheduling restart.
    Oct 08 13:51:57 loadgen systemd[1]: Stopped Jenkins Agent - On Premise Executor.
    Oct 08 13:51:57 loadgen systemd[1]: Started Jenkins Agent - On Premise Executor.
    Oct 08 13:51:57 loadgen java[11258]: Error: A JNI error has occurred, please check your installation and try again
    Oct 08 13:51:57 loadgen java[11258]: Exception in thread "main" java.lang.UnsupportedClassVersionError: hudson/remoting/Launcher has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
    Oct 08 13:51:57 loadgen java[11258]: at java.lang.ClassLoader.defineClass1(Native Method)
    ...
    Oct 08 13:51:57 loadgen java[11258]: at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:621)
    Oct 08 13:51:57 loadgen systemd[1]: jenkins-agent.service: main process exited, code=exited, status=1/FAILURE
    Oct 08 13:51:57 loadgen systemd[1]: Unit jenkins-agent.service entered failed state.
    Oct 08 13:51:57 loadgen systemd[1]: jenkins-agent.service failed.


Workaround

  • Switch default java to JDK 11 or 17

    Code Block
    update-alternatives --config java


  • Inform other users of system that you made that change, as it might affect builds that were dependent on previously configured java version.