Interestingly, whenever Jenkins agent is executing PMD/Findbugs task, it runs into " Resource temporarily unavailable". I have
- verified TCP parameters i.e. tcp_keepalive_time, tcp_keepalive_intvl, tcp_keepalive_probes & tcp_fin_timeout ; All of them have higher values then what cloudbees recommended at https://support.cloudbees.com/hc/en-us/articles/115001369667-dedicated-SSH-agents-formerly-slaves-get-disconnected - set heap space to 2-4Gb - disabled ping thread from Jenkins agent by setting "-Dhudson.remoting.Launcher.pingIntervalSec=-1" Please let me know if there is anything else I should consider. On Friday, December 11, 2020 at 12:11:39 PM UTC-5 [email protected] wrote: > The most common issue is related to disconnections because there is no > traffic between the Jenkins instance and the agent, for that, you have to > tune the TCP stack of your OS (see > https://support.cloudbees.com/hc/en-us/articles/115001369667-dedicated-SSH-agents-formerly-slaves-get-disconnected), > > or enable the keepalive option in the SSH protocol this can be configured > by setting ClientAliveInterval or TCPKeepAlive on the SSH server > (/etc/ssh/sshd_config), also by setting ServerAliveInterval or TCPKeepAlive > options for the user connection (/etc/ssh/ssh_config or ~/.ssh/ssh_config) > > https://www.freebsd.org/cgi/man.cgi?sshd_config(5) > https://www.freebsd.org/cgi/man.cgi?ssh_config(5) > > Also, check you follow the best practices to configure your SSH agents and > enable SSH verbose log output in your service (see > https://github.com/jenkinsci/ssh-slaves-plugin/blob/master/doc/TROUBLESHOOTING.md > ) > > El miércoles, 9 de diciembre de 2020 a las 17:53:19 UTC+1, > [email protected] escribió: > >> Hello Jenkins community, >> >> I have setup Jenkins agents over SSH and this agent frequently goes >> offline with Resource temporarily unavailable error. I had to configure my >> Jenkins agents to communicate over JNLP. Any advise on how to fix this >> issue? >> >> Jenkins v2.249.1 >> >> [12/04/20 15:23:00] [SSH] Checking java version of java >> [12/04/20 15:23:01] [SSH] java -version returned 1.8.0_202. >> [12/04/20 15:23:01] [SSH] Starting sftp client. >> [12/04/20 15:23:03] [SSH] Remote file system root $JENKINS_SSH_DATA does >> not exist. Will try to create it... >> [12/04/20 15:23:03] [SSH] Copying latest remoting.jar... >> [12/04/20 15:23:03] [SSH] Copied 1,521,553 bytes. >> Expanded the channel window size to 4MB >> [12/04/20 15:23:03] [SSH] Starting agent process: cd "$JENKINS_SSH_DATA" >> && java -jar remoting.jar -workDir $JENKINS_SSH_DATA -jar-cache >> $JENKINS_SSH_DATA/remoting/jarCache >> Dec 04, 2020 3:23:29 PM org.jenkinsci.remoting.engine.WorkDirManager >> initializeWorkDir >> INFO: Using $JENKINS_SSH_DATA/remoting as a remoting work directory >> Dec 04, 2020 3:23:29 PM org.jenkinsci.remoting.engine.WorkDirManager >> setupLogging >> INFO: Both error and output logs will be printed to >> $JENKINS_SSH_DATA/remoting >> <===[JENKINS REMOTING CAPACITY]===>channel started >> Remoting version: 4.5 >> This is a Unix agent >> Evacuated stdout >> Agent successfully connected and online >> The Agent is connected, disconnect it before to try to connect it again. >> Dec 04, 2020 3:33:58 PM org.eclipse.jgit.util.FS discoverGitSystemConfig >> WARNING: Exception caught during execution of command '[git, config, >> --system, --edit]' in '$GIT_PATH/bin', return code '128', error message >> 'fatal: Invalid path '$GIT_PATH/etc': No such file or directory >> ' >> Dec 04, 2020 3:33:58 PM org.eclipse.jgit.util.FS$FileStoreAttributes >> saveToConfig >> WARNING: locking FileBasedConfig[$JENKINS_PATH/.config/jgit/config] >> failed after 5 retries >> Dec 04, 2020 3:33:59 PM >> org.jenkinsci.remoting.util.AnonymousClassWarnings warn >> WARNING: Attempt to (de-)serialize anonymous class >> com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTriggerBuildChooser$1; >> >> see: https://jenkins.io/redirect/serialization-of-anonymous-classes/ >> Dec 04, 2020 3:34:24 PM hudson.remoting.Request$2 run >> WARNING: Failed to send back a reply to the request >> hudson.remoting.Request$2@493c5a4e >> java.io.IOException: Resource temporarily unavailable >> at java.io.FileOutputStream.writeBytes(Native Method) >> at java.io.FileOutputStream.write(FileOutputStream.java:313) >> at >> hudson.remoting.StandardOutputStream.write(StandardOutputStream.java:83) >> at >> hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:89) >> at >> hudson.remoting.ChunkedOutputStream.sendBreak(ChunkedOutputStream.java:62) >> at >> hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:46) >> at >> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:46) >> at hudson.remoting.Channel.send(Channel.java:766) >> at hudson.remoting.Request$2.run(Request.java:388) >> at >> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >> Dec 04, 2020 5:25:30 PM hudson.remoting.Request$2 run >> WARNING: Failed to send back a reply to the request >> hudson.remoting.Request$2@2511e2d4 >> java.io.IOException: Resource temporarily unavailable >> at java.io.FileOutputStream.writeBytes(Native Method) >> at java.io.FileOutputStream.write(FileOutputStream.java:326) >> at >> hudson.remoting.StandardOutputStream.write(StandardOutputStream.java:88) >> at >> hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:90) >> at >> hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:85) >> at >> hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:54) >> at java.io.OutputStream.write(OutputStream.java:75) >> at >> hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45) >> at >> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:46) >> at hudson.remoting.Channel.send(Channel.java:766) >> at hudson.remoting.Request$2.run(Request.java:388) >> at >> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >> ERROR: Connection terminated >> java.io.StreamCorruptedException: invalid stream header: 00025B42 >> at >> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:866) >> at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358) >> at >> hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) >> at hudson.remoting.Command.readFrom(Command.java:142) >> at hudson.remoting.Command.readFrom(Command.java:128) >> at >> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) >> at >> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63) >> Dec 04, 2020 5:32:30 PM hudson.slaves.ChannelPinger$1 onDead >> INFO: Ping failed. Terminating the channel channel. >> java.util.concurrent.TimeoutException: Ping started at 1607120910589 >> hasn't completed by 1607121150590 >> at hudson.remoting.PingThread.ping(PingThread.java:134) >> at hudson.remoting.PingThread.run(PingThread.java:90) >> >> Thanks, >> Jigar R >> >> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/030aced0-ba6e-4012-a60a-58208d350544n%40googlegroups.com.
