Root cause turned out to be an outdated commons-io jar I had installed for another project in the Jenkins user's ~/Library/Java/Extensions directory. When I updated to the latest version, both Jenkins and the other project were happy. The problem manifested at the time of the upgrade attempt because that was the first time the Jenkins slave on that host had been restarted since the installation of the outdated jar.
On Wednesday, June 10, 2015 at 3:16:14 PM UTC-7, Ross Oliver wrote: > > Greetings, > > I am running a Jenkins master and several slaves all on Mac OS 10.10. > Yesterday I attempted an upgrade from 1.598 to 1.609.1, and immediately > all jobs on one slave started failing, unable to check out from git. The > other slaves were unaffected an continued to operate normally. After some > basic troubleshooting, I decided I needed to get these jobs working again, > so I reverted to 1.598. I also had to manually restore the config.xml from > the previous day, as the storage of node data in 1.609.1 was incompatible > with 1.598. The revert seems successful, but the jobs on the 1 slave were > still failing with the same errors. The jobs would run fine if moved to > another slave. Still other than the slave.jar file, I could not find > anything on the slave that the upgrade/revert might have changed. For > troubleshooting, I created a very simple freestyle job that executed a > single shell command "echo this is here" That job failed with the > following log message (slave host name removed per company confidentiality > policy): > > Started by user ross_oliver <http://mobilejenkins.apple.com/user/ross_oliver> > [EnvInject] - Loading node environment variables. > Building remotely on <slave host name> (java7 master2) in workspace > <workspace path> > FATAL: command execution failedjava.io.IOException > <http://stacktrace.jenkins-ci.org/search?query=java.io.IOException>: Remote > call on <slave host name> failed > at hudson.remoting.Channel.call(Channel.java:760) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.remoting.Channel.call&entity=method> > at hudson.Launcher$RemoteLauncher.launch(Launcher.java:916) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.Launcher$RemoteLauncher.launch&entity=method> > at hudson.Launcher$ProcStarter.start(Launcher.java:381) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.Launcher$ProcStarter.start&entity=method> > at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.tasks.CommandInterpreter.perform&entity=method> > at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.tasks.CommandInterpreter.perform&entity=method> > at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.tasks.BuildStepMonitor$1.perform&entity=method> > at > hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761) > > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractBuild$AbstractBuildExecution.perform&entity=method> > at hudson.model.Build$BuildExecution.build(Build.java:199) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Build$BuildExecution.build&entity=method> > at hudson.model.Build$BuildExecution.doRun(Build.java:160) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Build$BuildExecution.doRun&entity=method> > at > hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractBuild$AbstractBuildExecution.run&entity=method> > at hudson.model.Run.execute(Run.java:1718) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Run.execute&entity=method> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.FreeStyleBuild.run&entity=method> > at hudson.model.ResourceController.execute(ResourceController.java:89) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.ResourceController.execute&entity=method> > at hudson.model.Executor.run(Executor.java:240) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Executor.run&entity=method> > Caused by: java.lang.NoSuchFieldError > <http://stacktrace.jenkins-ci.org/search?query=java.lang.NoSuchFieldError>: > NULL_OUTPUT_STREAM > at hudson.Launcher$ProcStarter.<init>(Launcher.java:152) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.Launcher$ProcStarter.%3Cinit%3E&entity=method> > at hudson.Launcher.launch(Launcher.java:408) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.Launcher.launch&entity=method> > at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1129) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.Launcher$RemoteLaunchCallable.call&entity=method> > at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.Launcher$RemoteLaunchCallable.call&entity=method> > at hudson.remoting.UserRequest.perform(UserRequest.java:121) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.remoting.UserRequest.perform&entity=method> > at hudson.remoting.UserRequest.perform(UserRequest.java:49) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.remoting.UserRequest.perform&entity=method> > at hudson.remoting.Request$2.run(Request.java:324) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.remoting.Request$2.run&entity=method> > at > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) > > <http://stacktrace.jenkins-ci.org/search/?query=hudson.remoting.InterceptingExecutorService$1.call&entity=method> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > <http://stacktrace.jenkins-ci.org/search/?query=java.util.concurrent.FutureTask.run&entity=method> > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > <http://stacktrace.jenkins-ci.org/search/?query=java.util.concurrent.ThreadPoolExecutor.runWorker&entity=method> > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > <http://stacktrace.jenkins-ci.org/search/?query=java.util.concurrent.ThreadPoolExecutor$Worker.run&entity=method> > at java.lang.Thread.run(Thread.java:745) > <http://stacktrace.jenkins-ci.org/search/?query=java.lang.Thread.run&entity=method> > at ......remote call to <slave host name>(Native Method) > at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) > at hudson.remoting.Channel.call(Channel.java:752) > ... 13 more > Build step 'Execute shell' marked build as failure > Finished: FAILURE > > > This job also succeeds on other slaves. I've tried disconnecting and > reconnecting the slave several times, and verified the slave.jar file > removed and replaced each time. I cleared the .jenkins jar cache on both > the master and slave. A lot of Google searching hasn't turned up anything > directly relavant. I've run out of ideas as to where to look for what else > might have changed. Any suggestions for additional troubleshooting would > be greatly appreciated. > > Thanks, > Ross Oliver > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7d898023-eb47-430d-b823-fe67a9a3e22e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
