[
https://issues.jenkins-ci.org/browse/JENKINS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160396#comment-160396
]
Paul Sokolovsky commented on JENKINS-6604:
------------------------------------------
We started to see these issues today with EC2 build slaves, haven't ever seen
errors like this before for a year we run this infrastructure. The only change
happened before today is that some additional plugins were installed. We're
still on Jenkins 1.419.
> Possible race condition in RemoteClassLoader renders slave unusable
> -------------------------------------------------------------------
>
> Key: JENKINS-6604
> URL: https://issues.jenkins-ci.org/browse/JENKINS-6604
> Project: Jenkins
> Issue Type: Bug
> Components: core
> Affects Versions: current
> Environment: CentOS 5.3, Sun JDK 1.6.0_19 64-bit
> Reporter: michal_grzejszczak
> Priority: Minor
>
> We are restarting hudson each Sunday afternoon to evade problems with memory
> leaks and have a couple of nightly builds that kick in at midnight. The
> scenario is that Hudson is fresh when multiple builds kick in, that is its
> remote class loader did not have a chance to read any classes yet. We have 3
> executors defined. I suppose that the SCM poll action that is sent in many
> build procedures causes multiple requests to load classes for the SCM (we use
> slightly hacked version of CVS SCM). We are getting the following exception:
> java.lang.LinkageError: loader (instance of
> hudson/remoting/RemoteClassLoader): attempted duplicate class definition for
> name: "hudson/model/ModelObject"
> I have looked around on the web and found this
> (http://jira.codehaus.org/browse/JETTY-418) that lead me to believe that lack
> of synchronization while loading classes in remote class loader is the cause.
> Full stack trace:
> {code}
> Started on May 24, 2010 12:00:54 AM
> FATAL: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR
> at hudson.remoting.Channel@1219b8c:slave-81
> hudson.util.IOException2: remote file operation failed:
> /home/hudson-slave/workspace/BPE_8.1SR at
> hudson.remoting.Channel@1219b8c:slave-81
> at hudson.FilePath.act(FilePath.java:743)
> at hudson.FilePath.act(FilePath.java:729)
> at com.syncron.hudson.cvs2.CVS2.isUpdatable(CVS2.java:813)
> at com.syncron.hudson.cvs2.CVS2.pollChanges(CVS2.java:310)
> at hudson.scm.SCM.poll(SCM.java:370)
> at hudson.model.AbstractProject.poll(AbstractProject.java:1153)
> at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:330)
> at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:359)
> at
> hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Remote call on slave-81 failed
> at hudson.remoting.Channel.call(Channel.java:560)
> at hudson.FilePath.act(FilePath.java:736)
> ... 14 more
> Caused by: java.lang.LinkageError: loader (instance of
> hudson/remoting/RemoteClassLoader): attempted duplicate class definition for
> name: "hudson/model/ModelObject"
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:466)
> at
> hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151)
> at
> hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:466)
> at
> hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151)
> at
> hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.getDeclaredFields0(Native Method)
> at java.lang.Class.privateGetDeclaredFields(Class.java:2291)
> at java.lang.Class.getDeclaredField(Class.java:1880)
> at
> java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1610)
> at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:52)
> at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:425)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
> at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310)
> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:547)
> at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1583)
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.UserRequest.deserialize(UserRequest.java:178)
> at hudson.remoting.UserRequest.perform(UserRequest.java:98)
> at hudson.remoting.UserRequest.perform(UserRequest.java:48)
> at hudson.remoting.Request$2.run(Request.java:270)
> ... 6 more
> Done. Took 63 ms
> No changes
> {code}
> If we start single job manually after restart it executes properly. Any
> consecutive jobs will also run fine. However if we get that exception once,
> no other jobs that use the class mentioned in exception (pretty much all)
> will execute anymore until slave is restarted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
