[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: Alexander Moiseenko Attachment: thread-dump.txt Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.190959.1527520483000.73.1577359740972%40Atlassian.JIRA.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Raphael Greger commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished Thank you David. I didn't find this configuration but I did some restriction with the log of job config history. Now the problem seems to be vanished. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.190959.1527520483000.6680.1564751701113%40Atlassian.JIRA.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished in the job config there is an option "Abort the build if its log file size is too big" from memory I used the configuration slicer plugin to remove it from all jobs Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.190959.1527520483000.17428.1563763800409%40Atlassian.JIRA.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Raphael Greger commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished Hi David I got exactly the same problem. What was your workaround? Where did you set the log size check? Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.190959.1527520483000.16275.1563526020741%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished seems to have fixed if for us too jenkins has been stable since I disabled the log file size check on all builds Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished We also have a heap of jobs with log file size check on I have disabled to see if it stops happening Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Reassigning acc. to diagnosis by original reporter. Other comments may well have completely unrelated issues. Change By: Jesse Glick Component/s: logfilesizechecker-plugin Component/s: workflow-durable-task-step-plugin Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko stopped work on JENKINS-51568 Change By: Alexander Moiseenko Status: In Progress Open Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title
Alexander Moiseenko edited a comment on JENKINS-51568
Re: Pipeline jobs hanging in Build Executor even if it is finished
I think we've found root cause of a problem. Broken `sleep` and `timeout` methods and executor hanging all relates to jenkins.util.Timer threads. In During hang state we cojvisualvm and watch threads state looks like this :!BusyTimers.png!And in normal case:!NormalCase.PNG! In [^thread-dump.txt] we could find find can see jenkins.util.Timer stack traces, in our case the root cause was logfilesizechecker plugin, [https://github.com/jenkinsci/logfilesizechecker-plugin/blob/master/src/main/java/hudson/plugins/logfilesizechecker/LogfilesizecheckerWrapper.java#L78]that uses timer every second to check log size and this operation creates additional cpu load : {noformat}"jenkins.util.Timer [#9]" - Thread t@267 java.lang.Thread.State: RUNNABLEat java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:655)at java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:682)at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:781)at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1546)at jenkins.model.Nodes.getNode(Nodes.java:295)at jenkins.model.Jenkins.getNode(Jenkins.java:2058)at hudson.model.Computer.getNode(Computer.java:590)at hudson.slaves.SlaveComputer.getNode(SlaveComputer.java:199)at hudson.slaves.SlaveComputer.getNode(SlaveComputer.java:96)at jenkins.model.Jenkins$7.compare(Jenkins.java:1927)at jenkins.model.Jenkins$7.compare(Jenkins.java:1925)at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360)at java.util.TimSort.sort(TimSort.java:234)at java.util.Arrays.sort(Arrays.java:1438)at jenkins.model.Jenkins.getComputers(Jenkins.java:1925)at hudson.model.Executor.of(Executor.java:941)at hudson.model.Run.getExecutor(Run.java:530)at hudson.plugins.logfilesizechecker.LogfilesizecheckerWrapper$LogSizeTimerTask.doRun(LogfilesizecheckerWrapper.java:107)at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers:- locked <4dd03582> (a java.util.concurrent.ThreadPoolExecutor$Worker) {noformat} Our jenkins.util.Timer threads are mostly in park state and sleep works fine, since we've changed DELAY value from 1 second to 10 in [https://github.com/jenkinsci/logfilesizechecker-plugin/blob/master/src/main/java/hudson/plugins/logfilesizechecker/LogfilesizecheckerWrapper.java#L46]
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished I think we've found root cause of a problem. Broken `sleep` and `timeout` methods and executor hanging all relates to jenkins.util.Timer threads. In hang state we cojvisualvm and watch threads state like this: And in normal case: In thread-dump.txt we could find find jenkins.util.Timer stack traces, in our case the root cause was logfilesizechecker plugin, https://github.com/jenkinsci/logfilesizechecker-plugin/blob/master/src/main/java/hudson/plugins/logfilesizechecker/LogfilesizecheckerWrapper.java#L78 that uses timer every second to check log size and this operation creates additional cpu load "jenkins.util.Timer [#9]" - Thread t@267 java.lang.Thread.State: RUNNABLE at java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:655) at java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:682) at java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:781) at java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1546) at jenkins.model.Nodes.getNode(Nodes.java:295) at jenkins.model.Jenkins.getNode(Jenkins.java:2058) at hudson.model.Computer.getNode(Computer.java:590) at hudson.slaves.SlaveComputer.getNode(SlaveComputer.java:199) at hudson.slaves.SlaveComputer.getNode(SlaveComputer.java:96) at jenkins.model.Jenkins$7.compare(Jenkins.java:1927) at jenkins.model.Jenkins$7.compare(Jenkins.java:1925) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:360) at java.util.TimSort.sort(TimSort.java:234) at java.util.Arrays.sort(Arrays.java:1438) at jenkins.model.Jenkins.getComputers(Jenkins.java:1925) at hudson.model.Executor.of(Executor.java:941) at hudson.model.Run.getExecutor(Run.java:530) at hudson.plugins.logfilesizechecker.LogfilesizecheckerWrapper$LogSizeTimerTask.doRun(LogfilesizecheckerWrapper.java:107) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - locked <4dd03582> (a java.util.concurrent.ThreadPoolExecutor$Worker) Our jenkins.util.Timer threads are mostly in park state and sleep works fine, since we've changed DELAY value from 1 second to 10 in https://github.com/jenkinsci/logfilesizechecker-plugin/blob/master/src/main/java/hudson/plugins/logfilesizechecker/LogfilesizecheckerWrapper.java#L46
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: Alexander Moiseenko Attachment: NormalCase.PNG Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: Alexander Moiseenko Attachment: BusyTimers.png Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: David van Laatum Attachment: image-2018-11-27-11-32-58-457.png Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished I seem to have the same problem, so I created a job with the simple sleep 5 pipeline above and setup a cron job to check if the job has failed to finish recently then grab a thread dump and restart jenkins. I also noticed the test job starts taking longer and longer in the sleep step. < /td> Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: David van Laatum Attachment: image-2018-11-27-11-31-06-881.png Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title David van Laatum updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: David van Laatum Attachment: thread_dump.html Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Jesse Glick commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished I am confused by the relationship between your screenshots and the thread dump. build-pipeline-steps.PNG and finished-build.PNG display build #4533. stuck-executor.PNG displays build #4908 running on a one-executor agent jenkins-agent-linux-008, and thread-dump.txt indicates that this agent is currently processing a (Git) checkout. They are not even builds of the same job: one is of KKA/TRIGGER_JOB_NEW_BUILD_IN_NEXUS_FLAG, the other KKA/TRIGGER_JOB_NEW_BUILD_IN_NEXUS. Nothing about this seems improper—you have some running builds, and some completed builds. Maybe I am missing something, or maybe you chose the wrong attachments. I do see one anomalous thing in the thread dump: most of the pool threads for DurableTaskStep retain a Thread.name set in this block even after the block has completed and the thread is parked; WithThreadName ought to be resetting the name reliably, even before the re-schedule call. The code which adds the waiting for JNLP4-connect connection from … suffix to the thread name is here, which also looks like it should be cleaning up properly. I have no explanation for this bug, though I also see no reason to think it is related to your problem. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Sam Van Oort edited a comment on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished [~jglick] Could you please take a look? Appears that the latest comment may reflect a regression due to controller.watch API. Edit: though it's not entirely clear from context Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Sam Van Oort commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished Jesse Glick Could you please take a look? Appears that the latest comment may reflect a regression due to controller.watch API. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title
Alexander Moiseenko commented on JENKINS-51568
Re: Pipeline jobs hanging in Build Executor even if it is finished
Hello. Same problem again, our new env: Jenkins 2.121.1, Pipeline Groovy 2.55 Pipeline: job 2.26 Durable Task Plugin: 1.26 Pipeline Nodes and Processes: 2.22 workaround script doesn't help anymore, after durable task plugin update probably. We've created simple pipeline job with `sleep` step to check if problem have reappered again:
pipeline {
options {
timeout(time: 60, unit: 'SECONDS')
}
stages {
stage('sleep') {
steps {
sleep 5
}
}
}
}
which finishes successfully in normal case and fails when executors gets stuck
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title
Devin Nusbaum edited a comment on JENKINS-51568
Re: Pipeline jobs hanging in Build Executor even if it is finished
[~brainsam] I thought this might be a dupe of JENKINS-45571, but in your case it looks like the stuck executors are full Executors running on build agents rather than the flyweight executors that run on the master, so it seems like it might be something else.What versions of the Pipeline Groovy Plugin, Pipeline Job Plugin, Durable Task Plugin, and the Pipeline Nodes and Processes Plugin are you running? EDIT: Also, how is the {{Linux_default}} agent configured inside of Jenkins (i.e. SSH, EC2, JNLP, etc.)?
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Devin Nusbaum commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished Alexander Moiseenko I thought this might be a dupe of JENKINS-45571, but in your case it looks like the stuck executors are full Executors running on build agents rather than the flyweight executors that run on the master, so it seems like it might be something else. What versions of the Pipeline Groovy Plugin, Pipeline Job Plugin, Durable Task Plugin, and the Pipeline Nodes and Processes Plugin are you running? Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Sam Van Oort commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished Devin Nusbaum Would you be able to take a peek at this please? Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title
Eslam ElHusseiny updated an issue
Jenkins / JENKINS-51568
Pipeline jobs hanging in Build Executor even if it is finished
Change By:
Eslam ElHusseiny
We have huge Jenkins instance, which runs about 20k builds a day.At some moment after couple days without restart of jenkins master, pipeline jobs starts to hang executors after build finish. Freestyle and maven jobs works fine.Busy executor looks like:!stuck-executor.png!But build status is "finished":!finished-build.PNG!!build-pipeline-steps.PNG! There are records about start and finish build in jenkins.log, but executor wasn't released at May 28, 2018 4:35:14 PM:{noformat}May 28, 2018 4:28:07 PM org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxResolvingClassLoader$4$1 loadWARNING: took 5,770ms to load/not load groovy.lang.GroovyObject$groovy$util$script15275137998231310805619$SSH_LOGIN from classLoader hudson.PluginManager$UberClassLoader2018/05/28 05:07:798 - job/KKA/job/TRIGGER_JOB_NEW_BUILD_IN_NEXUS_FLAG/ #4533 Started by timerMay 28, 2018 4:28:07 PM org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxResolvingClassLoader$4$1 loadWARNING: took 6,783ms to load/not load groovy.lang.GroovyObject$groovy$util$script15275138109721310805619$SSH_LOGIN from classLoader hudson.PluginManager$UberClassLoader...WARNING: Owner[PPRBTEAM/archive/Deimos/deimos-module-list-wf/2:PPRBTEAM/archive/Deimos/deimos-module-list-wf #2] was not in the list to begin with: [Owner[GringoTesting/Try/1:GringoTesting/Try #1], Owner[PPRB_DevOps/Install_EIP/Install_EIP_2_cli/211:PPRB_DevOps/Install_EIP/Install_EIP_2_cli #211], Owner[AutoTransaction/AutoTransaction_release_major-2018-04-30_deprecated/275:AutoTransaction/AutoTransaction_release_major-2018-04-30_deprecated #275], Owner[DataFactory/AHD/Pipeline_dev/49:DataFactory/AHD/Pipeline_dev #49], Owner[CSUO/PipelineBadCode/19:CSUO/PipelineBadCode #19], Owner[PPRB_DevOps/Install_EIP/Install_EIP_2_cli/616:PPRB_DevOps/Install_EIP/Install_EIP_2_cli #616], Owner[KKMB/bundle-barriers/barrier-sbof-commondealinf-r1.28.0/7:KKMB/bundle-barriers/barrier-sbof-commondealinf-r1.28.0 #7], Owner[ESB/FS/FS_CI_RFC_PR_8/3060:ESB/FS/FS_CI_RFC_PR_8 #3060], Owner[TFS/TestJobs/integrationOnly/809:TFS/TestJobs/integrationOnly #809], Owner[PPRB_DevOps/KBT/Install_KBT(DEV)_ear/265:PPRB_DevOps/KBT/Install_KBT(DEV)_ear #265], Owner[MBP/mbp-ci/123:MBP/mbp-ci #123], Owner[PRCRED/EKB3/loans-for-persons-ekb3-pipeline___force-WF-and-envelope_Stend1_dev/338:PRCRED/EKB3/loans-for-persons-ekb3-pipeline___force-WF-and-envelope_Stend1_dev #338], Owner[Kalita/tmp/test_pr/46:Kalita/tmp/test_pr #46], Owner[Kalita/tmp/test_pr/47:Kalita/tmp/test_pr #47], Owner[ASKOO/koo-release-build/feature%2F513/8:ASKOO/koo-release-build/feature%2F513 #8], Owner[ASKOO/koo-release-build/develop/117:ASKOO/koo-release-build/develop #117], Owner[ASKOO/koo-release-build/feature%2F513/9:ASKOO/koo-release-build/feature%2F513 #9], Owner[ASKOO/koo-release-build/develop/118:ASKOO/koo-release-build/develop #118],
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Andrew Bayer updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: Andrew Bayer Component/s: workflow-durable-task-step-plugin Component/s: pipeline Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko commented on JENKINS-51568 Re: Pipeline jobs hanging in Build Executor even if it is finished At the same time I see, that sleep step is not working properly too: simple job, that delays for 5 seconds and finish runs for hours: Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko updated an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Change By: Alexander Moiseenko Attachment: sleep-test-script.PNG Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51568) Pipeline jobs hanging in Build Executor even if it is finished
Title: Message Title Alexander Moiseenko created an issue Jenkins / JENKINS-51568 Pipeline jobs hanging in Build Executor even if it is finished Issue Type: Bug Assignee: Unassigned Attachments: build-pipeline-steps.PNG, finished-build.PNG, stuck-executor.png Components: pipeline Created: 2018-05-28 15:14 Environment: jenkins 2.89.4 EL7 Priority: Major Reporter: Alexander Moiseenko We have huge Jenkins instance, which runs about 20k builds a day. At some moment after couple days without restart of jenkins master, pipeline jobs starts to hang executors after build finish. Freestyle and maven jobs works fine. Busy executor looks like: But build status is "finished": There are records about start and finish build in jenkins.log, but executor wasn't released at May 28, 2018 4:35:14 PM: May 28, 2018 4:28:07 PM org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxResolvingClassLoader$4$1 load WARNING: took 5,770ms to load/not load groovy.lang.GroovyObject$groovy$util$script15275137998231310805619$SSH_LOGIN from classLoader hudson.PluginManager$UberClassLoader 2018/05/28 05:07:798 - job/KKA/job/TRIGGER_JOB_NEW_BUILD_IN_NEXUS_FLAG/ #4533 Started by timer May 28, 2018 4:28:07 PM org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxResolvingClassLoader$4$1 load WARNING: took 6,783ms to load/not load groovy.lang.GroovyObject$groovy$util$script15275138109721310805619$SSH_LOGIN
