[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev commented on JENKINS-54757 Re: High CPU caused by dumping the classloader in CpsFlowExecution Since my last comment: 1. Downgrading the DSL plugin does not solve the issue 2. The volume of items that DSL plugin generates does impact the performance (more items -> faster regression per run). I've minimized the issue for me by building a custom seed that generates only a fraction of all 1000 items, based on the changed files. Jenkins is running 4 days straight now and I'm not seeing anything like the CPU numbers I had. This is not a real solution, but it significantly increases the lifespan of the system. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev commented on JENKINS-54757 Re: High CPU caused by dumping the classloader in CpsFlowExecution Attaching a build trend screenshot, I'm reproducing this successfully with the following scenario: 1) Create 2 jobs, one should be executing a content generation using the DSL plugin. Other should be empty pipeline. Schedule both of them to execute recurrently every minute. 2) The empty pipeline will initially run for < 1 second. It will start regressing with time, in a couple of hours it will go over a minute execution time. 3) Stopping the generation job will stop the regression.As you can see in the screenshot, I stopped it at build #719. I've not confirmed if the item count that the generation job creates maters in this case (currently, my seed generates over 1000 items). I'll try downgrading the DSL plugin to 1.69 as this was the previous known stable version for me and go from there. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev updated an issue Jenkins / JENKINS-54757 High CPU caused by dumping the classloader in CpsFlowExecution Change By: Grigor Lechev Attachment: trend.png Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev commented on JENKINS-54757 Re: High CPU caused by dumping the classloader in CpsFlowExecution I've attached a couple of screenshots from JVM and a thread dump. This is after removing the log line at 1345. The CPU consumption is lower than before, but it's still pretty high. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev updated an issue Jenkins / JENKINS-54757 High CPU caused by dumping the classloader in CpsFlowExecution Change By: Grigor Lechev Attachment: memory-sampling.png Attachment: cpu-sampling.png Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev updated an issue Jenkins / JENKINS-54757 High CPU caused by dumping the classloader in CpsFlowExecution Change By: Grigor Lechev Attachment: tdump-23-11.txt Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Ben Herfurth edited a comment on JENKINS-54757 Re: High CPU caused by dumping the classloader in CpsFlowExecution Encountering the same issue here. Jenkins 2.138.3 Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Ben Herfurth commented on JENKINS-54757 Re: High CPU caused by dumping the classloader in CpsFlowExecution Encountering the same issue here. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-54757) High CPU caused by dumping the classloader in CpsFlowExecution
Title: Message Title Grigor Lechev created an issue Jenkins / JENKINS-54757 High CPU caused by dumping the classloader in CpsFlowExecution Issue Type: Bug Assignee: Unassigned Components: workflow-cps-plugin Created: 2018-11-21 13:58 Environment: Jenkins 2.144, Workflow CPS Plugin 2.60, Job DSL Plugin 1.70 Priority: Major Reporter: Grigor Lechev I have a pretty big system, 1000+ Jenkins items (mostly pipelines), 50+ slaves at all times.I'm investigating some high CPU issues after the upgrade to 2.144 (I've upgraded the plugins as well, so I doubt it's really Jenkins core). What caught my eye is the following stacktrace (a few variations of it): java.lang.Thread.State: RUNNABLE at java.lang.String.valueOf(String.java:2994) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpGlobalClassValue(CpsFlowExecution.java:1345) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpLoader(CpsFlowExecution.java:1291) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpHeap(CpsFlowExecution.java:1265) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:375) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266)
