[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Change By: Jesse Glick Labels: lazy-loading performance Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Change By: Jesse Glick Labels: lazy-loading Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Title: Message Title Matthew Mitchell commented on JENKINS-23244 Re: Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master (FYI this installation is around 6-7k builds a day) Even in the case of walking newer builds, it seems like this woudl be super expensive. Maybe it's better to keep an index of buildname/number to machine to avoid loading the metadata at all? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Title: Message Title Matthew Mitchell commented on JENKINS-23244 Re: Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master I'm seeing this in our installation. It severely impacts the repsonsiveness of the system. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Title: Message Title Lukasz Karnasiewicz updated an issue Jenkins / JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Steps to reproduce: 1. Display slave builds history page. Wait for it to render, there should be a small progress bar with Computation in progress hint 2. Request any other page (e.g. the main page) - it will hang Sample thread dump illustrating the problem attached. Thread 30745 is processing request for slave builds history (http://jenkins/computer/slave_name/builds) All other requests now hang on jenkins.model.lazy.AbstractLazyLoadRunMap.load for up to 2 minutes in our case. Change By: Lukasz Karnasiewicz Attachment: td.txt Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Title: Message Title Ivan Kalinin reopened an issue We are still experiencing great deal of trouble with slave buld history thing. I just tried to open that for one slave and got all the Jenkins master locked up UI-side. The thread that calls `AbstractLazyLoadRunMap.load` goes on foverer (yes, we have a great deal of builds), but somehow other threads from the UI pool keep getting locked. Eventually, Jenkins became unresponsive altogether – but the jobs were still running. Maybe we could use a separate thread pool for this kind of stuff so it wont lock all the UI threads? BTW, we are running current LTS Jenkins / JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Change By: Ivan Kalinin Resolution: Duplicate Status: Resolved Reopened Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Jesse Glick resolved JENKINS-23244 as Duplicate Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Change By: Jesse Glick (30/Jul/14 11:26 PM) Status: Open Resolved Resolution: Duplicate This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Daniel Beck commented on JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master abayer: What changed? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
abayer commented on JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Nothing - just time after startup and first attempt to load it. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Kohsuke Kawaguchi updated JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Adjusting the priority since it only affects relatively unvisited pages of large deployments. Change By: Kohsuke Kawaguchi (30/May/14 3:13 PM) Priority: Critical Major This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
Kohsuke Kawaguchi commented on JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Looking at the thread dump, the call stack indicates this call resulted in loading all the build records (via AbstractLazyLoadRunMap.all), which looks suspicious. I'd think this operation would only require walking newer build records. "Handling POST /computer/hadoop4/timeline/data/ : http-bio-8090-exec-895" Id=22685 Group=main RUNNABLE at java.io.UnixFileSystem.getBooleanAttributes0(Native Method) at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242) at java.io.File.exists(File.java:813) at hudson.model.RunMap.retrieve(RunMap.java:219) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:687) - locked hudson.model.RunMap@3fa6ce65 at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:649) - locked hudson.model.RunMap@3fa6ce65 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:381) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:219) at hudson.tasks.Fingerprinter$FingerprintAction.compact(Fingerprinter.java:360) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:337) at hudson.model.RunMap.retrieve(RunMap.java:223) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:687) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:670) at jenkins.model.lazy.AbstractLazyLoadRunMap.all(AbstractLazyLoadRunMap.java:622) - locked hudson.model.RunMap@3fa6ce65 at jenkins.model.lazy.AbstractLazyLoadRunMap.entrySet(AbstractLazyLoadRunMap.java:277) at java.util.AbstractMap$2$1.init(AbstractMap.java:378) at java.util.AbstractMap$2.iterator(AbstractMap.java:377) at hudson.util.RunList.iterator(RunList.java:97) at com.google.common.collect.Iterables$15.apply(Iterables.java:1128) at com.google.common.collect.Iterables$15.apply(Iterables.java:1125) at com.google.common.collect.Iterators$8.next(Iterators.java:812) at com.google.common.collect.Iterators$MergingIterator.init(Iterators.java:1306) at com.google.common.collect.Iterators.mergeSorted(Iterators.java:1274) at com.google.common.collect.Iterables$14.iterator(Iterables.java:1113) at com.google.common.collect.Iterables$UnmodifiableIterable.iterator(Iterables.java:94) at com.google.common.collect.Iterables$6.iterator(Iterables.java:585) at hudson.util.RunList$2.iterator(RunList.java:210) at hudson.util.RunList$2.iterator(RunList.java:210) at com.google.common.collect.Iterables$6.iterator(Iterables.java:585) at hudson.util.RunList.iterator(RunList.java:97) at hudson.model.BuildTimelineWidget.doData(BuildTimelineWidget.java:63) This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
abayer commented on JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master fwiw, it's now looking a lot better - no blocked threads, build history's showing up for all slaves now, so far as I can tell. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23244) Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master
abayer created JENKINS-23244 Slave build history page has no data and spawns a ton of very long-lived blocking threads on the master Issue Type: Bug Assignee: Unassigned Components: core Created: 30/May/14 12:11 AM Description: So I went to try to see the usage for a slave on builds.apache.org, and the page had no builds on it. I eventually noticed the "Calculation in progress" bit and thought "Oh, ok, I'll leave this up and check again later". That was a mistake. Now there are 30+ threads on the master like the ones in https://gist.github.com/abayer/88e390e3f0859f8b64e2 - i.e., a whole ton of HTTP POST requests to /computer/foo/timeline/data, all but one blocking on the one that's running, and the one that's running takes a long time to finish. This means (a) that the build history page for a slave is useless and (b) that we're churning CPU/IO and, I'm guessing, doing so repeatedly without caching, since when I check it now, even an hour and a half later, there's no data on the page. Environment: 1.554.1 with JENKINS-22822 backported onto it Project: Jenkins Priority: Critical Reporter: abayer This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
