[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts I was able to reproduce the problem on Windows. The relevant stack trace is: "Handling GET /job/JENKINS-32860/lastSuccessfulBuild/api/json from 172.16.215.1 : RequestHandlerThread[#5]" Id=24 Group=main RUNNABLE (in native) at sun.nio.fs.WindowsNativeDispatcher.FindFirstFile0(Native Method) at sun.nio.fs.WindowsNativeDispatcher.FindFirstFile(Unknown Source) at sun.nio.fs.WindowsLinkSupport.getRealPath(Unknown Source) at sun.nio.fs.WindowsPath.toRealPath(Unknown Source) at sun.nio.fs.WindowsPath.toRealPath(Unknown Source) at jenkins.util.VirtualFile$FileVF.isIllegalSymlink(VirtualFile.java:302) at jenkins.util.VirtualFile$FileVF.length(VirtualFile.java:277) at hudson.model.Run.addArtifacts(Run.java:1109) at hudson.model.Run.addArtifacts(Run.java:1126) at hudson.model.Run.getArtifactsUpTo(Run.java:1081) at hudson.model.Run.getArtifacts(Run.java:1071) The cause is https://github.com/jenkinsci/jenkins/pull/1731 which was introduced in 1.634 and fixed dangerously wrong behavior on Windows. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Bernhard Berbuir commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Here are my results (15.326 Files, 1.297 Folders): Version Duration (seconds) 1.625.2 5,5 1.642.2 48,3 When it is not possible to reproduce this problem on OS X then this is most probable a Windows specific problem. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Cannot reproduce on OS X. 15000 files in 5 folders show in 2.2 seconds in the API, takes about the same time on 1.625.3. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Bernhard Berbuir commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Daniel Beck Yes, i have rolled back my test instance to version 1.625.2 => there I don't have these performance issues. You can reproduce the problem by just copying a huge amount (= several thousand) files to an artifacts directory. Maybe this problem is related to the Windows operation system. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Bernhard Berbuir To clarify, you accessed the same URL for comparable builds in 1.625.x and it was not this slow to respond and CPU intensive? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Bernhard Berbuir commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts I have executed the following URL with Jenkins 1.642.2: https://jenkinsInstance.com/job/build_full_Development_checkin/lastSuccessfulBuild/api/json?tree=artifacts[*].This is the stack trace of the slow thread. The method jenkins.util.VirtualFile$FileVF.isIllegalSymlink(VirtualFile.java:302) uses 64% of the CPU time during execution of the request the request. "Handling GET /job/build_full_Development_checkin/lastSuccessfulBuild/api/json from 10.0.49.126 : RequestHandlerThread8" Id=37 Group=main RUNNABLE at sun.nio.fs.WindowsNativeDispatcher.GetFullPathName0(Native Method) at sun.nio.fs.WindowsNativeDispatcher.GetFullPathName(WindowsNativeDispatcher.java:933) at sun.nio.fs.WindowsLinkSupport.getRealPath(WindowsLinkSupport.java:174) at sun.nio.fs.WindowsPath.toRealPath(WindowsPath.java:836) at sun.nio.fs.WindowsPath.toRealPath(WindowsPath.java:44) at jenkins.util.VirtualFile$FileVF.isIllegalSymlink(VirtualFile.java:302) at jenkins.util.VirtualFile$FileVF.isDirectory(VirtualFile.java:236) at hudson.model.Run.addArtifacts(Run.java:1120) at hudson.model.Run.addArtifacts(Run.java:1126) at hudson.model.Run.addArtifacts(Run.java:1126) at hudson.model.Run.addArtifacts(Run.java:1126) at hudson.model.Run.addArtifacts(Run.java:1126) at hudson.model.Run.addArtifacts(Run.java:1126) at hudson.model.Run.getArtifactsUpTo(Run.java:1081) at hudson.model.Run.getArtifacts(Run.java:1071) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.export.MethodProperty.getValue(MethodProperty.java:66) at org.kohsuke.stapler.export.Property.writeTo(Property.java:116) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:197) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:192) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:192) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:192) at org.kohsuke.stapler.export.Model.writeTo(Model.java:164) at org.kohsuke.stapler.ResponseImpl.serveExposedBean(ResponseImpl.java:267) at hudson.model.Api.doJson(Api.java:211) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:211) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:211) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249) at
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Could you provide thread dumps while Jenkins is slow? It would be interesting to see who calls that method. I wonder why there's a zip() call in the tree of one of the attachments. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck updated an issue Jenkins / JENKINS-32860 Performance issue when retrieving build details of a job run with many artifacts Change By: Daniel Beck Labels: api regression Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Karsten Günther commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Whether it helps or not I would like to report the same behaviour with our Jenkins after updating from 1.625.3 to 1.642.1. We have several Jenkins instances running in my department and didn't recognize any problems after updating some of them. But when updating our productive system with 200 Jobs and 60 slaves we investigated very poor performance and the CPU load of the master was close to 100%. See 2016-02-15_13-15-06_mhtmlfileDtmpnotesCB3F46Last jenkins (as Web Page).mht - Internet Explor.png and 2016-02-15_13-15-19_mhtmlfileDtmpnotesCB3F46Last jenkins (as Web Page).mht - Internet Explor.png for details. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Karsten Günther updated an issue Jenkins / JENKINS-32860 Performance issue when retrieving build details of a job run with many artifacts Change By: Karsten Günther Attachment: 2016-02-15_13-15-19_mhtmlfileDtmpnotesCB3F46Last jenkins (as Web Page).mht - Internet Explor.png Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Karsten Günther updated an issue Jenkins / JENKINS-32860 Performance issue when retrieving build details of a job run with many artifacts Change By: Karsten Günther Attachment: 2016-02-15_13-15-06_mhtmlfileDtmpnotesCB3F46Last jenkins (as Web Page).mht - Internet Explor.png Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Bernhard Berbuir commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Daniel Beck Yes, I'm absolutely sure. I'm running 1.625.3 in production without any performance issues. After upgrading jenkins.war to 1.642.1 I noticed the extremly high CPU load. When isIllegalSymlink has not been changed, then I assume the problem is that isIllegalSymlink is called to often (perhaps it hasn't been called at all in the past when creating a list of artifacts). I have created a pull request against the Hudson Tray Tracker GitHub project with a fix (it uses the tree for selecting only the required xml nodes). The project has been inactive for several years so I'm not sure if a new version will be released. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts `isIllegalSymlink` was first introduced in 1.600/1.596.1 and has been unchanged since then. Weird it should only have an effect since after 1.625.x. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Daniel Beck commented on JENKINS-32860 Re: Performance issue when retrieving build details of a job run with many artifacts Are you absolutely sure you used 1.625.3 before, and it wasn't a problem then? Note also that you're supposed to filter with the tree parameter as documented on /job/jobname/api/xml. It's really quick when using e.g. http://localhost:8080/job/JENKINS-32860/1/api/xml?tree=actions,building,displayName,id,number,queueId,result Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-32860) Performance issue when retrieving build details of a job run with many artifacts
Title: Message Title Bernhard Berbuir created an issue Jenkins / JENKINS-32860 Performance issue when retrieving build details of a job run with many artifacts Issue Type: Bug Assignee: Unassigned Attachments: JenkinsCPUload.png Components: core Created: 09/Feb/16 5:00 PM Environment: Windows 7, Java 1.8.0_71 Labels: api Priority: Major Reporter: Bernhard Berbuir I have some jobs which have lots of artifacts (over 5000 files in over 400 folders). When i retrieve build details of job run via REST api (e.g. https://jenkins/job/JenkinsJob/123/api/xml) then a huge response XML document has to be created. With the last LTS version (1.625.3) this wasn't an issue but after upgrading (1.642.1) I
