[
https://issues.jenkins-ci.org/browse/JENKINS-11362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=161386#comment-161386
]
Gustavo Chaves commented on JENKINS-11362:
------------------------------------------
Last week we upgraded our instance to Jenkins 1.458 and confirmed that the
problem persists.
So far we are still managing this by segregating our big maven jobs to an old
Hudson instance in which the problem doesn't happen.
> Strange delay during a maven job build
> --------------------------------------
>
> Key: JENKINS-11362
> URL: https://issues.jenkins-ci.org/browse/JENKINS-11362
> Project: Jenkins
> Issue Type: Bug
> Components: maven2
> Environment: We're running Jenkins on a RHEL 5.2, 32 bits box, with
> Java 1.6.0_04. The build agent runs on a RHEL 5.4, 64 bits box, with Java
> 1.6.0_23.
> Reporter: Gustavo Chaves
> Assignee: kutzi
> Attachments: config.xml, etics-6.1.x-es_76_Console.html,
> fine-stack-traces.zip, JENKINS-11362_stack-traces.zip, m3.log.gz,
> simplificado-config.xml, simplificado.zip, System_Information_Jenkins.html
>
>
> Hi. We upgraded our old Hudson server to Jenkins 1.409.2 on Sep, 20th. Since
> then we're facing bigger build times, like twice the time it used to take
> with Hudson.
> We're running Jenkins on a RHEL 5.2, 32 bits box, with Java 1.6.0_04. The
> build agent runs on a RHEL 5.4, 64 bits box, with Java 1.6.0_23.
> I'm attaching the [^System_Information_Jenkins.html] page which contains the
> configuration of our Jenkins server.
> I'm also attaching the [^config.xml] file of a particular job which shows the
> problem well.
> I'm also attaching the [^etics-6.1.x-es_76_Console.html] file with the
> Console output of a particular build. The output lines are prefixed with the
> time of day (via the timestamper plugin) so that we can see where most of the
> time was spent. As you can see, the build started at 10:47:17 and ended at
> 11:26:36, having taken 39 minutes and 19 seconds in the whole. However, as
> you can see at the end of the log, maven tells us that the build took only 11
> minutes and 3 seconds. Most of the remaining 28 minutes and 16 seconds was
> spent between the lines prefixed with times 10:47:54 and 11:15:21.
> I don't know what the Jenkins server or the agent were doing during this
> quiet period. I tried to see which processes were busy during this period and
> the only strange thing I noticed was that there was a process in the agent
> machine very busy, which was this:
> {noformat}
> /l/disk0/ipscate/tools/java/jdk1.6.0_23/bin/java -jar slave.jar
> {noformat}
> I straced it during a few seconds and saw that it was busy opening lots of
> files under the /l/disk0/ipscate/ip/nexus/.m2/repository/br/com/cpqd
> directory. In fact, in a short period of about 15 or 20 seconds it opened
> more than 40000 files, some of them being repeatedly opened thousands of
> times. These were the most often opened ones:
> {noformat}
> # sort /tmp/trace | uniq -c | sort -n | tail -5
> 3502 32319
> open("/l/disk0/ipscate/ip/nexus/.m2/repository/br/com/cpqd/public/cpqd-public-bom/6.5.2/_maven.repositories",
> O_RDONLY) = 16
> 3502 32319
> stat("/l/disk0/ipscate/ip/nexus/.m2/repository/br/com/cpqd/public/cpqd-public-bom/6.5.2/cpqd-public-bom-6.5.2.pom",
> {st_mode=S_IFREG|0644, st_size=45211, ...}) = 0
> 3793 32319
> open("/l/disk0/ipscate/ip/nexus/.m2/repository/br/com/cpqd/parent/cpqd-super-parent/6.5.2/cpqd-super-parent-6.5.2.pom",
> O_RDONLY) = 16
> 3793 32319
> stat("/l/disk0/ipscate/ip/nexus/.m2/repository/br/com/cpqd/parent/cpqd-super-parent/6.5.2/cpqd-super-parent-6.5.2.pom",
> {st_mode=S_IFREG|0644, st_size=14792, ...}) = 0
> 7004 32319
> stat("/l/disk0/ipscate/ip/nexus/.m2/repository/br/com/cpqd/public/cpqd-public-bom/6.5.2/_maven.repositories",
> {st_mode=S_IFREG|0644, st_size=163, ...}) = 0
> {noformat}
> I'm not sure this behaviour is directly related with the delay I was seeing
> in the build, but it doesn't seem right to me.
> Do you have any suggestion as to how can I go about investigating this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
