[
https://issues.jenkins-ci.org/browse/JENKINS-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=162075#comment-162075
]
SCM/JIRA link daemon commented on JENKINS-13154:
------------------------------------------------
Code changed in jenkins
User: Kohsuke Kawaguchi
Path:
xstream/src/java/com/thoughtworks/xstream/mapper/AnnotationMapper.java
http://jenkins-ci.org/commit/xstream/0903e75b981bc3ee91995890de91b800aebe11e6
Log:
[JENKINS-13154] No, the previous fix was incomplete.
The type gets added to annotatedTypes before its annotations are actually
processed, so it is possible for one thread to visit those annotations while
another thread comes in and finds it already, then carry on even though the
first thread hasn't finished visiting annotations.
So I added a second WeakHashSet to guard against this problem.
The serializedClass field also needs to be concurrent-safe, as we don't seem to
lock access to it
> Heavy thread congestion with FingerPrint.save
> ---------------------------------------------
>
> Key: JENKINS-13154
> URL: https://issues.jenkins-ci.org/browse/JENKINS-13154
> Project: Jenkins
> Issue Type: Bug
> Components: core
> Affects Versions: current
> Environment: System Properties
> Name ↓ Value
> awt.toolkit sun.awt.X11.XToolkit
> executable-war /foobar/jenkins/jenkins.war
> file.encoding UTF-8
> file.encoding.pkg sun.io
> file.separator /
> guice.disable.misplaced.annotation.check true
> hudson.diyChunking true
> hudson.upstreamCulprits true
> java.awt.graphicsenv sun.awt.X11GraphicsEnvironment
> java.awt.headless true
> java.awt.printerjob sun.print.PSPrinterJob
> java.class.path /foobar/jenkins/jenkins.war
> java.class.version 51.0
> java.endorsed.dirs /foobar/dist/jdk/jdk1.7.0_02/jre/lib/endorsed
> java.ext.dirs
> /foobar/dist/jdk/jdk1.7.0_02/jre/lib/ext:/usr/java/packages/lib/ext
> java.home /foobar/dist/jdk/jdk1.7.0_02/jre
> java.io.tmpdir /foobar/scratch
> java.library.path
> /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> java.runtime.name Java(TM) SE Runtime Environment
> java.runtime.version 1.7.0_02-b13
> java.specification.name Java Platform API Specification
> java.specification.vendor Oracle Corporation
> java.specification.version 1.7
> java.vendor Oracle Corporationa
> java.vendor.url http://java.oracle.com/
> java.vendor.url.bug http://bugreport.sun.com/bugreport/
> java.version 1.7.0_02
> java.vm.info mixed mode
> java.vm.name Java HotSpot(TM) 64-Bit Server VM
> java.vm.specification.name Java Virtual Machine Specification
> java.vm.specification.vendor Oracle Corporation
> java.vm.specification.version 1.7
> java.vm.vendor Oracle Corporation
> java.vm.version 22.0-b10
> jna.platform.library.path /usr/lib64:/lib64:/usr/lib:/lib
> line.separator
> mail.smtp.sendpartial true
> mail.smtps.sendpartial true
> os.arch amd64
> os.name Linux
> os.version 2.6.18-164.15.1.el5
> path.separator :
> securerandom.source file:/dev/./urandom
> sun.arch.data.model 64
> sun.boot.class.path
> /foobar/dist/jdk/jdk1.7.0_02/jre/lib/resources.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/rt.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/sunrsasign.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jsse.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jce.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/charsets.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/classes
> sun.boot.library.path /foobar/dist/jdk/jdk1.7.0_02/jre/lib/amd64
> sun.cpu.endian little
> sun.cpu.isalist
> sun.io.unicode.encoding UnicodeLittle
> sun.java.command /foobar/jenkins/jenkins.war --httpPort=8088
> sun.java.launcher SUN_STANDARD
> sun.jnu.encoding UTF-8
> sun.management.compiler HotSpot 64-Bit Tiered Compilers
> sun.os.patch.level unknown
> svnkit.http.methods Digest,Basic,NTLM,Negotiate
> svnkit.ssh2.persistent false
> user.country FI
> user.dir /foobar/jenkins
> user.home /foobar/baz
> user.language fi
> user.name baz
> user.timezone Europe/Helsinki
> Environment Variables
> Name ↓ Value
> G_BROKEN_FILENAMES 1
> HISTSIZE 1000
> HOME /foobar/baz
> HUDSON_HOME /foobar/jenkins
> INPUTRC /etc/inputrc
> JAVA_HOME /foobar/dist/jdk/current
> JENKINS_HOME /foobar/jenkins
> JETTYENV_HOME /foobar/baz/env-jetty
> LANG en_US.UTF-8
> LC_ADDRESS fi_FI.UTF-8
> LC_ALL fi_FI.utf8
> LC_CTYPE fi_FI.UTF-8
> LC_IDENTIFICATION fi_FI.UTF-8
> LC_MEASUREMENT fi_FI.UTF-8
> LC_MONETARY fi_FI.UTF-8
> LC_NAME fi_FI.UTF-8
> LC_PAPER fi_FI.UTF-8
> LC_TELEPHONE fi_FI.UTF-8
> LESSOPEN |/usr/bin/lesspipe.sh %s
> LOGNAME baz
> LS_COLORS
> M2_HOME /foobar/dist/maven/current
> MAIL /var/spool/mail/baz
> MAVEN_OPTS -XX:+UseCompressedOops -Djava.security.egd=file:///dev/urandom
> -Xmx512m
> MVN /foobar/dist/maven/current/bin/mvn
> NLSPATH /usr/dt/lib/nls/msg/%L/%N.cat
> PAGER less
> PATH
> /opt/CollabNet_Subversion/bin:/opt/CollabNet_Subversion/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/foobar/baz/env-jetty/bin:/foobar/baz/env-jetty/dist/jdk/current/bin:/foobar/baz/env-fab/bin:/foobar/baz/env-apache/bin:/foobar/baz/bin:/foobar/baz/python/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/opt/CollabNet_Subversion/bin/:/foobar/scala-2.9.1.final/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/symcli/bin
> PWD /foobar/jenkins
> READLINK readlink -f
> SHELL /bin/bash
> SHLVL 1
> SSH_TTY /dev/pts/2
> TERM xterm-256color
> TMP /foobar/baz/scratch
> TMPDIR /foobar/baz/scratch
> USER baz
> USE_SSL on
> XFILESEARCHPATH /usr/dt/app-defaults/%L/Dt
> Plugins
> Name ↓ Version Enabled Pinned
> ant 1.1 true false
> javadoc 1.0 true false
> timestamper 1.2.2 true false
> greenballs 1.11 true false
> maven-plugin 1.455 true false
> instant-messaging 1.21 true false
> ircbot 2.18 true false
> join 1.13 true false
> rsync 1.0-SNAPSHOT (private-02/04/2011 11:58-mpe) true false
> PrioritySorter 1.3 false false
> claim 1.7 true false
> radiatorviewplugin 1.13 true false
> build-pipeline-plugin 1.2.3 true false
> BlameSubversion 1.25 true false
> downstream-buildview 1.6 true false
> audit-trail 1.7 true false
> cvs 1.6 true false
> port-allocator 1.5 true false
> disk-usage 0.15 true false
> m2-extra-steps 1.1.7 true false
> subversion 1.34 true false
> ssh-slaves 0.21 true false
> next-build-number 1.0 true false
> email-ext 2.18 true false
> git 1.1.16 true false
> translation 1.8
> Reporter: Teppo Kurki
> Attachments: Thread Dump Analyzer screenshot.jpg,
> threaddump_samples.txt
>
>
> We are seeing repeatable very heavy lock congestion with FingerPrint.save.
> Jenkins apparently slides down to a state where a lot of threads have locked
> / are competing for a lock on FingerPrint instance and are competing for a
> lock on AnnotationMapper (a singleton). Everything grings to a halt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
