[
https://issues.jenkins-ci.org/browse/JENKINS-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=162071#comment-162071
]
SCM/JIRA link daemon commented on JENKINS-13154:
------------------------------------------------
Code changed in jenkins
User: Kohsuke Kawaguchi
Path:
xstream/src/java/com/thoughtworks/xstream/mapper/AnnotationMapper.java
http://jenkins-ci.org/commit/xstream/bc2bd6df986cc54a90a013d014d4b1fa36a6e023
Log:
[JENKINS-13154] removed hot lock contention
annotatedTypes is used to keep track of classes whose annotations we've already
processed. So on a sufficiently warm system, annotatedTypes.contains(type)
should almost always return true. Thus we just need to move it out of the
synchronization block to be able to eliminate the contention. For this, we use
ConcurrentWeakHashMap instead of regular WeakHashMap.
The processTypes method still needs to run in a synchronized block to prevent
concurrent calls of processAnnotation with the same class to serialize their
executions. It is safe to enter this synchronized block after the initial
annotatedTypes.contains(type) check, as the processTypes method itself guards
against repeated processing of the same type.
> Heavy thread congestion with FingerPrint.save
> ---------------------------------------------
>
> Key: JENKINS-13154
> URL: https://issues.jenkins-ci.org/browse/JENKINS-13154
> Project: Jenkins
> Issue Type: Bug
> Components: core
> Affects Versions: current
> Environment: System Properties
> Name ↓ Value
> awt.toolkit sun.awt.X11.XToolkit
> executable-war /foobar/jenkins/jenkins.war
> file.encoding UTF-8
> file.encoding.pkg sun.io
> file.separator /
> guice.disable.misplaced.annotation.check true
> hudson.diyChunking true
> hudson.upstreamCulprits true
> java.awt.graphicsenv sun.awt.X11GraphicsEnvironment
> java.awt.headless true
> java.awt.printerjob sun.print.PSPrinterJob
> java.class.path /foobar/jenkins/jenkins.war
> java.class.version 51.0
> java.endorsed.dirs /foobar/dist/jdk/jdk1.7.0_02/jre/lib/endorsed
> java.ext.dirs
> /foobar/dist/jdk/jdk1.7.0_02/jre/lib/ext:/usr/java/packages/lib/ext
> java.home /foobar/dist/jdk/jdk1.7.0_02/jre
> java.io.tmpdir /foobar/scratch
> java.library.path
> /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> java.runtime.name Java(TM) SE Runtime Environment
> java.runtime.version 1.7.0_02-b13
> java.specification.name Java Platform API Specification
> java.specification.vendor Oracle Corporation
> java.specification.version 1.7
> java.vendor Oracle Corporationa
> java.vendor.url http://java.oracle.com/
> java.vendor.url.bug http://bugreport.sun.com/bugreport/
> java.version 1.7.0_02
> java.vm.info mixed mode
> java.vm.name Java HotSpot(TM) 64-Bit Server VM
> java.vm.specification.name Java Virtual Machine Specification
> java.vm.specification.vendor Oracle Corporation
> java.vm.specification.version 1.7
> java.vm.vendor Oracle Corporation
> java.vm.version 22.0-b10
> jna.platform.library.path /usr/lib64:/lib64:/usr/lib:/lib
> line.separator
> mail.smtp.sendpartial true
> mail.smtps.sendpartial true
> os.arch amd64
> os.name Linux
> os.version 2.6.18-164.15.1.el5
> path.separator :
> securerandom.source file:/dev/./urandom
> sun.arch.data.model 64
> sun.boot.class.path
> /foobar/dist/jdk/jdk1.7.0_02/jre/lib/resources.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/rt.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/sunrsasign.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jsse.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/jce.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/lib/charsets.jar:/foobar/dist/jdk/jdk1.7.0_02/jre/classes
> sun.boot.library.path /foobar/dist/jdk/jdk1.7.0_02/jre/lib/amd64
> sun.cpu.endian little
> sun.cpu.isalist
> sun.io.unicode.encoding UnicodeLittle
> sun.java.command /foobar/jenkins/jenkins.war --httpPort=8088
> sun.java.launcher SUN_STANDARD
> sun.jnu.encoding UTF-8
> sun.management.compiler HotSpot 64-Bit Tiered Compilers
> sun.os.patch.level unknown
> svnkit.http.methods Digest,Basic,NTLM,Negotiate
> svnkit.ssh2.persistent false
> user.country FI
> user.dir /foobar/jenkins
> user.home /foobar/baz
> user.language fi
> user.name baz
> user.timezone Europe/Helsinki
> Environment Variables
> Name ↓ Value
> G_BROKEN_FILENAMES 1
> HISTSIZE 1000
> HOME /foobar/baz
> HUDSON_HOME /foobar/jenkins
> INPUTRC /etc/inputrc
> JAVA_HOME /foobar/dist/jdk/current
> JENKINS_HOME /foobar/jenkins
> JETTYENV_HOME /foobar/baz/env-jetty
> LANG en_US.UTF-8
> LC_ADDRESS fi_FI.UTF-8
> LC_ALL fi_FI.utf8
> LC_CTYPE fi_FI.UTF-8
> LC_IDENTIFICATION fi_FI.UTF-8
> LC_MEASUREMENT fi_FI.UTF-8
> LC_MONETARY fi_FI.UTF-8
> LC_NAME fi_FI.UTF-8
> LC_PAPER fi_FI.UTF-8
> LC_TELEPHONE fi_FI.UTF-8
> LESSOPEN |/usr/bin/lesspipe.sh %s
> LOGNAME baz
> LS_COLORS
> M2_HOME /foobar/dist/maven/current
> MAIL /var/spool/mail/baz
> MAVEN_OPTS -XX:+UseCompressedOops -Djava.security.egd=file:///dev/urandom
> -Xmx512m
> MVN /foobar/dist/maven/current/bin/mvn
> NLSPATH /usr/dt/lib/nls/msg/%L/%N.cat
> PAGER less
> PATH
> /opt/CollabNet_Subversion/bin:/opt/CollabNet_Subversion/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/foobar/baz/env-jetty/bin:/foobar/baz/env-jetty/dist/jdk/current/bin:/foobar/baz/env-fab/bin:/foobar/baz/env-apache/bin:/foobar/baz/bin:/foobar/baz/python/bin:/foobar/dist/jdk/current/bin:/foobar/dist/maven/current/bin:/opt/CollabNet_Subversion/bin/:/foobar/scala-2.9.1.final/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/symcli/bin
> PWD /foobar/jenkins
> READLINK readlink -f
> SHELL /bin/bash
> SHLVL 1
> SSH_TTY /dev/pts/2
> TERM xterm-256color
> TMP /foobar/baz/scratch
> TMPDIR /foobar/baz/scratch
> USER baz
> USE_SSL on
> XFILESEARCHPATH /usr/dt/app-defaults/%L/Dt
> Plugins
> Name ↓ Version Enabled Pinned
> ant 1.1 true false
> javadoc 1.0 true false
> timestamper 1.2.2 true false
> greenballs 1.11 true false
> maven-plugin 1.455 true false
> instant-messaging 1.21 true false
> ircbot 2.18 true false
> join 1.13 true false
> rsync 1.0-SNAPSHOT (private-02/04/2011 11:58-mpe) true false
> PrioritySorter 1.3 false false
> claim 1.7 true false
> radiatorviewplugin 1.13 true false
> build-pipeline-plugin 1.2.3 true false
> BlameSubversion 1.25 true false
> downstream-buildview 1.6 true false
> audit-trail 1.7 true false
> cvs 1.6 true false
> port-allocator 1.5 true false
> disk-usage 0.15 true false
> m2-extra-steps 1.1.7 true false
> subversion 1.34 true false
> ssh-slaves 0.21 true false
> next-build-number 1.0 true false
> email-ext 2.18 true false
> git 1.1.16 true false
> translation 1.8
> Reporter: Teppo Kurki
> Attachments: Thread Dump Analyzer screenshot.jpg,
> threaddump_samples.txt
>
>
> We are seeing repeatable very heavy lock congestion with FingerPrint.save.
> Jenkins apparently slides down to a state where a lot of threads have locked
> / are competing for a lock on FingerPrint instance and are competing for a
> lock on AnnotationMapper (a singleton). Everything grings to a halt.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
