See <https://builds.apache.org/job/Qpid-Python-Java-Test/169/changes>
Changes: [robbie] QPID-3911: Fix deadlock on concurrent invocation of MessageConsumer#close() and Session#rollback() from consumer MessageListener This patch contains the following changes: - Add synchronization on AMSession#_messageDeliveryLock into MessageConsumer#close() in order to block until message listener in progress has completed(as required in JMS javadoc for MessageConsumer#close()). - Change the session dispatcher to stop messages delivery into consumer local message queue if the consumer in the process of closing. This eliminates the need to stop the dispatcher on rejecting pending messages for closing consumer. - Remove the synchronization on the dispatcher lock from AMQSession.Dispatcher#rejectPending and code to stop the dispatcher, as we are synchronizing on the deliveryLock now and incoming messages are not dispatched into closing consumers anymore. - Add a system test to reproduce the deadlock and verify its resolution. Applied patch from Oleksandr Rudyy <[email protected]> ------------------------------------------ [...truncated 977 lines...] copy-etc: [copy] Copying 3 files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/etc> postbuild: build: [echo] Running ant for module : broker-plugins/access-control [delete] Deleting: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/test-java-mms.0-10.properties> prepare: [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/src> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/api/broker-plugins/access-control> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/test-api/broker-plugins/access-control> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/results/broker-plugins/access-control> compile_velocity: check_velocity_deps: gen_logging: [echo] logmessages is <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/broker-plugins/access-control/src/main/java/org/apache/qpid/server/security/access/logging/AccessControl_logmessages.properties> [java] Running LogMessage Generator [java] Creating Class for bundle:org.apache.qpid.server.security.access.logging.AccessControl_logmessages [touch] Creating <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/src/velocity.timestamp> precompile: compile: [echo] Targeting : 1.5 [javac] Compiling 11 source files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/classes> [copy] Copying 1 file to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/classes> [copy] Copying <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/broker-plugins/access-control/src/main/java/org/apache/qpid/server/security/access/logging/AccessControl_logmessages.properties> to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/classes/org/apache/qpid/server/security/access/logging/AccessControl_logmessages.properties> jar.manifest: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/plugins/qpid-broker-plugins-access-control-0.17.jar> jar.nomanifest: jar: precompile-tests: compile-tests: [javac] Compiling 3 source files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/access-control/test/classes> jar-tests: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-access-control-tests-0.17.jar> jar-sources: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-access-control-0.17-sources.jar> libs: copy-bin: copy-etc: postbuild: build: [echo] Running ant for module : broker-plugins/experimental/shutdown [delete] Deleting: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/test-java-mms.0-10.properties> prepare: [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/experimental/shutdown/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/experimental/shutdown/src> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/api/broker-plugins/experimental/shutdown> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/test-api/broker-plugins/experimental/shutdown> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/experimental/shutdown/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/results/broker-plugins/experimental/shutdown> precompile: compile: [echo] Targeting : 1.5 [javac] Compiling 3 source files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/experimental/shutdown/classes> [copy] Copying 1 file to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/experimental/shutdown/classes> [copy] Copying <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/broker-plugins/experimental/shutdown/src/main/java/shutdown.bnd> to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/experimental/shutdown/classes/shutdown.bnd> jar.manifest: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/plugins/qpid-broker-plugins-experimental-shutdown-0.17.jar> jar.nomanifest: jar: precompile-tests: compile-tests: jar-tests: [jar] Building MANIFEST-only jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-experimental-shutdown-tests-0.17.jar> jar-sources: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-experimental-shutdown-0.17-sources.jar> libs: copy-bin: copy-etc: postbuild: build: [echo] Running ant for module : broker-plugins/extras [delete] Deleting: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/test-java-mms.0-10.properties> prepare: [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/extras/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/extras/src> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/api/broker-plugins/extras> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/test-api/broker-plugins/extras> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/extras/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/results/broker-plugins/extras> precompile: compile: [echo] Targeting : 1.5 [javac] Compiling 5 source files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/extras/classes> [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. jar.manifest: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/plugins/qpid-broker-plugins-extras-0.17.jar> jar.nomanifest: jar: precompile-tests: compile-tests: [javac] Compiling 1 source file to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/extras/test/classes> jar-tests: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-extras-tests-0.17.jar> jar-sources: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-extras-0.17-sources.jar> libs: copy-bin: copy-etc: postbuild: build: [echo] Running ant for module : broker-plugins/firewall [delete] Deleting: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/test-java-mms.0-10.properties> prepare: [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/firewall/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/firewall/src> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/api/broker-plugins/firewall> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/test-api/broker-plugins/firewall> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/firewall/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/results/broker-plugins/firewall> precompile: compile: [echo] Targeting : 1.5 [javac] Compiling 5 source files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/firewall/classes> jar.manifest: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/plugins/qpid-broker-plugins-firewall-0.17.jar> jar.nomanifest: jar: precompile-tests: compile-tests: [javac] Compiling 2 source files to <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/scratch/broker-plugins/firewall/test/classes> jar-tests: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-firewall-tests-0.17.jar> jar-sources: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-broker-plugins-firewall-0.17-sources.jar> libs: copy-bin: copy-etc: postbuild: build: check-manifest: manifest: [jar] Building jar: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/java/build/lib/qpid-all.jar> BUILD SUCCESSFUL Total time: 1 minute 11 seconds [Qpid-Python-Java-Test] $ /bin/bash -xe /tmp/hudson4451445831123956407.sh + pwd <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/> + cd trunk/qpid/python + ulimit -n 40000 + ulimit -n 1024 + ulimit -n 1024 + ant -file qpid-python-test-ant.xml -Dqpid.executable=../java/build/bin/qpid-server -Dpython.test.ignore=-I../java/test-profiles/python_tests/Java010PythonExcludes '-Dpython.test.modules=-mqpid_tests.broker_0_10 -m qmf.console' -Dqpid.port=25672 '-Dqpid.executable.args=-p 25672 -m 25000' test Buildfile: qpid-python-test-ant.xml clean: [delete] Deleting directory <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/python/build/results> [delete] Deleting directory <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/python/build/work> init: [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/python/build/results> [mkdir] Created dir: <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/python/build/work> ensure-port-free: start-broker: [echo] Starting Qpid with ../java/build/bin/qpid-server -p 25672 -m 25000 run-tests: [echo] Running test-suite [exec] Result: 1 stop-broker: [echo] Stopping Qpid 8067 kill-broker: report: BUILD FAILED <https://builds.apache.org/job/Qpid-Python-Java-Test/ws/trunk/qpid/python/qpid-python-test-ant.xml>:113: Test(s) failed Total time: 15 seconds Build step 'Execute shell' marked build as failure [locks-and-latches] Releasing all the locks [locks-and-latches] All the locks released Recording test results
