[
https://issues.apache.org/jira/browse/ZOOKEEPER-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated ZOOKEEPER-2183:
-------------------------------------
Attachment: ZOOKEEPER-2183.001.patch
I'm attaching a patch. Here is a summary of the changes:
# build.xml: Add support for concurrent execution of tests. The default is
still a single JUnit process (same as today), but you can override it by
passing a command line argument. This would require everyone to upgrade to Ant
1.9.4. I don't know what version Jenkins runs, so if it fails on this, then
I'll provide another patch without the build.xml change. On my fairly wimpy
CentOS VM, I can do a full test run with {{ant -Dtest.junit.threads=8
-Dtest.output=yes clean test-core-java}} in ~10-15 minutes, down from ~60
minutes.
# {{PortAssignment}}: I rewrote this to assign ports by binding to an ephemeral
port temporarily. I documented that there is still a very small race condition
risk, but it's going to be much better than the current situation where
multiple JUnit processes starting from the same base port number could collide.
I did explore trying to bootstrap test servers with port 0 to eliminate this
race condition completely, but that would have required intrusive changes in
product code and tests that expect to know all their port numbers upfront
before running anything.
# Multiple test suites are updated to skip starting {{JettyAdminServer}}, so
that we don't get a {{BindException}} when they all try to use the same port.
All test suites that inherit from {{ZKTestCase}} already do this.
# Multiple tests are updated to remove "+ 1000" logic from their port
assignments. This looks like it was an attempt to offset port numbers a bit in
certain tests to try to increase the probability of uniqueness. This is no
longer necessary with the new {{PortAssignment}}. Also, the logic is
incorrect, because it could inadvertently cause a port collision or overflow
past the valid range for port numbers.
I've tried multiple concurrent test runs with this patch, and there are no
signs of a {{BindException}}, so I think this covers the changes we need in
port assignments.
> Change test port assignments to improve uniqueness of ports for multiple
> concurrent test processes on the same host.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-2183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2183
> Project: ZooKeeper
> Issue Type: Improvement
> Components: tests
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: ZOOKEEPER-2183.001.patch
>
>
> Tests use {{PortAssignment#unique}} for assignment of the ports to bind
> during tests. Currently, this method works by using a monotonically
> increasing counter from a static starting point. Generally, this is
> sufficient to achieve uniqueness within a single JVM process, but it does not
> achieve uniqueness across multiple processes on the same host. This can
> cause tests to get bind errors if there are multiple pre-commit jobs running
> concurrently on the same Jenkins host. This also prevents running tests in
> parallel to improve the speed of pre-commit runs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
