[
https://issues.apache.org/jira/browse/ZOOKEEPER-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14545826#comment-14545826
]
Chris Nauroth commented on ZOOKEEPER-2183:
------------------------------------------
Hi [~rakeshr].
Unfortunately, the times on the Jenkins /testReport URLs won't be an accurate
indicator of wall clock time. The Jenkins report works by parsing the JUnit
output for the lines that show the elapsed time of each test suite. This goes
into the Duration column for each test suite. Then, at the package level,
Duration is calculated as the sum of all suite durations in the package.
Finally, the total time for the whole test run is calculated as the sum of all.
Jenkins is not aware that the test suites were distributed across multiple
JUnit processes, so each of these sum operations accidentally double-counts the
actual clock time. (Actually, it can be as much as 8x-counts the clock time
when you consider we chose 8 concurrent processes.)
For example, assuming 2 test suites that each need 1 minute to run, and we use
2 JUnit processes, Jenkins will report the time as 2 minutes, even though you
and I looking at a wall clock know it finished in 1 minute.
You're correct that looking at the overall job execution time still shows the
benefit of this change, 13 minutes in your example. That's because this is a
raw measurement of the Jenkins job execution time.
> Concurrent Testing Processes and Port Assignments
> -------------------------------------------------
>
> Key: ZOOKEEPER-2183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2183
> Project: ZooKeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.5.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Fix For: 3.5.1, 3.6.0
>
> Attachments: ZOOKEEPER-2183.001.patch, ZOOKEEPER-2183.002.patch,
> ZOOKEEPER-2183.003.patch, ZOOKEEPER-2183.004.patch, ZOOKEEPER-2183.005.patch,
> threads-change.patch
>
>
> Tests use {{PortAssignment#unique}} for assignment of the ports to bind
> during tests. Currently, this method works by using a monotonically
> increasing counter from a static starting point. Generally, this is
> sufficient to achieve uniqueness within a single JVM process, but it does not
> achieve uniqueness across multiple processes on the same host. This can
> cause tests to get bind errors if there are multiple pre-commit jobs running
> concurrently on the same Jenkins host. This also prevents running tests in
> parallel to improve the speed of pre-commit runs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
