[
https://issues.apache.org/jira/browse/HADOOP-11984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-11984:
-----------------------------------
Attachment: HADOOP-11984.007.patch
I'm uploading patch v007. Here are the differences since last time:
# I'm no longer refactoring the parallel-tests profile up to
hadoop-project/pom.xml. YARN and MapReduce aren't ready to inherit this, so it
will have to be duplicated in hadoop-common and hadoop-hdfs for now. We can
refactor later after the corresponding work is done in YARN and MapReduce.
# {{TestConfiguration#testConcurrentAccesses}} is no longer being removed, but
I did lower its footprint by running fewer iterations in the main loop of each
thread. I verified that this is still sufficient to catch the bug it was
written for by temporarily reverting the {{Configuration}} fix that went with
this test.
# Some tests use {{test.build.dir}} instead of {{test.build.data}}, so I set
that up to be parameterized by the JUnit process number.
# To resolve the ZooKeeper-related failures, I introduced
{{test.build.shared.data}}, which is guaranteed to be shared across all of the
forked JUnit processes. {{ClientBaseWithFixes}} had already been set up to use
file locking to prevent port sharing, but since {{test.build.data}} is
parameterized by JUnit process ID now, each process would take its lock files
in a separate directory. That defeats the purpose of file locking. :-) By
moving this back to a shared directory, the file locking works again.
# {{KeyStoreTestUtil}} was using {{target/test-classes}} for putting SSL
configuration files on the classpath. Concurrent tests could collide and mess
up their configurations. Unfortunately, we can't use {{surefire.forkNumber}}
in an {{additionalClasspathElement}} due to Maven limitations. Instead, I've
set it up so that the individual SSL configuration file names can be
parameterized by JUnit process number. Several tests have been updated to set
their configurations accordingly.
> Enable parallel JUnit tests in pre-commit.
> ------------------------------------------
>
> Key: HADOOP-11984
> URL: https://issues.apache.org/jira/browse/HADOOP-11984
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build, scripts, test
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-11984.001.patch, HADOOP-11984.002.patch,
> HADOOP-11984.003.patch, HADOOP-11984.004.patch, HADOOP-11984.005.patch,
> HADOOP-11984.006.patch, HADOOP-11984.007.patch
>
>
> HADOOP-9287 and related issues implemented the parallel-tests Maven profile
> for running JUnit tests in multiple concurrent processes. This issue
> proposes to activate that profile during pre-commit to speed up execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
