[
https://issues.apache.org/jira/browse/HADOOP-11984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546938#comment-14546938
]
Chris Nauroth commented on HADOOP-11984:
----------------------------------------
We saw {{OutOfMemoryError}} in that hadoop-common run too. We'll need to take
a closer look at how Surefire allocates heap size for multiple concurrent JUnit
processes.
A few of the file system contract tests failed, and I think it's because they
collided on the {{test.build.data}} directory. In hadoop-hdfs, we made a
pom.xml change to parameterize that directory based on the JUnit process
number. That isn't done in the hadoop-common pom.xml, so we'll need to change
it.
{code}
<systemPropertyVariables>
<test.build.data>${test.build.data}/${surefire.forkNumber}</test.build.data>
<hadoop.tmp.dir>${hadoop.tmp.dir}/${surefire.forkNumber}</hadoop.tmp.dir>
</systemPropertyVariables>
{code}
I'm not sure about {{TestZKFailoverControllerStress}} yet, although that one
has been flaky in the past.
bq. Does it make sense to move some of these work towards a Python script so
that it is possible to leverage more libraries?
I didn't have any particular solution in mind, so I think anything is on the
table at this point. If we can find another heuristic for identifying timed
out tests using bash/grep/awk, then I think that's fine. If we can't find a
reliable heuristic, then parsing the XML sounds good.
> Enable parallel JUnit tests in pre-commit.
> ------------------------------------------
>
> Key: HADOOP-11984
> URL: https://issues.apache.org/jira/browse/HADOOP-11984
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-11984.001.patch, HADOOP-11984.002.patch
>
>
> HADOOP-9287 and related issues implemented the parallel-tests Maven profile
> for running JUnit tests in multiple concurrent processes. This issue
> proposes to activate that profile during pre-commit to speed up execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
