[
https://issues.apache.org/jira/browse/HADOOP-13139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327713#comment-15327713
]
Chris Nauroth commented on HADOOP-13139:
----------------------------------------
{code}
mvn test -Dtest=TestS3A* -Pparallel-tests -DtestsThreadCount=2
{code}
[[email protected]], when you pass a {{-Dtest=}} argument, you are overriding
the {{<includes>}} list passed to Surefire during the build. Passing
{{TestS3A*}} matches all the S3A test suites, including the ones I listed in
{{<excludes>}} to prevent parallel execution. I just tested this, and it looks
like if Surefire finds the same test matching both the whitelist and the
blacklist, then the whitelist wins. Therefore, tests that can't handle
parallel execution end up running in the parallel phase.
Unfortunately, I think this means you can't reliably do both
{{-Pparallel-tests}} and {{-Dtest=TestS3A*}} in the same {{mvn}} run. I had
flagged all multi-part upload tests to run outside the parallel phase because
of the abort-existing-uploads-on-init logic, so I wouldn't expect them to work
well in the parallel phase.
It would be nice to be able to run only S3A tests in parallel for more rapid
iterations on patches that touch only S3A code. I think we'd need to use some
other naming convention on all non-parallel-safe test suite classes to support
that, so that you could specify a wildcard to {{-Dtest=}} that wouldn't match
those suites.
> Branch-2: S3a to use thread pool that blocks clients
> ----------------------------------------------------
>
> Key: HADOOP-13139
> URL: https://issues.apache.org/jira/browse/HADOOP-13139
> Project: Hadoop Common
> Issue Type: Task
> Components: fs/s3
> Affects Versions: 2.8.0
> Reporter: Pieter Reuse
> Assignee: Pieter Reuse
> Attachments: HADOOP-13139-001.patch, HADOOP-13139-branch-2-003.patch,
> HADOOP-13139-branch-2.001.patch, HADOOP-13139-branch-2.002.patch
>
>
> HADOOP-11684 is accepted into trunk, but was not applied to branch-2. I will
> attach a patch applicable to branch-2.
> It should be noted in CHANGES-2.8.0.txt that the config parameter
> 'fs.s3a.threads.core' has been been removed and the behavior of the
> ThreadPool for s3a has been changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
