[
https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848361#comment-17848361
]
ASF GitHub Bot commented on HADOOP-18508:
-----------------------------------------
mukund-thakur commented on code in PR #5081:
URL: https://github.com/apache/hadoop/pull/5081#discussion_r1608853482
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md:
##########
@@ -539,12 +539,51 @@ Otherwise, set a large timeout in
`fs.s3a.scale.test.timeout`
The tests are executed in an order to only clean up created files after
the end of all the tests. If the tests are interrupted, the test data will
remain.
+## <a name="CI"/> Testing through continuous integration
+
+### Parallel CI builds.
+For CI testing of the module, including the integration tests,
+it is generally necessary to support testing multiple PRs simultaneously.
+
+To do this
+1. A job ID must be supplied in the `job.id` property, so each job works on an
isolated directory
+ tree. This should be a number or unique string, which will be used within a
path element, so
+ must only contain characters valid in an S3/hadoop path element.
+2. Root directory tests need to be disabled by setting
`fs.s3a.root.tests.enabled` to
+ `false`, either in the command line to maven or in the XML configurations.
+
+```
+mvn verify -T 1C -Dparallel-tests -DtestsThreadCount=14 -Dscale
-Dfs.s3a.root.tests.enabled=false -Djob.id=001
+```
+
+This parallel execution feature is only for isolated builds sharing a single
S3 bucket; it does
+not support parallel builds and tests from the same local source tree.
Review Comment:
ah..already mentioned here so we are good.
> support multiple s3a integration test runs on same bucket in parallel
> ---------------------------------------------------------------------
>
> Key: HADOOP-18508
> URL: https://issues.apache.org/jira/browse/HADOOP-18508
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.9
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Labels: pull-request-available
>
> to have (internal, sorry) jenkins test runs work in parallel, they need to
> share the same bucket so
> # must have a prefix for job id which is passed in to the path used for forks
> # support disabling root tests so they don't stamp on each other
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
