Repository: hadoop Updated Branches: refs/heads/HADOOP-13345 30e1146c5 -> b8ca3bdf3
HADOOP-14548. S3Guard: documenting issues running parallel tests w/ s3n. Contributed by Aaron Fabbri. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/b8ca3bdf Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/b8ca3bdf Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/b8ca3bdf Branch: refs/heads/HADOOP-13345 Commit: b8ca3bdf340622fc5fde8ccad9dc8321d9829add Parents: 30e1146c Author: Sean Mackrory <mackror...@apache.org> Authored: Fri Jul 7 10:38:03 2017 -0600 Committer: Sean Mackrory <mackror...@apache.org> Committed: Fri Jul 7 10:51:16 2017 -0600 ---------------------------------------------------------------------- .../src/site/markdown/tools/hadoop-aws/s3guard.md | 10 ++++++++-- .../src/site/markdown/tools/hadoop-aws/testing.md | 4 ++++ 2 files changed, 12 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/b8ca3bdf/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md ---------------------------------------------------------------------- diff --git a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md index 49eb75f..c28e354 100644 --- a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md +++ b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md @@ -443,7 +443,7 @@ at a time until the operations appear to be working. mvn -T 1C verify -Dtest=skip -Dit.test=ITestS3AMiscOperations -Ds3guard -Ddynamo ``` -Notes +### Notes 1. If the `s3guard` profile is not set, then the s3guard properties are those of the test configuration set in `contract-test-options.xml` or `auth-keys.xml` @@ -452,8 +452,14 @@ If the `s3guard` profile *is* set, 1. The s3guard options from maven (the dynamo and authoritative flags) overwrite any previously set. in the configuration files. 1. Dynamo will be configured to create any missing tables. -1. +### Warning About Concurrent Tests + +You should not run S3A and S3N tests in parallel on the same bucket. This is +especially true when S3Guard is enabled. S3Guard requires that all clients +that are modifying the bucket have S3Guard enabled, so having S3N +integration tests running in parallel with S3A tests will cause strange +failures. ### Scale Testing MetadataStore Directly http://git-wip-us.apache.org/repos/asf/hadoop/blob/b8ca3bdf/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md ---------------------------------------------------------------------- diff --git a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md index 0bf2261..dcc6d46 100644 --- a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md +++ b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md @@ -107,6 +107,10 @@ each filesystem for its testing. 1. `test.fs.s3n.name` : the URL of the bucket for S3n tests 1. `test.fs.s3a.name` : the URL of the bucket for S3a tests +*Note* that running s3a and s3n tests in parallel mode, against the same bucket +is unreliable. We recommend using separate buckets or testing one connector +at a time. + The contents of each bucket will be destroyed during the test process: do not use the bucket for any purpose other than testing. Furthermore, for s3a, all in-progress multi-part uploads to the bucket will be aborted at the --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
