This is an automated email from the ASF dual-hosted git repository.
stevel pushed a change to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from c48c774 HADOOP-17397. ABFS: SAS Test updates for version and
permission update (#2492)
new 1eeb9d9 HADOOP-17318. Support concurrent S3A commit jobs with same
app attempt ID. (#2399)
new 1e59bf7 HADOOP-17385. ITestS3ADeleteCost.testDirMarkersFileCreation
failure (#2473).
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/main/resources/core-default.xml | 13 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 18 +-
.../apache/hadoop/fs/s3a/WriteOperationHelper.java | 9 +-
.../hadoop/fs/s3a/commit/AbstractS3ACommitter.java | 376 ++++++++++++++++++---
.../hadoop/fs/s3a/commit/CommitConstants.java | 68 +++-
.../hadoop/fs/s3a/commit/CommitOperations.java | 17 +-
.../hadoop/fs/s3a/commit/CommitUtilsWithMR.java | 45 +--
.../fs/s3a/commit/InternalCommitterConstants.java | 31 +-
.../hadoop/fs/s3a/commit/files/PendingSet.java | 29 +-
.../fs/s3a/commit/files/PersistentCommitData.java | 2 +-
.../fs/s3a/commit/files/SinglePendingCommit.java | 2 +-
.../hadoop/fs/s3a/commit/files/SuccessData.java | 27 +-
.../fs/s3a/commit/magic/MagicS3GuardCommitter.java | 26 +-
.../staging/PartitionedStagingCommitter.java | 5 +-
.../apache/hadoop/fs/s3a/commit/staging/Paths.java | 16 +-
.../fs/s3a/commit/staging/StagingCommitter.java | 94 +-----
.../commit/staging/StagingCommitterConstants.java | 15 -
.../site/markdown/tools/hadoop-aws/committers.md | 314 ++++++++++++++---
.../tools/hadoop-aws/troubleshooting_s3a.md | 61 +++-
.../apache/hadoop/fs/contract/s3a/S3AContract.java | 4 +
.../apache/hadoop/fs/s3a/AbstractS3ATestBase.java | 3 +
.../hadoop/fs/s3a/commit/AbstractCommitITest.java | 17 +-
.../fs/s3a/commit/AbstractITCommitProtocol.java | 301 ++++++++++++++++-
.../fs/s3a/commit/LoggingTextOutputFormat.java | 2 +-
.../commit/integration/ITestS3ACommitterMRJob.java | 7 +-
.../s3a/commit/magic/ITestMagicCommitProtocol.java | 43 ++-
.../fs/s3a/commit/staging/StagingTestBase.java | 4 +-
.../s3a/commit/staging/TestStagingCommitter.java | 147 ++++++--
.../staging/TestStagingPartitionedJobCommit.java | 2 +-
.../integration/ITestStagingCommitProtocol.java | 14 +-
.../fs/s3a/commit/terasort/ITestTerasortOnS3A.java | 2 +-
.../fs/s3a/performance/AbstractS3ACostTest.java | 9 +
.../fs/s3a/performance/ITestS3ADeleteCost.java | 1 +
.../fs/s3a/scale/AbstractSTestS3AHugeFiles.java | 2 +-
34 files changed, 1421 insertions(+), 305 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
