This is an automated email from the ASF dual-hosted git repository.
jinglun pushed a change to branch HADOOP-19236
in repository https://gitbox.apache.org/repos/asf/hadoop.git
discard 07e387cf7d1 Integration of TOS: Improve the documentation.
discard a1a9b0bd441 Integration of TOS: Rebase trunk.
discard fbf804478c9 Integration of TOS: Fix documents code style.
discard eb86dbe834f Integration of TOS: Not package hadoop-cloud-storage in
hadoop-dist, do it in the future.
discard 115ff7949e7 Integration of TOS: Shade tos dependencies.
discard af476e5786d Integration of TOS: Add documents.
discard 624aaad0cb7 Integration of TOS: Add switch to unit tests that
depending on tos.
discard f567151de39 Integration of TOS: Fix build issues.
discard c55cce745ff Integration of TOS: Fix checkstyle.
discard c458447dc14 Integration of TOS: Add TosFileSystem.
discard 74173e89519 Integration of TOS: Fix ut.
discard 9354e341434 Integration of TOS: Add RawFS
discard cf7dab249a6 Integration of TOS: Add Committer test.
discard f22c31e7385 Integration of TOS: Refactor config keys.
discard 963b70a6bd1 Integration of TOS: Add RawFileSystem unit tests.
discard 61fbddf8b71 Integration of TOS: Add RawFileSystem.
discard be50de2f8ae Integration of TOS: Add test cases of ObjectOutputStream,
MagicOutputStream, FSUtils.
discard 50ce72a1983 Integration of TOS: Add ObjectMultiRangeInputStream and
ObjectRangeInputStream.
discard 4066c26a146 Integration of TOS: Add ObjectOutputStream,
MagicOutputStream.
discard 53756e16188 Integration of TOS: Add tos magic Committer.
discard 3ad651f0264 Integration of TOS: Add FsOps.
discard eee384fa938 Integration of TOS: TOS test cases.
discard eb3c4113de0 Integration of TOS: Add TOS.
discard b61f7228205 Integration of TOS: Add TosKeys and constants.
discard e761031758f Integration of TOS: Add RetryableUtils.
discard 914bd3030ac Integration of TOS: Add TosKeys and ConfKeys.
discard e7ed442fd5b Integration of TOS: Add common Chain.
discard ace2f307712 Integration of TOS: Auth test cases, PrefixStorage and
ConfKeys udpate.
discard 6cde2702a25 Integration of TOS: Volcano Engine Auth.
discard 1047ad98fc7 Integration of TOS: Add lazy reload interface to common.
test
discard a39a6113efb Integration of TOS: Add ObjectUtils UUIDUtils and Range.
discard 2c749888602 Integration of TOS: Add FileStore.
discard 76538e78c94 Integration of TOS: Add ObjectUtils UUIDUtils and Range.
discard b187b81b3b5 Integration of TOS: Add configuration keys.
discard a3370866ccf Integration of TOS: Add DirectoryStorage interface.
discard 0e725bb5cae Integration of TOS: Refactor package name.
discard 8cce6525398 Integration of TOS: Add ObjectStorage interface.
discard 9ff1ed60a2f Integration of TOS: Add lazy reload interface to common.
discard 97e0390b598 Integration of TOS: Add lazy reload interface to common.
discard 480a58c9e50 Integration of TOS: Add common exceptions.
discard cb222f30251 Integration of TOS: Add module hadoop-tos.
add 7543f3aadab HADOOP-19317. S3A: fs.s3a.connection.expect.continue
controls 100 CONTINUE behavior (#7134)
add c2f13cb7621 HADOOP-18325: [ABFS] Fix metric related test failures due
to missing config (#6847)
add cd2cffe73f9 HADOOP-19342. SaslRpcServer.AuthMethod print INFO messages
in client side. (#7174)
add 31b8b4a8bd5 HADOOP-19336: S3A: Test failures after CSE support added
(#7164)
add b9060fc00df YARN-11738 Modernize SecretManager config (#7144)
add 2ab33c672ea HDFS-17671 Suppress callstack when adding a datanode to
deadnodes (#7180)
add 919bd18c06f HDFS-17641. Add badly distributed blocks metric (#7123)
add 65a5bf3b20c HADOOP-19226: [ABFS][FNSOverBlob] Implementing Azure Rest
APIs on Blob Endpoint for AbfsBlobClient (#6944)
add 964e0897d4a HADOOP-19346. ViewFileSystem.InnerCache: Replaced
ReentrantReadWriteLock with ConcurrentHashMap/putIfAbsent() (#7187)
new ed72edfc9af Integration of TOS: Add module hadoop-tos.
new bfa6ebaa943 Integration of TOS: Add common exceptions.
new c165d84ef36 Integration of TOS: Add lazy reload interface to common.
new 5bfc041cca9 Integration of TOS: Add lazy reload interface to common.
new ad155eecb46 Integration of TOS: Add ObjectStorage interface.
new 94fdae4c3c2 Integration of TOS: Refactor package name.
new be9c88557f4 Integration of TOS: Add DirectoryStorage interface.
new 166379eb6e2 Integration of TOS: Add configuration keys.
new d0664d90078 Integration of TOS: Add ObjectUtils UUIDUtils and Range.
new 260d2a5d2ee Integration of TOS: Add FileStore.
new 49fe1b6a22d Integration of TOS: Add ObjectUtils UUIDUtils and Range.
new 72ce7cee092 Integration of TOS: Add lazy reload interface to common.
test
new 194af999940 Integration of TOS: Volcano Engine Auth.
new 63f162a73a7 Integration of TOS: Auth test cases, PrefixStorage and
ConfKeys udpate.
new 0c5bb1f7b46 Integration of TOS: Add common Chain.
new a9c765a6ba7 Integration of TOS: Add TosKeys and ConfKeys.
new 66bf0459dda Integration of TOS: Add RetryableUtils.
new 941ebcb8806 Integration of TOS: Add TosKeys and constants.
new 3a4966f5259 Integration of TOS: Add TOS.
new 42c88327e16 Integration of TOS: TOS test cases.
new 5774f37b1f4 Integration of TOS: Add FsOps.
new 3f6662dd064 Integration of TOS: Add tos magic Committer.
new 30ba5c87e89 Integration of TOS: Add ObjectOutputStream,
MagicOutputStream.
new 12ddf2b16a4 Integration of TOS: Add ObjectMultiRangeInputStream and
ObjectRangeInputStream.
new 43ed890e453 Integration of TOS: Add test cases of ObjectOutputStream,
MagicOutputStream, FSUtils.
new 59b0ea5dfee Integration of TOS: Add RawFileSystem.
new 25879366d0a Integration of TOS: Add RawFileSystem unit tests.
new 9363fe6251c Integration of TOS: Refactor config keys.
new c7cd0ba4d69 Integration of TOS: Add Committer test.
new 45cb0151f93 Integration of TOS: Add RawFS
new efe2e68aa75 Integration of TOS: Fix ut.
new 49a5931863e Integration of TOS: Add TosFileSystem.
new 0c0981ef870 Integration of TOS: Fix checkstyle.
new c1cd76aa103 Integration of TOS: Fix build issues.
new e5ed6d99122 Integration of TOS: Add switch to unit tests that
depending on tos.
new 5f48ed2fab1 Integration of TOS: Add documents.
new 72b31bc7ce8 Integration of TOS: Shade tos dependencies.
new b906a9dbf48 Integration of TOS: Not package hadoop-cloud-storage in
hadoop-dist, do it in the future.
new cdcecc1dd9d Integration of TOS: Fix documents code style.
new 59075697a61 Integration of TOS: Rebase trunk.
new 3538027f3ff Integration of TOS: Improve the documentation.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (07e387cf7d1)
\
N -- N -- N refs/heads/HADOOP-19236 (3538027f3ff)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 41 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../hadoop/fs/CommonConfigurationKeysPublic.java | 9 +
.../apache/hadoop/fs/viewfs/ViewFileSystem.java | 50 +-
.../hadoop/security/SaslMechanismFactory.java | 28 +-
.../org/apache/hadoop/security/SaslRpcServer.java | 15 +-
.../hadoop/security/token/SecretManager.java | 46 +-
.../src/main/resources/core-default.xml | 22 +
.../hadoop-common/src/site/markdown/Metrics.md | 1 +
.../org/apache/hadoop/hdfs/DFSInputStream.java | 6 +-
.../hadoop/hdfs/protocol/ECBlockGroupStats.java | 26 +-
.../hadoop/hdfs/protocol/ReplicatedBlockStats.java | 23 +-
.../hadoop/hdfs/protocolPB/PBHelperClient.java | 13 +-
.../src/main/proto/ClientNamenodeProtocol.proto | 2 +
.../server/federation/metrics/FederationMBean.java | 7 +
.../federation/metrics/NamenodeBeanMetrics.java | 10 +
.../hdfs/server/federation/metrics/RBFMetrics.java | 6 +
.../resolver/MembershipNamenodeResolver.java | 2 +
.../federation/resolver/NamenodeStatusReport.java | 15 +
.../router/NamenodeHeartbeatService.java | 5 +-
.../federation/store/records/MembershipStats.java | 7 +-
.../records/impl/pb/MembershipStatsPBImpl.java | 14 +-
.../src/main/proto/FederationProtocol.proto | 3 +-
.../server/federation/metrics/TestRBFMetrics.java | 4 +
.../hdfs/server/blockmanagement/BlockManager.java | 12 +-
.../blockmanagement/LowRedundancyBlocks.java | 20 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 22 +-
.../hdfs/server/namenode/NameNodeMXBean.java | 8 +
.../namenode/metrics/ReplicatedBlocksMBean.java | 5 +
.../org/apache/hadoop/hdfs/tools/DFSAdmin.java | 6 +-
.../TestLowRedundancyBlockQueues.java | 42 +-
.../namenode/metrics/TestNameNodeMetrics.java | 7 +
.../java/org/apache/hadoop/fs/s3a/Constants.java | 14 +
.../apache/hadoop/fs/s3a/impl/AWSClientConfig.java | 21 +-
.../fs/s3a/impl/EncryptionS3ClientFactory.java | 2 +
.../site/markdown/tools/hadoop-aws/connecting.md | 77 +-
.../site/markdown/tools/hadoop-aws/encryption.md | 216 +++-
.../tools/hadoop-aws/troubleshooting_s3a.md | 158 ---
.../fs/contract/s3a/ITestS3AContractCreate.java | 19 +-
.../fs/s3a/ITestS3AClientSideEncryption.java | 8 +-
.../fs/s3a/ITestS3AClientSideEncryptionCustom.java | 2 -
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 6 +-
.../hadoop/fs/s3a/commit/ITestUploadRecovery.java | 31 +-
.../hadoop/fs/s3a/impl/ITestAwsSdkWorkarounds.java | 1 +
.../fs/s3a/scale/ITestS3AHugeFilesNoMultipart.java | 6 +
.../src/config/checkstyle-suppressions.xml | 4 +
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 2 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 2 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 148 +--
.../fs/azurebfs/constants/AbfsHttpConstants.java | 45 +-
.../fs/azurebfs/constants/FSOperationType.java | 3 +-
.../constants/HttpHeaderConfigurations.java | 31 +-
.../fs/azurebfs/constants/HttpQueryParams.java | 26 +
.../services/AppendRequestParameters.java | 59 ++
.../contracts/services/AzureServiceErrorCode.java | 2 +
...tatus.java => BlobAppendRequestParameters.java} | 29 +-
.../fs/azurebfs/services/AbfsBlobClient.java | 1087 ++++++++++++++++++++
.../hadoop/fs/azurebfs/services/AbfsClient.java | 32 +-
.../fs/azurebfs/services/AbfsClientHandler.java | 62 +-
.../hadoop/fs/azurebfs/services/AbfsDfsClient.java | 2 +
.../fs/azurebfs/services/AbfsRestOperation.java | 4 +
.../azurebfs/services/AbfsRestOperationType.java | 15 +
.../hadoop-azure/src/site/markdown/blobEndpoint.md | 102 ++
.../hadoop-azure/src/site/markdown/fns_blob.md | 8 +-
.../fs/azurebfs/ITestAbfsReadFooterMetrics.java | 19 +
.../ITestAzureBlobFileSystemInitAndCreate.java | 2 +-
.../fs/azurebfs/services/ITestAbfsClient.java | 6 +-
.../azurebfs/services/ITestAbfsClientHandler.java | 49 +
.../azurebfs/services/TestAbfsRestOperation.java | 12 +-
.../security/TestNMTokenSecretManagerInNM.java | 2 +
68 files changed, 2333 insertions(+), 417 deletions(-)
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/{ReadBufferStatus.java
=> BlobAppendRequestParameters.java} (57%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/blobEndpoint.md
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsClientHandler.java
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
