This is an automated email from the ASF dual-hosted git repository.
ChenSammi pushed a change to branch HDDS-8342
in repository https://gitbox.apache.org/repos/asf/ozone.git
omit fa19431a184 Revert "HDDS-15467. Do not fall back to the OM starter
user in OMClientRequest (#10469)"
add b7478f1f4f9 HDDS-15786. Intermittent pipeline limit exceeded in debug
tests (#10738)
add 9de98bd4202 HDDS-15726. Improve SCM block deletion summary error
handling (#10650)
add 1b9491ae33e HDDS-15842. Speed up
TestDuplicateContainerDirScannerIntegration (#10740)
add 2c48d33f0c8 HDDS-9154. Design doc for snapshot diff optimization.
(#10335)
add 89a45ea68e0 HDDS-15751. Rename test parent classes for naming
convention (#10713)
add 958465f6e28 HDDS-13472. Shut down datanode if Error occurs while
processing delete blocks command (#10733)
add 87d2bd8577b HDDS-15387. Selectively deserialize KeyInfo/DirectoryInfo
fields for snapshot diff (#10603)
add ff5d03288c4 HDDS-15084. Core Runtime: SCM + OM (#10456)
add 06d972484dc HDDS-15843. Update upgrade/xcompat tests to use 2.1.1
(#10739)
add bf142be6116 HDDS-14661. Handle split table writes for Multipart Upload
(#10588)
add 7d89300dcf7 HDDS-13063. Support splitting json output to multiple
valid json files in a directory (#10604)
add 7a49f649958 HDDS-15794. StreamBlockInputStream.onNext may throw
IndexOutOfBoundsException on short payloads (#10698)
add 8e32bc46964 HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo
(#10181)
add 4bfdb1f7546 HDDS-14957. (addendum) Fix compile error in
TestSnapshotDiffValueParser
add e09eeaa7285 HDDS-15616. ListBuckets API ignores MaxBuckets parameter
and returns all buckets (#10639)
add cb4d088e223 Revert "HDDS-15467. Do not fall back to the OM starter
user in OMClientRequest" (#10756)
add f9e7bc4cffc HDDS-15836. Add Archiver.appendFile for incremental TAR
writes (#10735)
add d2f92864686 HDDS-15851. Return 403 for expired or out-of-range
pre-signed S3 URLs (#10749)
add 575fe4f1cd7 HDDS-15837. Add health-aware container ID listing in SCM
(#10736)
add a8fcb09c023 HDDS-15823. Fix flaky
TestFailureHandlingByClient#testContainerExclusionWithClosedContainerException
(#10721)
add e31f36d2316 HDDS-15667. `containerbalancer status`: show stop reason
and iteration details after stop (#10681)
add 418d07dde59 HDDS-15656. Fix non-atomic data size accumulation in
ContainerBalancer move callback (#10708)
add b18ea9284c4 HDDS-15600. Fix ListObjects response for encoding-type and
empty delimiter (#10586).
new 2ebf64de05c Merge remote-tracking branch 'origin/master' into
HDDS-8342-object-ttl
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (fa19431a184)
\
N -- N -- N refs/heads/HDDS-8342 (2ebf64de05c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../hdds/scm/storage/StreamBlockInputStream.java | 9 +-
.../scm/storage/TestStreamBlockInputStream.java | 156 +++++++++
.../commandhandler/DeleteBlocksCommandHandler.java | 7 +
.../TestDeleteBlocksCommandHandler.java | 108 +++++++
...achine.java => ContainerStateMachineTests.java} | 4 +-
.../ratis/TestContainerStateMachineFollower.java | 2 +-
.../ratis/TestContainerStateMachineLeader.java | 2 +-
...ersAbstract.java => ContainerScannerTests.java} | 2 +-
.../TestBackgroundContainerDataScanner.java | 2 +-
.../TestBackgroundContainerMetadataScanner.java | 2 +-
.../ozoneimpl/TestOnDemandContainerScanner.java | 2 +-
.../docs/content/design/efficient-snapdiff.md | 254 +++++++++++++++
.../{TestCoderBase.java => CoderTests.java} | 2 +-
...estRSRawCoderBase.java => RSRawCoderTests.java} | 4 +-
.../{TestRawCoderBase.java => RawCoderTests.java} | 6 +-
.../erasurecode/rawcoder/TestDummyRawCoder.java | 2 +-
.../erasurecode/rawcoder/TestNativeRSRawCoder.java | 2 +-
.../rawcoder/TestNativeXORRawCoder.java | 2 +-
.../ozone/erasurecode/rawcoder/TestRSRawCoder.java | 2 +-
.../erasurecode/rawcoder/TestXORRawCoder.java | 2 +-
...tXORRawCoderBase.java => XORRawCoderTests.java} | 4 +-
.../org/apache/hadoop/hdds/utils/Archiver.java | 76 +++++
.../org/apache/hadoop/hdds/utils/TestArchiver.java | 40 +++
.../src/main/proto/ScmAdminProtocol.proto | 3 +
.../interface-admin/src/main/resources/proto.lock | 18 ++
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 9 +-
.../SCMDeletedBlockTransactionStatusManager.java | 123 +++++--
.../hdds/scm/container/ContainerManager.java | 4 +-
.../hdds/scm/container/ContainerManagerImpl.java | 5 +-
.../hdds/scm/container/ContainerStateManager.java | 5 +-
.../scm/container/ContainerStateManagerImpl.java | 5 +-
.../scm/container/balancer/ContainerBalancer.java | 19 +-
.../balancer/ContainerBalancerStatusInfo.java | 43 ++-
.../balancer/ContainerBalancerStopReason.java | 63 ++++
.../container/balancer/ContainerBalancerTask.java | 143 +++++---
.../scm/container/states/ContainerStateMap.java | 49 ++-
.../ha/invoker/ContainerStateManagerInvoker.java | 83 ++---
.../hdds/scm/server/SCMClientProtocolServer.java | 15 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 2 +-
.../scm/container/TestContainerStateManager.java | 2 +-
.../container/balancer/TestContainerBalancer.java | 43 +++
.../balancer/TestContainerBalancerStatusInfo.java | 66 ++++
.../balancer/TestContainerBalancerTask.java | 58 ++++
...andler.java => MisReplicationHandlerTests.java} | 2 +-
.../replication/TestECMisReplicationHandler.java | 2 +-
.../TestRatisMisReplicationHandler.java | 2 +-
.../container/states/TestContainerStateMap.java | 102 ++++--
.../scm/cli/ContainerBalancerStatusSubcommand.java | 133 +++++---
.../datanode/TestContainerBalancerSubCommand.java | 242 +++++++++++++-
.../ozone/debug/replicas/ReplicasVerify.java | 112 ++++++-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 71 +++-
.../ozone/om/helpers/OmMultipartAbortInfo.java | 42 ++-
.../ozone/om/helpers/OmMultipartKeyInfo.java | 17 +-
.../ozone/om/helpers/OmMultipartPartInfo.java | 22 ++
.../hadoop/ozone/om/helpers/RepeatedOmKeyInfo.java | 44 ++-
.../dist/src/main/compose/common/replicas-test.sh | 2 +
hadoop-ozone/dist/src/main/compose/testlib.sh | 13 +
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 2 +-
.../dist/src/main/compose/xcompat/clients.yaml | 4 +-
hadoop-ozone/dist/src/main/compose/xcompat/lib.sh | 2 +-
.../src/main/smoketest/balancer/testBalancer.robot | 2 +-
.../hadoop/ozone/s3/awssdk/S3SDKTestUtils.java | 59 ++++
.../ozone/s3/awssdk/v1/AbstractS3SDKV1Tests.java | 169 +++++++++-
.../ozone/s3/awssdk/v2/AbstractS3SDKV2Tests.java | 170 +++++++++-
...eldsBase.java => ConfigurationFieldsTests.java} | 8 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 4 +-
.../client/rpc/TestFailureHandlingByClient.java | 13 +-
...tInputStreamBase.java => InputStreamTests.java} | 2 +-
.../client/rpc/read/TestChunkInputStream.java | 2 +-
.../ozone/client/rpc/read/TestKeyInputStream.java | 2 +-
.../rpc/read/TestStreamBlockInputStream.java | 2 +-
...estDuplicateContainerDirScannerIntegration.java | 1 +
....java => ContainerScannerIntegrationTests.java} | 2 +-
...tBackgroundContainerDataScannerIntegration.java | 2 +-
...kgroundContainerMetadataScannerIntegration.java | 2 +-
.../TestOnDemandContainerScannerIntegration.java | 2 +-
...estDataValidate.java => DataValidateTests.java} | 2 +-
.../freon/TestDataValidateWithDummyContainers.java | 2 +-
.../TestDataValidateWithSafeByteOperations.java | 2 +-
.../TestDataValidateWithUnsafeByteOperations.java | 2 +-
.../ozone/local/TestLocalOzoneClusterRuntime.java | 136 ++++++++
...d.java => OzoneManagerHAFollowerReadTests.java} | 2 +-
...zoneManagerHA.java => OzoneManagerHATests.java} | 2 +-
.../hadoop/ozone/om/TestAddRemoveOzoneManager.java | 2 +-
...stOzoneManagerHAFollowerReadWithAllRunning.java | 2 +-
...OzoneManagerHAFollowerReadWithStoppedNodes.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithAllRunning.java | 2 +-
.../om/TestOzoneManagerHAWithStoppedNodes.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerPrepare.java | 2 +-
.../{TestOmSnapshot.java => OmSnapshotTests.java} | 4 +-
.../snapshot/TestOmSnapshotFsoWithNativeLib.java | 2 +-
.../TestOmSnapshotFsoWithoutNativeLib.java | 2 +-
...apshotFsoWithoutNativeLibWithLinkedBuckets.java | 2 +-
.../om/snapshot/TestOmSnapshotObjectStore.java | 2 +-
.../TestOmSnapshotWithoutBucketLinkingLegacy.java | 2 +-
.../ozone/shell/TestOzoneDebugReplicasVerify.java | 136 ++++++++
.../src/main/proto/OmClientProtocol.proto | 4 +
.../ozone/om/helpers/TestOmKeyInfoCodec.java | 132 +++++++-
.../om/helpers/TestRepeatedOmKeyInfoCodec.java | 109 ++++++-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 36 +++
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 4 +-
.../hadoop/ozone/om/codec/OMDBDefinition.java | 23 +-
.../ozone/om/request/key/OMKeyCommitRequest.java | 3 -
.../om/request/key/OMKeyCommitRequestWithFSO.java | 4 +-
.../S3ExpiredMultipartUploadsAbortRequest.java | 42 ++-
.../multipart/S3MultipartUploadAbortRequest.java | 42 ++-
.../S3MultipartUploadAbortRequestWithFSO.java | 9 +-
.../S3MultipartUploadCommitPartRequest.java | 74 ++++-
.../S3MultipartUploadCommitPartRequestWithFSO.java | 8 +-
.../S3MultipartUploadCompleteRequest.java | 78 +++--
.../S3MultipartUploadCompleteRequestWithFSO.java | 6 +-
.../om/request/util/OMMultipartUploadUtils.java | 102 ++++++
.../AbstractS3MultipartAbortResponse.java | 53 ++-
.../S3ExpiredMultipartUploadsAbortResponse.java | 3 +-
.../multipart/S3MultipartUploadAbortResponse.java | 30 +-
.../S3MultipartUploadAbortResponseWithFSO.java | 14 +-
.../S3MultipartUploadCommitPartResponse.java | 23 +-
...S3MultipartUploadCommitPartResponseWithFSO.java | 8 +-
.../S3MultipartUploadCompleteResponse.java | 15 +-
.../S3MultipartUploadCompleteResponseWithFSO.java | 10 +-
.../ozone/om/snapshot/SnapshotDiffValueParser.java | 359 +++++++++++++++++++++
.../hadoop/ozone/om/TestOmMetadataManager.java | 33 ++
.../om/request/key/TestOMKeyCommitRequest.java | 2 -
.../TestS3MultipartUploadCommitPartRequest.java | 355 ++++++++++++++++++++
.../s3/multipart/S3MultipartResponseTests.java | 10 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 4 +-
.../om/snapshot/TestSnapshotDiffValueParser.java | 249 ++++++++++++++
.../ozone/s3/commontypes/ObjectKeyNameAdapter.java | 3 +-
.../ozone/s3/commontypes/RequestParameters.java | 11 +
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 12 +-
.../ozone/s3/endpoint/ListBucketResponse.java | 13 +-
.../hadoop/ozone/s3/endpoint/RootEndpoint.java | 41 ++-
.../ozone/s3/signature/AWSSignatureProcessor.java | 7 +-
...ser.java => AccessDeniedResourceException.java} | 12 +-
.../s3/signature/AuthorizationV4QueryParser.java | 7 +-
.../org/apache/hadoop/ozone/s3/util/S3Consts.java | 3 +
.../org/apache/hadoop/ozone/s3/util/S3Utils.java | 11 +
.../s3/commontypes/TestObjectKeyNameAdapter.java | 2 +-
.../hadoop/ozone/s3/endpoint/TestBucketList.java | 38 ++-
.../hadoop/ozone/s3/endpoint/TestRootList.java | 94 ++++++
.../s3/signature/TestAWSSignatureProcessor.java | 37 +++
.../signature/TestAuthorizationV4QueryParser.java | 16 +-
hadoop-ozone/tools/pom.xml | 12 +
.../hadoop/ozone/local/LocalOzoneCluster.java | 294 ++++++++++++++++-
.../hadoop/ozone/local/TestLocalOzoneCluster.java | 9 +-
145 files changed, 4996 insertions(+), 486 deletions(-)
rename
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/{TestContainerStateMachine.java
=> ContainerStateMachineTests.java} (99%)
rename
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/{TestContainerScannersAbstract.java
=> ContainerScannerTests.java} (99%)
create mode 100644 hadoop-hdds/docs/content/design/efficient-snapdiff.md
rename
hadoop-hdds/erasurecode/src/test/java/org/apache/ozone/erasurecode/{TestCoderBase.java
=> CoderTests.java} (99%)
rename
hadoop-hdds/erasurecode/src/test/java/org/apache/ozone/erasurecode/rawcoder/{TestRSRawCoderBase.java
=> RSRawCoderTests.java} (97%)
rename
hadoop-hdds/erasurecode/src/test/java/org/apache/ozone/erasurecode/rawcoder/{TestRawCoderBase.java
=> RawCoderTests.java} (98%)
rename
hadoop-hdds/erasurecode/src/test/java/org/apache/ozone/erasurecode/rawcoder/{TestXORRawCoderBase.java
=> XORRawCoderTests.java} (95%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerStopReason.java
rename
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/{TestMisReplicationHandler.java
=> MisReplicationHandlerTests.java} (99%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/conf/{TestConfigurationFieldsBase.java
=> ConfigurationFieldsTests.java} (99%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/read/{TestInputStreamBase.java
=> InputStreamTests.java} (99%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/scanner/{TestContainerScannerIntegrationAbstract.java
=> ContainerScannerIntegrationTests.java} (99%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/{TestDataValidate.java
=> DataValidateTests.java} (99%)
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/local/TestLocalOzoneClusterRuntime.java
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/{TestOzoneManagerHAFollowerRead.java
=> OzoneManagerHAFollowerReadTests.java} (96%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/{TestOzoneManagerHA.java
=> OzoneManagerHATests.java} (94%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/snapshot/{TestOmSnapshot.java
=> OmSnapshotTests.java} (99%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/SnapshotDiffValueParser.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestSnapshotDiffValueParser.java
copy
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/signature/{SignatureParser.java
=> AccessDeniedResourceException.java} (73%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
