This is an automated email from the ASF dual-hosted git repository.
siyao pushed a change to branch HDDS-4944
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from bb67e13 HDDS-6004. Use changes from HDDS-5881 for volume chroot.
(#2924)
add 5dd1144 HDDS-6066. Cleanup OmService injection. (#2896)
add 0bc100a HDDS-6090. Collect output.log from acceptance tests (#2908)
add 8587de7 HDDS-6096. S3 copy object fails for source key with special
char (#2912)
add dfcc69c HDDS-6018 Closing state to quasiclosed (#2854)
add 09b06eb HDDS-6104. Update log4j version to 2.16.0 version. (#2920)
add 4d1b5da HDDS-6053. Fix too short container scrubber data scan
interval. (#2875)
add e1bef2c HDDS-6080. Simplify the logic in
SimpleContainerDownloader.getContainerDataFromReplicas (#2903)
add 47cd204 HDDS-6107. Exclude hugo lock file from source tarball (#2923)
add c4579ec HDDS-6089. Remove OMClientResponse#getBucketLayout override
in subclasses - part2 (#2915)
add b75ec9d HDDS-6072. Fix increased integration test execution time
(#2900)
add 56d0ebb HDDS-6105. remove db cache size config (#2922)
add af4730a HDDS-5912. Add description of hdds.datanode.dir in Getting
Started (#2783)
add 3fa3f36 HDDS-6110. Add hugo lock file to gitignore (#2928)
add 4770eab HDDS-5795. [FSO] Add unit test case with mpu open key during
abort. (#2931)
add fe7f5b4 HDDS-5490. Remove (Decommission) an OM node from HA ring
(#2886)
add 8ceb6cb HDDS-6124. Update log4j version to 2.17.0 version (#2933)
add 69e8617 HDDS-6125. Support Hugo 0.91.0. (#2934)
add e01b471 HDDS-5602. make it configurable to choose the nearest one as
the target in the candidates according to networkTopology (#2756)
add 26a9cdb HDDS-6101. use FileUtils.moveDirectory instead of Files.move
when installing snapshot (#2918)
add 6cd9211 HDDS-5975. Serve url-encoded key and prefix name for
ListObjectResponse (#2834)
add 9dc63a3 HDDS-5807. [FSO] Merge HDDS-4653 changes to support TDE for
MPU Keys in FSO bucket layout. (#2941)
add 97143b8 HDDS-6129. OM has thread unsafe issues that can cause OM
shutdown. (#2938)
add b570d0a HDDS-6070. ContainerBalancerConfig doesn't read config from
ozone-site.xml (#2893)
add 4ed3ef8 HDDS-6121. DatanodeAdminMonitor should log detailed
information for a limited number of containers (#2932)
add 9103aa0 HDDS-5993. Disable flaky
TestRandomKeyGenerator#cleanObjectsTest
add a5cc886 HDDS-5319 Intermittent failure in
TestSCMUpdateServiceGrpcServer (#2558)
add d6cbb15 HDDS-6143. Update log4j version to 2.17.1 (#2952)
add b5c5853 HDDS-6142. Remove unused dependency: jdom (#2951)
add cb59f01 HDDS-6119. Reset the property name to configure the datanode
kerberos keytab (#2949)
add e6179c1 HDDS-6118. Documentation missing settings to secure S3g with
Kerberos (#2950)
add 144e982 HDDS-6108. Add slf4j and log4j.properties for tests in
hadoop-ozone/client (#2925)
add b7496c9 HDDS-6146. TestDefaultCAServer#testIntermediaryCA failure
(#2955)
add 4d3f8ba HDDS-6103. Avoid refresh pipeline for key override in
Key/File Create. (#2926)
add 644ef95 HDDS-6145. Include docker-compose output in acceptance test
output.log (#2958)
add 30cb5e7 HDDS-6123. Disable system exit in integration tests (#2959)
add 1ea978a HDDS-6149. Remove unused keytabs (#2960)
add 243cc3f HDDS-6094. Some unit tests are skipped due to using JUnit4
runner (#2909)
add 61f1c70 HDDS-6075. OzoneConfiguration constructor overrides input
configuration keys. (#2921)
add ca25193 HDDS-4177. SCM Container DB bootstrap on Recon startup (#2942)
add 9644f83c HDDS-6086. Compute MD5MD5CRC file checksum using chunk
checksums from DataNodes (#2919)
add a31b79a HDDS-6148. Validate ContainerBalancerConfiguration before
start ContainerBalancer (#2957)
add c8f6ad0 HDDS-6161. SCM StateMachine failing to reinitialize doesn't
terminate the process. (#2971)
add a348980 HDDS-6134. Move replication-specific config to
ReplicationServer (#2943)
add 5200eab HDDS-4010. S3G startup fails when multiple service ids are
configured. (#2976)
add a7aac99 HDDS-6170. Add metrics to replication manager to track
container health states (#2975)
add bcfb64a HDDS-3231. Cleanup KeyManagerImpl (#2961)
add ac99b47 HDDS-5927. Improve defaults in ContainerBalancerConfiguration
(#2892)
add e1f9f21 HDDS-6157. More consistent synchronization in InputStreams
(#2965)
add d09cdd4 HDDS-4743. [FSO] Add FSO variant of ITestOzoneContractDistcp.
(#2980)
add 195737d HDDS-6114. Intermittent error due to Failed to init RocksDB
(#2947)
add 5447f58 HDDS-6175. Use s3Auth during proxy during decrypt in
RpcClient. (#2981)
add 3eb7235 HDDS-5740. Enable ratis by default for SCM. (#2637)
add 46f305d HDDS-6183. Intermittent failure in
TestKeyDeletingService.checkIfDeleteServiceWithFailingSCM. (#2991)
add 1dd1d0b HDDS-4190. Intermittent failure in
TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest. (#2982)
add 9785941 HDDS-6120. Compute block checksum using chunk checksums
(#2930)
add cde7cb7 HDDS-6147. Add ability in OM to get limited delta updates
(#2956)
add fc3015b HDDS-6195. Remove unused jmh-core dependency. (#2997)
add 0c071ba HDDS-6167. Update ozone-runner version to 20211202-1 (#2969)
add b8d97eb HDDS-6171. Create an API to change Bucket Owner (#2988)
add a99ec03 HDDS-6163. Fix PATH in docker image (#2967)
add 431ce39 HDDS-6202. Avoid using jmh-generator-annprocess since it is
GPL2.0. (#2998)
add 05773af HDDS-6135. SCM Container DB bootstrap on Recon startup for
SCM HA. (#2972)
new f2fdf04 Merge remote-tracking branch 'asf/master' into HDDS-4944
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 +
CONTRIBUTING.md | 3 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 2 +-
hadoop-hdds/common/pom.xml | 15 +
.../apache/hadoop/hdds/DFSConfigKeysLegacy.java | 4 +
.../hadoop/hdds/conf/OzoneConfiguration.java | 8 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 6 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 4 +-
.../protocol/StorageContainerLocationProtocol.java | 6 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 +-
.../common/src/main/resources/ozone-default.xml | 93 ++-
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 28 +
.../hadoop/hdds/tracing/TestStringCodec.java | 18 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 6 +-
.../ozone/container/common/interfaces/Handler.java | 2 +-
.../common/statemachine/DatanodeConfiguration.java | 32 -
.../common/statemachine/DatanodeStateMachine.java | 7 +-
.../CloseContainerCommandHandler.java | 17 +-
.../common/transport/server/XceiverServerGrpc.java | 2 +-
.../transport/server/ratis/XceiverServerRatis.java | 2 +-
.../ozoneimpl/ContainerScrubberConfiguration.java | 65 +-
.../ozone/container/ozoneimpl/OzoneContainer.java | 11 +
.../container/replication/ContainerDownloader.java | 3 +-
.../replication/DownloadAndImportReplicator.java | 19 +-
.../container/replication/ReplicationServer.java | 56 +-
.../replication/ReplicationSupervisor.java | 8 +
.../replication/SimpleContainerDownloader.java | 39 +-
.../statemachine/TestDatanodeConfiguration.java | 11 -
.../TestCloseContainerCommandHandler.java | 4 +-
.../TestContainerScrubberConfiguration.java | 97 +++
.../replication/TestReplicationConfig.java | 75 ++
.../replication/TestReplicationSupervisor.java | 4 +-
.../replication/TestSimpleContainerDownloader.java | 13 +-
hadoop-hdds/docs/config.yaml | 5 +
hadoop-hdds/docs/content/security/SecureOzone.md | 10 +-
.../docs/content/security/SecureOzone.zh.md | 34 +-
.../docs/content/security/SecuringDatanodes.md | 2 +-
.../docs/content/security/SecuringDatanodes.zh.md | 2 +-
hadoop-hdds/docs/content/start/OnPrem.md | 1 +
hadoop-hdds/docs/content/start/OnPrem.zh.md | 2 +
hadoop-hdds/docs/content/tools/TestTools.md | 129 +--
hadoop-hdds/docs/content/tools/TestTools.zh.md | 129 +--
hadoop-hdds/docs/content/tools/_index.md | 1 -
hadoop-hdds/docs/content/tools/_index.zh.md | 1 -
.../java/org/apache/hadoop/hdds/ExitManager.java | 4 +
...inerLocationProtocolClientSideTranslatorPB.java | 48 +-
.../java/org/apache/hadoop/hdds/utils/HAUtils.java | 2 +-
.../org/apache/hadoop/hdds/utils/db/DBStore.java | 10 +
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 11 +
.../certificate/authority/TestDefaultCAServer.java | 6 +-
.../apache/hadoop/hdds/utils/db/TestRDBStore.java | 24 +
.../src/main/proto/ScmAdminProtocol.proto | 17 +-
hadoop-hdds/server-scm/pom.xml | 10 -
.../hdds/scm/container/ContainerReplicaCount.java | 10 +
.../hdds/scm/container/ReplicationManager.java | 62 +-
.../scm/container/ReplicationManagerReport.java | 192 +++++
...etGreedy.java => AbstractFindTargetGreedy.java} | 87 +-
.../scm/container/balancer/ContainerBalancer.java | 90 +-
.../balancer/ContainerBalancerConfiguration.java | 218 +++--
.../FindTargetGreedyByNetworkTopology.java | 79 ++
.../balancer/FindTargetGreedyByUsageInfo.java | 52 ++
.../replication/ReplicationManagerMetrics.java | 42 +
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 35 +-
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 3 +-
.../apache/hadoop/hdds/scm/ha/SCMStateMachine.java | 6 +-
.../hdds/scm/node/DatanodeAdminMonitorImpl.java | 24 +
...inerLocationProtocolServerSideTranslatorPB.java | 45 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 45 +-
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 3 +-
.../hdds/scm/server/StorageContainerManager.java | 9 +-
.../server/StorageContainerManagerHttpServer.java | 4 +-
.../hdds/scm/container/TestReplicationManager.java | 161 +++-
.../container/TestReplicationManagerReport.java | 113 +++
.../container/balancer/TestContainerBalancer.java | 119 +--
.../container/balancer/TestFindTargetStrategy.java | 196 +++++
.../replication/TestReplicationManagerMetrics.java | 97 +++
.../states/TestContainerReplicaCount.java | 10 +
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 10 +-
.../server/TestSCMUpdateServiceGrpcServer.java | 8 +-
.../TestSCMHAUnfinalizedStateValidationAction.java | 1 +
.../org/apache/hadoop/hdds/cli/OzoneAdmin.java | 11 +
.../hdds/scm/cli/ContainerBalancerCommands.java | 19 +-
.../scm/cli/ContainerBalancerStartSubcommand.java | 50 +-
.../hdds/scm/cli/ContainerOperationClient.java | 6 +-
.../datanode/TestContainerBalancerSubCommand.java | 8 +-
hadoop-ozone/client/pom.xml | 14 +
.../apache/hadoop/ozone/client/OzoneBucket.java | 11 +
.../checksum/AbstractBlockChecksumComputer.java | 25 +-
.../client/checksum/BaseFileChecksumHelper.java | 200 +++++
.../checksum/ReplicatedBlockChecksumComputer.java | 72 ++
.../checksum/ReplicatedFileChecksumHelper.java | 187 +++++
.../ozone/client/{ => checksum}/package-info.java | 2 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 8 +-
.../ozone/client/protocol/ClientProtocol.java | 10 +
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 33 +-
.../TestReplicatedBlockChecksumComputer.java | 68 ++
.../checksum/TestReplicatedFileChecksumHelper.java | 323 ++++++++
.../ozone/client/checksum}/package-info.java | 4 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 94 ++-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 7 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 3 +-
.../hadoop/ozone/om/helpers/OMNodeDetails.java | 25 +-
.../hadoop/ozone/om/helpers/OmBucketArgs.java | 36 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 10 +-
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 17 +-
.../hadoop/ozone/om/helpers/RepeatedOmKeyInfo.java | 4 +
.../hadoop/ozone/om/protocol/OMAdminProtocol.java | 6 +
.../hadoop/ozone/om/protocol/OMConfiguration.java | 30 +-
.../ozone/om/protocol/OzoneManagerProtocol.java | 12 +
.../protocolPB/OMAdminProtocolClientSideImpl.java | 112 ++-
...OzoneManagerProtocolClientSideTranslatorPB.java | 22 +
hadoop-ozone/dev-support/checks/acceptance.sh | 5 +-
hadoop-ozone/dev-support/checks/coverage.sh | 1 -
hadoop-ozone/dev-support/checks/kubernetes.sh | 4 +-
hadoop-ozone/dist/pom.xml | 2 +-
.../dist/src/main/assemblies/ozone-src.xml | 1 +
.../dist/src/main/compose/ozone-csi/docker-config | 1 +
.../dist/src/main/compose/ozone-ha/docker-config | 4 +
.../dist/src/main/compose/ozone/docker-config | 1 +
.../src/main/compose/ozonesecure-ha/docker-config | 2 +-
.../compose/ozonesecure-ha/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/om.keytab | Bin 278 -> 0 bytes
.../compose/ozonesecure-ha/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure-ha/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../ozonesecure-ha/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../src/main/compose/ozonesecure-mr/docker-config | 3 +-
.../compose/ozonesecure-mr/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/om.keytab | Bin 278 -> 0 bytes
.../compose/ozonesecure-mr/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure-mr/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../ozonesecure-mr/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../src/main/compose/ozonesecure/docker-config | 4 +-
.../main/compose/ozonesecure/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../src/main/compose/ozonesecure/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../src/main/compose/ozonesecure/keytabs/om.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../compose/ozonesecure/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../main/compose/upgrade/compose/ha/docker-config | 4 +-
.../compose/upgrade/compose/non-ha/docker-config | 2 +-
hadoop-ozone/dist/src/main/docker/Dockerfile | 1 -
.../dist/src/main/dockerlibexec/entrypoint.sh | 2 +-
hadoop-ozone/dist/src/main/k8s/examples/testlib.sh | 5 +-
.../dist/src/main/smoketest/s3/awss3.robot | 8 +
.../dist/src/main/smoketest/s3/objectcopy.robot | 20 +-
.../dist/src/main/smoketest/s3/objectdelete.robot | 32 +-
.../src/main/smoketest/s3/objectmultidelete.robot | 24 +-
.../dist/src/main/smoketest/s3/objectputget.robot | 42 +-
hadoop-ozone/dist/src/shell/ozone/ozone | 17 -
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 39 +-
hadoop-ozone/insight/pom.xml | 1 -
hadoop-ozone/integration-test/pom.xml | 10 -
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 2 +
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 4 +-
...p.java => ITestOzoneContractDistCpWithFSO.java} | 16 +-
.../hadoop/fs/ozone/contract/OzoneContract.java | 18 +-
.../metrics/TestSCMContainerManagerMetrics.java | 2 +
.../hdds/scm/pipeline/TestMultiRaftSetup.java | 1 +
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 2 +
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 4 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 16 -
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 368 ++++++---
.../hadoop/ozone/MiniOzoneOMHAClusterImpl.java | 116 ---
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 23 +
.../ozone/TestContainerBalancerOperations.java | 19 +-
.../hadoop/ozone/TestMiniOzoneOMHACluster.java | 6 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 3 +
.../hadoop/ozone/TestSecureOzoneCluster.java | 15 +-
.../hadoop/ozone/TestStorageContainerManager.java | 3 +
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 7 +-
.../client/rpc/TestContainerStateMachine.java | 5 +-
.../rpc/TestContainerStateMachineFailures.java | 85 ++
.../rpc/TestContainerStateMachineFlushDelay.java | 4 +-
.../client/rpc/TestOzoneAtRestEncryption.java | 27 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 22 +
.../TestOzoneRpcClientWithKeyLatestVersion.java | 209 +++++
.../commandhandler/TestBlockDeletion.java | 1 +
.../transport/server/ratis/TestCSMMetrics.java | 7 +
.../ozoneimpl/TestSecureOzoneContainer.java | 2 +
.../server/TestSecureContainerServer.java | 3 +
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 1 +
...otstrap.java => TestAddRemoveOzoneManager.java} | 107 ++-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 361 ++++----
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 6 +-
.../hadoop/ozone/om/TestOMUpgradeFinalization.java | 6 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 8 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithFailover.java | 2 +-
.../om/multitenant/TestMultiTenantVolume.java | 28 +-
.../snapshot/TestOzoneManagerSnapshotProvider.java | 8 +-
.../hadoop/ozone/recon/TestReconScmHASnapshot.java | 70 ++
.../hadoop/ozone/recon/TestReconScmSnapshot.java | 133 +++
.../ozone/recon/TestReconWithOzoneManagerHA.java | 12 +-
.../ozone/scm/TestSCMInstallSnapshotWithHA.java | 18 +-
.../scm/node/TestDecommissionAndMaintenance.java | 6 +-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 4 +-
.../src/main/proto/OMAdminProtocol.proto | 29 +-
.../src/main/proto/OmClientProtocol.proto | 4 +-
.../ozone/om/codec/RepeatedOmKeyInfoCodec.java | 2 +-
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 36 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 111 ---
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 912 ---------------------
.../org/apache/hadoop/ozone/om/OzoneManager.java | 140 +++-
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 4 +-
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 6 -
.../apache/hadoop/ozone/om/ha/OMHANodeDetails.java | 3 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 74 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 9 +-
...tyRequest.java => OMBucketSetOwnerRequest.java} | 223 ++---
.../om/request/file/OMDirectoryCreateRequest.java | 2 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 2 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 6 +-
.../request/file/OMFileCreateRequestWithFSO.java | 5 +-
.../ozone/om/request/file/OMFileRequest.java | 25 +
.../om/request/key/OMAllocateBlockRequest.java | 4 +-
.../request/key/OMAllocateBlockRequestWithFSO.java | 4 +-
.../ozone/om/request/key/OMKeyCommitRequest.java | 2 +-
.../om/request/key/OMKeyCommitRequestWithFSO.java | 2 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 6 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 32 +-
.../ozone/om/request/key/OMKeyDeleteRequest.java | 3 +-
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 2 +-
.../ozone/om/request/key/OMKeyRenameRequest.java | 2 +-
.../om/request/key/OMKeyRenameRequestWithFSO.java | 2 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 35 +-
.../ozone/om/request/key/OMKeysDeleteRequest.java | 3 +-
.../om/request/key/OMPathsPurgeRequestWithFSO.java | 2 +-
.../ozone/om/request/key/acl/OMKeyAclRequest.java | 3 +-
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 2 +-
.../request/key/acl/OMKeyAddAclRequestWithFSO.java | 3 +-
.../key/acl/OMKeyRemoveAclRequestWithFSO.java | 3 +-
.../request/key/acl/OMKeySetAclRequestWithFSO.java | 3 +-
.../S3InitiateMultipartUploadRequest.java | 4 +-
.../S3InitiateMultipartUploadRequestWithFSO.java | 19 +-
.../multipart/S3MultipartUploadAbortRequest.java | 4 +-
.../S3MultipartUploadAbortRequestWithFSO.java | 4 +-
.../S3MultipartUploadCommitPartRequest.java | 2 +-
.../S3MultipartUploadCommitPartRequestWithFSO.java | 2 +-
.../S3MultipartUploadCompleteRequest.java | 12 +-
.../S3MultipartUploadCompleteRequestWithFSO.java | 20 +-
...Response.java => OMBucketSetOwnerResponse.java} | 64 +-
.../response/file/OMDirectoryCreateResponse.java | 10 +-
.../file/OMDirectoryCreateResponseWithFSO.java | 15 +-
.../om/response/file/OMFileCreateResponse.java | 6 +-
.../response/file/OMFileCreateResponseWithFSO.java | 5 +-
.../response/key/AbstractOMKeyDeleteResponse.java | 5 +-
.../om/response/key/OMAllocateBlockResponse.java | 13 +-
.../key/OMAllocateBlockResponseWithFSO.java | 14 +-
.../ozone/om/response/key/OMKeyCommitResponse.java | 6 +-
.../response/key/OMKeyCommitResponseWithFSO.java | 5 +-
.../ozone/om/response/key/OMKeyCreateResponse.java | 6 +-
.../response/key/OMKeyCreateResponseWithFSO.java | 6 +-
.../ozone/om/response/key/OMKeyDeleteResponse.java | 6 +-
.../response/key/OMKeyDeleteResponseWithFSO.java | 5 +-
.../ozone/om/response/key/OMKeyPurgeResponse.java | 3 +-
.../ozone/om/response/key/OMKeyRenameResponse.java | 5 +-
.../response/key/OMKeyRenameResponseWithFSO.java | 5 +-
.../om/response/key/OMKeysDeleteResponse.java | 6 +-
.../om/response/key/OMOpenKeysDeleteRequest.java | 2 +-
.../om/response/key/OMOpenKeysDeleteResponse.java | 6 +-
.../response/key/OMPathsPurgeResponseWithFSO.java | 5 +-
.../ozone/om/response/key/OmKeyResponse.java | 25 -
.../om/response/key/acl/OMKeyAclResponse.java | 17 +-
.../response/key/acl/OMKeyAclResponseWithFSO.java | 16 +-
.../S3InitiateMultipartUploadResponse.java | 14 +-
.../S3InitiateMultipartUploadResponseWithFSO.java | 10 +-
.../multipart/S3MultipartUploadAbortResponse.java | 18 +-
.../S3MultipartUploadAbortResponseWithFSO.java | 13 +-
.../S3MultipartUploadCommitPartResponse.java | 14 +-
...S3MultipartUploadCommitPartResponseWithFSO.java | 9 +-
.../S3MultipartUploadCompleteResponse.java | 40 +-
.../S3MultipartUploadCompleteResponseWithFSO.java | 11 +-
.../om/snapshot/OzoneManagerSnapshotProvider.java | 7 +
.../protocolPB/OMAdminProtocolServerSideImpl.java | 50 ++
.../protocolPB/OzoneManagerRequestHandler.java | 1 +
.../hadoop/ozone/security/AWSV4AuthValidator.java | 10 -
.../org/apache/hadoop/ozone/om/OmTestManagers.java | 122 +++
.../hadoop/ozone/om/TestKeyDeletingService.java | 99 ++-
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 56 +-
.../apache/hadoop/ozone/om/TestTrashService.java | 34 +-
.../volume/TestOMVolumeSetOwnerRequest.java | 7 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 7 +-
.../ozone/om/response/TestCleanupTableInfo.java | 7 +-
.../file/TestOMDirectoryCreateResponse.java | 2 +-
.../file/TestOMDirectoryCreateResponseWithFSO.java | 4 +-
.../response/key/TestOMAllocateBlockResponse.java | 2 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 2 +-
.../s3/multipart/TestS3MultipartResponse.java | 13 +-
.../TestS3MultipartUploadAbortResponse.java | 20 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 22 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 77 +-
.../hadoop/ozone/security/acl/TestParentAcl.java | 44 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 41 +-
.../apache/hadoop/ozone/recon/ReconConstants.java | 2 +
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 20 +-
.../ozone/recon/fsck/ContainerHealthTask.java | 2 +-
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 11 +
.../scm/ReconStorageContainerManagerFacade.java | 129 ++-
.../recon/spi/StorageContainerServiceProvider.java | 12 +
.../spi/impl/OzoneManagerServiceProviderImpl.java | 8 +-
.../impl/StorageContainerServiceProviderImpl.java | 139 +++-
.../TestStorageContainerServiceProviderImpl.java | 12 +
.../hadoop/ozone/s3/OzoneClientProducer.java | 10 +-
.../hadoop/ozone/s3/OzoneServiceProvider.java | 94 ---
.../hadoop/ozone/s3/commontypes/CommonPrefix.java | 2 +
.../hadoop/ozone/s3/commontypes/KeyMetadata.java | 1 +
...oDateAdapter.java => ObjectKeyNameAdapter.java} | 29 +-
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 19 +-
.../s3/signature/AuthorizationV4QueryParser.java | 5 +-
.../ozone/s3/signature/StringToSignProducer.java | 5 +-
.../apache/hadoop/ozone/s3/util/OzoneS3Util.java | 81 --
.../org/apache/hadoop/ozone/s3/util/S3Utils.java | 30 +-
.../hadoop/ozone/s3/TestOzoneClientProducer.java | 33 +
.../TestObjectKeyNameAdapter.java} | 27 +-
.../hadoop/ozone/s3/endpoint/TestObjectPut.java | 15 +-
.../hadoop/ozone/s3/util/TestOzoneS3Util.java | 136 ---
.../tools/dev-support/findbugsExcludeFile.xml | 4 -
hadoop-ozone/tools/pom.xml | 15 -
.../ozone/admin/om/DecommissionOMSubcommand.java | 212 +++++
.../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 7 +-
.../hadoop/ozone/genesis/BenchMarkCRCBatch.java | 141 ----
.../ozone/genesis/BenchMarkCRCStreaming.java | 173 ----
.../ozone/genesis/BenchMarkContainerStateMap.java | 199 -----
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 339 --------
.../ozone/genesis/BenchMarkOzoneManager.java | 193 -----
.../apache/hadoop/ozone/genesis/BenchMarkSCM.java | 126 ---
.../ozone/genesis/BenchmarkBlockDataToString.java | 166 ----
.../ozone/genesis/BenchmarkChunkManager.java | 180 ----
.../org/apache/hadoop/ozone/genesis/Genesis.java | 108 ---
.../ozone/genesis/GenesisMemoryProfiler.java | 61 --
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 162 ----
.../apache/hadoop/ozone/genesis/package-info.java | 25 -
.../hadoop/ozone/shell/bucket/BucketCommands.java | 3 +-
.../UpdateBucketHandler.java} | 46 +-
pom.xml | 20 +-
345 files changed, 6382 insertions(+), 5667 deletions(-)
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestContainerScrubberConfiguration.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationConfig.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManagerReport.java
rename
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/{FindTargetGreedy.java
=> AbstractFindTargetGreedy.java} (80%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindTargetGreedyByNetworkTopology.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindTargetGreedyByUsageInfo.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestReplicationManagerReport.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestFindTargetStrategy.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationManagerMetrics.java
copy
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/InsufficientDatanodesException.java
=>
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/AbstractBlockChecksumComputer.java
(59%)
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/BaseFileChecksumHelper.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/ReplicatedBlockChecksumComputer.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/ReplicatedFileChecksumHelper.java
copy hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/{ =>
checksum}/package-info.java (94%)
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum/TestReplicatedBlockChecksumComputer.java
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum/TestReplicatedFileChecksumHelper.java
copy {hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client =>
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum}/package-info.java
(86%)
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser2.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser2.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser2.keytab
copy
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/contract/{ITestOzoneContractDistCp.java
=> ITestOzoneContractDistCpWithFSO.java} (79%)
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneOMHAClusterImpl.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithKeyLatestVersion.java
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/{TestOzoneManagerBootstrap.java
=> TestAddRemoveOzoneManager.java} (78%)
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmHASnapshot.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmSnapshot.java
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/{OMBucketSetPropertyRequest.java
=> OMBucketSetOwnerRequest.java} (50%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/bucket/{OMBucketCreateResponse.java
=> OMBucketSetOwnerResponse.java} (64%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/OmTestManagers.java
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
copy
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/{IsoDateAdapter.java
=> ObjectKeyNameAdapter.java} (63%)
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/OzoneS3Util.java
copy
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/ClientVersions.java =>
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Utils.java
(60%)
copy
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/{endpoint/TestBucketResponse.java
=> commontypes/TestObjectKeyNameAdapter.java} (59%)
delete mode 100644
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/DecommissionOMSubcommand.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkCRCBatch.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkCRCStreaming.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkContainerStateMap.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkDatanodeDispatcher.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOzoneManager.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkSCM.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkBlockDataToString.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkChunkManager.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/Genesis.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisMemoryProfiler.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisUtil.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/package-info.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/{keys/RenameKeyHandler.java
=> bucket/UpdateBucketHandler.java} (59%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
