This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a change to branch HDDS-6030
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from cf6c985 HDDS-6033. Create a new docker compose env for External Root
CA (#2966)
add 1ea978a HDDS-6149. Remove unused keytabs (#2960)
add 243cc3f HDDS-6094. Some unit tests are skipped due to using JUnit4
runner (#2909)
add 61f1c70 HDDS-6075. OzoneConfiguration constructor overrides input
configuration keys. (#2921)
add ca25193 HDDS-4177. SCM Container DB bootstrap on Recon startup (#2942)
add 9644f83c HDDS-6086. Compute MD5MD5CRC file checksum using chunk
checksums from DataNodes (#2919)
add a31b79a HDDS-6148. Validate ContainerBalancerConfiguration before
start ContainerBalancer (#2957)
add c8f6ad0 HDDS-6161. SCM StateMachine failing to reinitialize doesn't
terminate the process. (#2971)
add a348980 HDDS-6134. Move replication-specific config to
ReplicationServer (#2943)
add 5200eab HDDS-4010. S3G startup fails when multiple service ids are
configured. (#2976)
add a7aac99 HDDS-6170. Add metrics to replication manager to track
container health states (#2975)
add bcfb64a HDDS-3231. Cleanup KeyManagerImpl (#2961)
add ac99b47 HDDS-5927. Improve defaults in ContainerBalancerConfiguration
(#2892)
add e1f9f21 HDDS-6157. More consistent synchronization in InputStreams
(#2965)
add d09cdd4 HDDS-4743. [FSO] Add FSO variant of ITestOzoneContractDistcp.
(#2980)
new c1790ce Merge remote-tracking branch 'origin/master' into HDDS-6030
add 195737d HDDS-6114. Intermittent error due to Failed to init RocksDB
(#2947)
add 5447f58 HDDS-6175. Use s3Auth during proxy during decrypt in
RpcClient. (#2981)
add 3eb7235 HDDS-5740. Enable ratis by default for SCM. (#2637)
add 46f305d HDDS-6183. Intermittent failure in
TestKeyDeletingService.checkIfDeleteServiceWithFailingSCM. (#2991)
add 1dd1d0b HDDS-4190. Intermittent failure in
TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest. (#2982)
add 9785941 HDDS-6120. Compute block checksum using chunk checksums
(#2930)
add cde7cb7 HDDS-6147. Add ability in OM to get limited delta updates
(#2956)
add fc3015b HDDS-6195. Remove unused jmh-core dependency. (#2997)
add 0c071ba HDDS-6167. Update ozone-runner version to 20211202-1 (#2969)
add b8d97eb HDDS-6171. Create an API to change Bucket Owner (#2988)
add a99ec03 HDDS-6163. Fix PATH in docker image (#2967)
add 431ce39 HDDS-6202. Avoid using jmh-generator-annprocess since it is
GPL2.0. (#2998)
add 05773af HDDS-6135. SCM Container DB bootstrap on Recon startup for
SCM HA. (#2972)
add 66aadb0 HDDS-6109. Preserve the underlying exception raised in client
lib. (#2989)
add bd91ab9 HDDS-3408. Rename ChunkLayOutVersion to
ContainerLayoutVersion. (#2983)
add 18f1461 HDDS-6203. CleanUp incomplete gz files during Container move
(#3000)
add 2af225d HDDS-6216. Move OMOpenKeysDeleteRequest to package
om.request.key (#3011)
add ada4a3b HDDS-6191. Intermittent failure in TestDeleteWithSlowFollower
(#3015)
add 6180d21 HDDS-6128. CLI tool that downloads all the block replicas and
creates a manifest file (#2987)
add 3a16ebe HDDS-6177. Extend container info command to include replica
details (#2995)
add 5823f56 HDDS-6211. [Docs] Image styling on deployed site does not
replicate local builds. (#3007)
add d3b1a06 HDDS-6219. Switch to RATIS ReplicationType from STAND_ALONE
in our tests. (#3014)
add dc13410 HDDS-6192. feature/Observability.md translated to Chinese
(#2994)
add dc6f279 HDDS-6205. Add CLI command to display the latest Replication
Manager report (#3013)
add 11605e1 HDDS-6227. Test helpers should observe naming conditions
(#3020)
add 634d666 HDDS-6239. ozonesecure-mr failing with No URLs in mirrorlist
(#3029)
new c48b559 Merge remote-tracking branch 'origin/master' into HDDS-6030
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CONTRIBUTING.md | 3 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 2 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 4 +-
hadoop-hdds/common/pom.xml | 15 +
.../hadoop/hdds/conf/OzoneConfiguration.java | 7 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 6 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 24 +-
.../hdds/scm/container/ContainerReplicaInfo.java | 129 +++
.../scm/container/ReplicationManagerReport.java | 283 +++++++
.../protocol/StorageContainerLocationProtocol.java | 24 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 +-
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +-
.../common/src/main/resources/ozone-default.xml | 41 +-
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 28 +
.../scm/container/TestContainerReplicaInfo.java | 59 ++
.../container/TestReplicationManagerReport.java | 162 ++++
.../hadoop/hdds/scm/container}/package-info.java | 5 +-
.../hadoop/hdds/tracing/TestStringCodec.java | 18 +-
...TestUpgradeUtils.java => UpgradeTestUtils.java} | 4 +-
.../common/helpers/DatanodeVersionFile.java | 8 +-
.../ozone/container/common/impl/ContainerData.java | 19 +-
.../container/common/impl/ContainerDataYaml.java | 11 +-
...OutVersion.java => ContainerLayoutVersion.java} | 34 +-
.../common/statemachine/DatanodeConfiguration.java | 32 -
.../common/statemachine/DatanodeStateMachine.java | 5 +-
.../container/keyvalue/KeyValueContainerCheck.java | 10 +-
.../container/keyvalue/KeyValueContainerData.java | 8 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 6 +-
.../keyvalue/impl/ChunkManagerDispatcher.java | 18 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 4 +-
.../keyvalue/impl/FilePerChunkStrategy.java | 4 +-
.../replication/GrpcReplicationClient.java | 15 +-
.../container/replication/ReplicationServer.java | 54 +-
.../replication/ReplicationSupervisor.java | 8 +
.../ozone/container/common/ContainerTestUtils.java | 4 +-
.../container/common/TestBlockDeletingService.java | 13 +-
...ersion.java => TestContainerLayoutVersion.java} | 12 +-
.../common/TestKeyValueContainerData.java | 10 +-
.../common/impl/TestContainerDataYaml.java | 16 +-
.../impl/TestContainerDeletionChoosingPolicy.java | 8 +-
.../common/impl/TestContainerPersistence.java | 8 +-
.../container/common/impl/TestContainerSet.java | 8 +-
.../container/common/impl/TestHddsDispatcher.java | 8 +-
.../statemachine/TestDatanodeConfiguration.java | 11 -
.../TestCloseContainerCommandHandler.java | 10 +-
.../common/volume/TestStorageVolumeChecker.java | 8 +-
...tTestInfo.java => ContainerLayoutTestInfo.java} | 24 +-
.../keyvalue/TestKeyValueBlockIterator.java | 10 +-
.../container/keyvalue/TestKeyValueContainer.java | 16 +-
.../keyvalue/TestKeyValueContainerCheck.java | 9 +-
.../TestKeyValueContainerMarkUnhealthy.java | 8 +-
.../container/keyvalue/TestKeyValueHandler.java | 8 +-
.../container/keyvalue/TestTarContainerPacker.java | 8 +-
.../keyvalue/impl/AbstractTestChunkManager.java | 8 +-
.../keyvalue/impl/TestBlockManagerImpl.java | 10 +-
.../keyvalue/impl/TestChunkManagerDummyImpl.java | 6 +-
.../keyvalue/impl/TestFilePerBlockStrategy.java | 6 +-
.../keyvalue/impl/TestFilePerChunkStrategy.java | 12 +-
.../container/ozoneimpl/TestContainerReader.java | 8 +-
.../container/ozoneimpl/TestOzoneContainer.java | 10 +-
.../replication/TestReplicationConfig.java | 75 ++
.../replication/TestReplicationSupervisor.java | 10 +-
.../upgrade/TestDataNodeStartupSlvLessThanMlv.java | 4 +-
hadoop-hdds/docs/content/feature/Observability.md | 2 +-
.../{Observability.md => Observability.zh.md} | 63 +-
hadoop-hdds/docs/content/tools/TestTools.md | 129 +--
hadoop-hdds/docs/content/tools/TestTools.zh.md | 129 +--
hadoop-hdds/docs/content/tools/_index.md | 1 -
hadoop-hdds/docs/content/tools/_index.zh.md | 1 -
.../themes/ozonedoc/layouts/shortcodes/image.html | 2 +-
...inerLocationProtocolClientSideTranslatorPB.java | 86 +-
.../org/apache/hadoop/hdds/utils/db/DBStore.java | 10 +
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 11 +
.../apache/hadoop/hdds/utils/db/TestRDBStore.java | 24 +
.../src/main/proto/ScmAdminProtocol.proto | 40 +-
.../interface-client/src/main/proto/hdds.proto | 25 +
hadoop-hdds/server-scm/pom.xml | 10 -
.../hdds/scm/container/ContainerReplicaCount.java | 10 +
.../hdds/scm/container/ReplicationManager.java | 67 +-
.../scm/container/balancer/ContainerBalancer.java | 51 +-
.../balancer/ContainerBalancerConfiguration.java | 127 +--
.../replication/ReplicationManagerMetrics.java | 42 +
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 34 +-
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 3 +-
.../apache/hadoop/hdds/scm/ha/SCMStateMachine.java | 4 +-
...inerLocationProtocolServerSideTranslatorPB.java | 78 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 75 +-
.../server/StorageContainerManagerHttpServer.java | 4 +-
.../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 692 +++++++++++++++-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 699 ----------------
.../hadoop/hdds/scm/block/TestBlockManager.java | 18 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 4 +-
.../command/TestCommandStatusReportHandler.java | 4 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 6 +-
.../container/TestCloseContainerEventHandler.java | 4 +-
.../scm/container/TestContainerReportHandler.java | 4 +-
.../TestIncrementalContainerReportHandler.java | 4 +-
.../hdds/scm/container/TestReplicationManager.java | 165 +++-
.../scm/container/TestUnknownContainerReport.java | 2 +-
.../container/balancer/TestContainerBalancer.java | 99 ++-
.../algorithms/TestContainerPlacementFactory.java | 12 +-
.../TestSCMContainerPlacementCapacity.java | 12 +-
.../TestSCMContainerPlacementRackAware.java | 22 +-
.../TestSCMContainerPlacementRandom.java | 16 +-
.../replication/TestReplicationManagerMetrics.java | 97 +++
.../states/TestContainerReplicaCount.java | 10 +
.../hdds/scm/crl/TestCRLStatusReportHandler.java | 4 +-
.../hdds/scm/node/TestContainerPlacement.java | 6 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 43 +-
.../hdds/scm/node/TestNodeDecommissionManager.java | 4 +-
.../hdds/scm/node/TestNodeReportHandler.java | 10 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 78 +-
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 15 +-
.../hadoop/hdds/scm/node/TestStatisticsUpdate.java | 16 +-
.../hdds/scm/pipeline/TestPipelineManagerImpl.java | 11 +-
.../scm/server/TestSCMBlockProtocolServer.java | 4 +-
.../TestSCMHAUnfinalizedStateValidationAction.java | 5 +-
.../scm/upgrade/TestScmStartupSlvLessThanMlv.java | 4 +-
.../ozone/container/common/TestEndPoint.java | 23 +-
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 11 +-
hadoop-hdds/tools/pom.xml | 9 +
.../hdds/scm/cli/ContainerBalancerCommands.java | 19 +-
.../scm/cli/ContainerBalancerStartSubcommand.java | 50 +-
.../hdds/scm/cli/ContainerOperationClient.java | 34 +-
.../hdds/scm/cli/container/ContainerCommands.java | 3 +-
.../hdds/scm/cli/container/InfoSubcommand.java | 61 +-
.../hdds/scm/cli/container/ReportSubcommand.java | 116 +++
.../hdds/scm/cli/container/TestInfoSubCommand.java | 249 ++++++
.../scm/cli/container/TestReportSubCommand.java | 159 ++++
.../datanode/TestContainerBalancerSubCommand.java | 8 +-
hadoop-ozone/client/pom.xml | 4 +
.../apache/hadoop/ozone/client/OzoneBucket.java | 11 +
.../checksum/AbstractBlockChecksumComputer.java | 25 +-
.../client/checksum/BaseFileChecksumHelper.java | 200 +++++
.../checksum/ReplicatedBlockChecksumComputer.java | 72 ++
.../checksum/ReplicatedFileChecksumHelper.java | 187 +++++
.../ozone/client/{ => checksum}/package-info.java | 2 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 8 +-
.../ozone/client/protocol/ClientProtocol.java | 27 +
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 96 ++-
.../TestReplicatedBlockChecksumComputer.java | 68 ++
.../checksum/TestReplicatedFileChecksumHelper.java | 323 ++++++++
.../ozone/client/checksum}/package-info.java | 4 +-
.../client/src/test/resources/log4j.properties | 23 -
.../hadoop/ozone/om/helpers/OMNodeDetails.java | 6 +-
.../hadoop/ozone/om/helpers/OmBucketArgs.java | 36 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 10 +-
.../ozone/om/protocol/OzoneManagerProtocol.java | 12 +
...OzoneManagerProtocolClientSideTranslatorPB.java | 22 +
hadoop-ozone/dev-support/checks/coverage.sh | 1 -
hadoop-ozone/dist/pom.xml | 2 +-
.../dist/src/main/compose/ozone-csi/docker-config | 1 +
.../dist/src/main/compose/ozone-ha/docker-config | 4 +
.../dist/src/main/compose/ozone/docker-config | 1 +
.../compose/ozonesecure-ha/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/om.keytab | Bin 278 -> 0 bytes
.../compose/ozonesecure-ha/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure-ha/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../ozonesecure-ha/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../src/main/compose/ozonesecure-mr/docker-config | 1 +
.../compose/ozonesecure-mr/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/om.keytab | Bin 278 -> 0 bytes
.../compose/ozonesecure-mr/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure-mr/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../ozonesecure-mr/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../dist/src/main/compose/ozonesecure-mr/test.sh | 4 +-
.../src/main/compose/ozonesecure/docker-config | 2 +
.../main/compose/ozonesecure/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../src/main/compose/ozonesecure/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../src/main/compose/ozonesecure/keytabs/om.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../compose/ozonesecure/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../main/compose/upgrade/compose/ha/docker-config | 4 +-
.../compose/upgrade/compose/non-ha/docker-config | 2 +-
hadoop-ozone/dist/src/main/docker/Dockerfile | 1 -
.../dist/src/main/dockerlibexec/entrypoint.sh | 2 +-
hadoop-ozone/dist/src/main/k8s/examples/testlib.sh | 5 +-
hadoop-ozone/dist/src/shell/ozone/ozone | 17 -
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 39 +-
hadoop-ozone/insight/pom.xml | 1 -
hadoop-ozone/integration-test/pom.xml | 10 -
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 2 +
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 4 +-
...p.java => ITestOzoneContractDistCpWithFSO.java} | 16 +-
.../hadoop/fs/ozone/contract/OzoneContract.java | 18 +-
.../metrics/TestSCMContainerManagerMetrics.java | 2 +
.../hdds/scm/pipeline/TestMultiRaftSetup.java | 1 +
.../hdds/scm/pipeline/TestPipelineClose.java | 4 +-
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 2 +
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 2 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 22 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 374 ++++++---
.../hadoop/ozone/MiniOzoneOMHAClusterImpl.java | 116 ---
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 23 +
.../ozone/TestContainerBalancerOperations.java | 19 +-
.../java/org/apache/hadoop/ozone/TestDataUtil.java | 2 +-
.../hadoop/ozone/TestMiniOzoneOMHACluster.java | 6 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 35 +-
.../hadoop/ozone/TestStorageContainerManager.java | 11 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 7 +-
.../client/rpc/TestContainerStateMachine.java | 5 +-
.../rpc/TestContainerStateMachineFailures.java | 161 +++-
.../rpc/TestContainerStateMachineFlushDelay.java | 4 +-
.../client/rpc/TestDeleteWithSlowFollower.java | 11 +-
.../client/rpc/TestOzoneAtRestEncryption.java | 12 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 46 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 115 +--
.../hadoop/ozone/client/rpc/TestReadRetries.java | 4 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 6 +-
.../client/rpc/read/TestChunkInputStream.java | 4 +-
.../ozone/client/rpc/read/TestInputStreamBase.java | 15 +-
.../ozone/client/rpc/read/TestKeyInputStream.java | 4 +-
.../apache/hadoop/ozone/container/TestHelper.java | 7 +-
.../commandhandler/TestBlockDeletion.java | 1 +
.../commandhandler/TestCloseContainerHandler.java | 4 +-
.../commandhandler/TestDeleteContainerHandler.java | 3 +-
.../container/metrics/TestContainerMetrics.java | 3 +-
.../container/server/TestContainerServer.java | 3 +-
.../server/TestSecureContainerServer.java | 3 +-
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +
.../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 7 +-
.../TestDatanodeHddsVolumeFailureDetection.java | 7 +-
.../freon/TestHadoopDirTreeGeneratorWithFSO.java | 4 +-
.../ozone/om/TestContainerReportWithKeys.java | 2 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 383 ++++-----
.../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +-
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 6 +-
.../hadoop/ozone/om/TestOMUpgradeFinalization.java | 6 +-
.../hadoop/ozone/om/TestObjectStoreWithFSO.java | 8 +-
.../org/apache/hadoop/ozone/om/TestOmLDBCli.java | 4 +-
.../ozone/om/TestOmStartupSlvLessThanMlv.java | 4 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 12 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithFailover.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerPrepare.java | 2 +-
.../hadoop/ozone/om/TestRecursiveAclWithFSO.java | 4 +-
.../om/ratis/TestOzoneManagerRatisRequest.java | 4 +-
.../snapshot/TestOzoneManagerSnapshotProvider.java | 8 +-
.../hadoop/ozone/recon/TestReconScmHASnapshot.java | 70 ++
.../hadoop/ozone/recon/TestReconScmSnapshot.java | 133 +++
.../ozone/recon/TestReconWithOzoneManager.java | 4 +-
.../ozone/recon/TestReconWithOzoneManagerFSO.java | 6 +-
.../ozone/recon/TestReconWithOzoneManagerHA.java | 12 +-
.../hadoop/ozone/scm/TestFailoverWithSCMHA.java | 2 +-
.../ozone/scm/TestSCMInstallSnapshotWithHA.java | 18 +-
.../ozone/scm/TestStorageContainerManagerHA.java | 4 +-
.../scm/node/TestDecommissionAndMaintenance.java | 6 +-
.../hadoop/ozone/shell/TestNSSummaryAdmin.java | 4 +-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 4 +-
.../src/main/proto/OmClientProtocol.proto | 4 +-
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 4 +-
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 4 +-
.../ozone/om/helpers/TestInstanceHelper.java | 68 --
.../hadoop/ozone/om/helpers/TestOmPrefixInfo.java | 39 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 111 ---
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 912 ---------------------
.../org/apache/hadoop/ozone/om/OzoneManager.java | 6 +-
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 4 +-
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 6 -
.../om/ratis/utils/OzoneManagerRatisUtils.java | 9 +-
...tyRequest.java => OMBucketSetOwnerRequest.java} | 223 ++---
.../key/OMOpenKeysDeleteRequest.java | 4 +-
...Response.java => OMBucketSetOwnerResponse.java} | 64 +-
.../org/apache/hadoop/ozone/om/OmTestManagers.java | 122 +++
.../hadoop/ozone/om/TestBucketManagerImpl.java | 6 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 106 ++-
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 70 +-
.../hadoop/ozone/om/TestOmMetadataManager.java | 36 +-
.../apache/hadoop/ozone/om/TestTrashService.java | 40 +-
...TestOzoneManagerDoubleBufferWithOMResponse.java | 8 +-
...OMRequestUtils.java => OMRequestTestUtils.java} | 10 +-
.../request/TestOMClientRequestWithUserInfo.java | 2 +-
.../request/bucket/TestOMBucketCreateRequest.java | 8 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 4 +-
.../request/bucket/TestOMBucketDeleteRequest.java | 6 +-
.../bucket/TestOMBucketSetPropertyRequest.java | 8 +-
.../bucket/acl/TestOMBucketAddAclRequest.java | 12 +-
.../bucket/acl/TestOMBucketRemoveAclRequest.java | 14 +-
.../bucket/acl/TestOMBucketSetAclRequest.java | 12 +-
.../request/file/TestOMDirectoryCreateRequest.java | 22 +-
.../file/TestOMDirectoryCreateRequestWithFSO.java | 48 +-
.../om/request/file/TestOMFileCreateRequest.java | 30 +-
.../file/TestOMFileCreateRequestWithFSO.java | 22 +-
.../om/request/key/TestOMAllocateBlockRequest.java | 12 +-
.../key/TestOMAllocateBlockRequestWithFSO.java | 10 +-
.../ozone/om/request/key/TestOMKeyAclRequest.java | 10 +-
.../om/request/key/TestOMKeyAclRequestWithFSO.java | 8 +-
.../om/request/key/TestOMKeyCommitRequest.java | 12 +-
.../request/key/TestOMKeyCommitRequestWithFSO.java | 8 +-
.../om/request/key/TestOMKeyCreateRequest.java | 8 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 6 +-
.../om/request/key/TestOMKeyDeleteRequest.java | 10 +-
.../request/key/TestOMKeyDeleteRequestWithFSO.java | 10 +-
.../key/TestOMKeyPurgeRequestAndResponse.java | 8 +-
.../om/request/key/TestOMKeyRenameRequest.java | 15 +-
.../om/request/key/TestOMKeysDeleteRequest.java | 6 +-
.../om/request/key/TestOMKeysRenameRequest.java | 6 +-
.../request/key/TestOMOpenKeysDeleteRequest.java | 11 +-
.../om/request/key/TestOMPrefixAclRequest.java | 6 +-
.../TestS3InitiateMultipartUploadRequest.java | 6 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 4 +-
.../s3/multipart/TestS3MultipartRequest.java | 12 +-
.../TestS3MultipartUploadAbortRequest.java | 8 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 4 +-
.../TestS3MultipartUploadCommitPartRequest.java | 12 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 10 +-
.../TestS3MultipartUploadCompleteRequest.java | 12 +-
...estS3MultipartUploadCompleteRequestWithFSO.java | 8 +-
.../upgrade/TestOMCancelPrepareRequest.java | 2 +-
.../request/volume/TestOMVolumeCreateRequest.java | 4 +-
.../request/volume/TestOMVolumeDeleteRequest.java | 12 +-
.../volume/TestOMVolumeSetOwnerRequest.java | 27 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 27 +-
.../volume/acl/TestOMVolumeAddAclRequest.java | 12 +-
.../volume/acl/TestOMVolumeRemoveAclRequest.java | 14 +-
.../volume/acl/TestOMVolumeSetAclRequest.java | 12 +-
.../file/TestOMDirectoryCreateResponse.java | 4 +-
.../file/TestOMDirectoryCreateResponseWithFSO.java | 4 +-
.../file/TestOMFileCreateResponseWithFSO.java | 4 +-
.../response/key/TestOMAllocateBlockResponse.java | 4 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 4 +-
.../om/response/key/TestOMKeyCommitResponse.java | 6 +-
.../key/TestOMKeyCommitResponseWithFSO.java | 8 +-
.../key/TestOMKeyCreateResponseWithFSO.java | 4 +-
.../om/response/key/TestOMKeyDeleteResponse.java | 4 +-
.../key/TestOMKeyDeleteResponseWithFSO.java | 12 +-
.../om/response/key/TestOMKeyRenameResponse.java | 14 +-
.../ozone/om/response/key/TestOMKeyResponse.java | 4 +-
.../om/response/key/TestOMKeysDeleteResponse.java | 4 +-
.../om/response/key/TestOMKeysRenameResponse.java | 8 +-
.../response/key/TestOMOpenKeysDeleteResponse.java | 8 +-
...S3MultipartUploadCommitPartResponseWithFSO.java | 10 +-
...stS3MultipartUploadCompleteResponseWithFSO.java | 18 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 83 +-
.../hadoop/ozone/security/acl/TestParentAcl.java | 50 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 47 +-
.../apache/hadoop/ozone/recon/ReconConstants.java | 2 +
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 16 +
.../ozone/recon/fsck/ContainerHealthTask.java | 2 +-
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 11 +
.../scm/ReconStorageContainerManagerFacade.java | 129 ++-
.../recon/spi/StorageContainerServiceProvider.java | 12 +
.../spi/impl/OzoneManagerServiceProviderImpl.java | 8 +-
.../impl/StorageContainerServiceProviderImpl.java | 139 +++-
.../TestStorageContainerServiceProviderImpl.java | 12 +
.../hadoop/ozone/s3/OzoneClientProducer.java | 5 +-
.../hadoop/ozone/s3/OzoneServiceProvider.java | 78 --
.../hadoop/ozone/client/OzoneBucketStub.java | 2 +-
.../hadoop/ozone/s3/TestOzoneClientProducer.java | 33 +
.../hadoop/ozone/s3/endpoint/TestObjectHead.java | 2 +-
.../tools/dev-support/findbugsExcludeFile.xml | 4 -
hadoop-ozone/tools/pom.xml | 15 -
.../apache/hadoop/ozone/debug/ChunkKeyHandler.java | 6 +-
.../apache/hadoop/ozone/debug/ReadReplicas.java | 247 ++++++
.../hadoop/ozone/freon/ChunkManagerDiskWrite.java | 8 +-
.../containergenerator/GeneratorDatanode.java | 6 +-
.../hadoop/ozone/genesis/BenchMarkCRCBatch.java | 141 ----
.../ozone/genesis/BenchMarkCRCStreaming.java | 173 ----
.../ozone/genesis/BenchMarkContainerStateMap.java | 199 -----
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 339 --------
.../ozone/genesis/BenchMarkOzoneManager.java | 193 -----
.../apache/hadoop/ozone/genesis/BenchMarkSCM.java | 126 ---
.../ozone/genesis/BenchmarkBlockDataToString.java | 166 ----
.../ozone/genesis/BenchmarkChunkManager.java | 180 ----
.../org/apache/hadoop/ozone/genesis/Genesis.java | 108 ---
.../ozone/genesis/GenesisMemoryProfiler.java | 61 --
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 162 ----
.../apache/hadoop/ozone/genesis/package-info.java | 25 -
.../hadoop/ozone/shell/bucket/BucketCommands.java | 3 +-
.../UpdateBucketHandler.java} | 46 +-
pom.xml | 13 -
381 files changed, 7240 insertions(+), 6395 deletions(-)
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaInfo.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManagerReport.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReplicaInfo.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/TestReplicationManagerReport.java
copy hadoop-hdds/{server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha =>
common/src/test/java/org/apache/hadoop/hdds/scm/container}/package-info.java
(89%)
rename
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/{TestUpgradeUtils.java
=> UpgradeTestUtils.java} (95%)
rename
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/{ChunkLayOutVersion.java
=> ContainerLayoutVersion.java} (77%)
rename
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/{TestChunkLayOutVersion.java
=> TestContainerLayoutVersion.java} (73%)
rename
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/{ChunkLayoutTestInfo.java
=> ContainerLayoutTestInfo.java} (84%)
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationConfig.java
copy hadoop-hdds/docs/content/feature/{Observability.md =>
Observability.zh.md} (65%)
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestUtils.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationManagerMetrics.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ReportSubcommand.java
create mode 100644
hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/container/TestInfoSubCommand.java
create mode 100644
hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/container/TestReportSubCommand.java
copy
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/InsufficientDatanodesException.java
=>
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/AbstractBlockChecksumComputer.java
(59%)
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/BaseFileChecksumHelper.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/ReplicatedBlockChecksumComputer.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/ReplicatedFileChecksumHelper.java
copy hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/{ =>
checksum}/package-info.java (94%)
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum/TestReplicatedBlockChecksumComputer.java
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum/TestReplicatedFileChecksumHelper.java
copy {hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client =>
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum}/package-info.java
(86%)
delete mode 100644 hadoop-ozone/client/src/test/resources/log4j.properties
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser2.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser2.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser2.keytab
copy
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/contract/{ITestOzoneContractDistCp.java
=> ITestOzoneContractDistCpWithFSO.java} (79%)
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneOMHAClusterImpl.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmHASnapshot.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmSnapshot.java
delete mode 100644
hadoop-ozone/interface-storage/src/test/java/org/apache/hadoop/ozone/om/helpers/TestInstanceHelper.java
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/{OMBucketSetPropertyRequest.java
=> OMBucketSetOwnerRequest.java} (50%)
rename
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/{response
=> request}/key/OMOpenKeysDeleteRequest.java (98%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/bucket/{OMBucketCreateResponse.java
=> OMBucketSetOwnerResponse.java} (64%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/OmTestManagers.java
rename
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/{TestOMRequestUtils.java
=> OMRequestTestUtils.java} (99%)
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/ReadReplicas.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkCRCBatch.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkCRCStreaming.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkContainerStateMap.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkDatanodeDispatcher.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOzoneManager.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkSCM.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkBlockDataToString.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkChunkManager.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/Genesis.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisMemoryProfiler.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisUtil.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/package-info.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/{keys/RenameKeyHandler.java
=> bucket/UpdateBucketHandler.java} (59%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
