This is an automated email from the ASF dual-hosted git repository.
siyao pushed a change to branch HDDS-4944
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from 8516765 HDDS-5750. [Multi-Tenant] GetS3Secret should retrieve secret
from new tables as well (#2649)
add a595d06 Revert "HDDS-5483. Validate block file length during put
block (#2463)"
add 04816a6 HDDS-5620. Fix and extend hadoop.http.idle_timeout.ms to 60k
(#2540)
add 18aad55 HDDS-5377. Add a Namespace Summary tab to Recon Web UI (#2530)
add db574d1 HDDS-5632. Intermittent failure in
TestOzoneManagerBootstrap#testBootstrapTwoNewOMs (#2550)
add 52ec27c HDDS-5610. Remove or clarify doc regarding OM federation
(#2529)
add 541a0dc HDDS-5640. Add scm ratis metrics (#2553)
add d90bc30 HDDS-5495. Ozone version mismatch in Kubernetes test lib
(#2544)
add 33fcd77 HDDS-5556. GrpcReplication Client might fail in SCM HA
Cluster (#2556)
add 1d8bfe0 HDDS-5540. Acceptance test: able to generate result in
xunit/junit format as well (#2514)
add d3f223c HDDS-5658. Handle InterruptedException in
DeleteBlocksCommandHandler and ContainerBalancer (#2560)
add b1eaf8f HDDS-5393. Return latest version of key location for client
on createKey/createFile (#2432)
add 6ce3036 HDDS-5554. Option to disable checksum verification. (#2522)
add f92ee75 HDDS-5619. Ozone data corruption issue on follower node.
(#2538)
add 03b0b5b HDDS-5391 Fix issues in SCMHAConfiguration (#2567)
add 318456d HDDS-5637. Add documentation for Recon Namespace REST APIs
(#2566)
add 52e9bb5 HDDS-5655. SCM terminates when allocatecontainer happens on
closed pipeline. (#2569)
add fdbf8c3 HDDS-5668. Fix TestOzoneConfigurationFields. (#2576)
add af6c018 HDDS-5637. Addendum: Add documentation for Recon Namespace
REST APIs (#2579)
add 5b4b21a HDDS-5521. Use short principal name as owner of a volume.
(#2573)
add 25561f8 HDDS-5623. Add data validation checks on datanode. (#2542)
add e12a6b9 HDDS-5611. Fixed NullPointerException in
ContainerStateMachine during Pipeline Close. (#2548)
add 30859fe HDDS-5687. support Optional<T> as parameters of commandLine
(#2588)
add 7dcaad1 HDDS-5681. Speed up TestOzoneManagerHAWithACL (#2587)
add b593fa8 HDDS-5688. Rpc should not retry if exception is
ContainerNotFoundException. (#2589)
add db75046 HDDS-5662. Fix blank traceId for FsShell command (#2568)
add 2958ab3 HDDS-5535. SCM-HA documents configuration error (#2492)
add 33b3f9e HDDS-5669. Fix NSSummaryEndpoint's NPE bug during OM
initialization (#2592)
add aab19a6 HDDS-5368. Add CLI command: ozone admin namespace summary
(#2565)
add 9baca7c HDDS-5692. Combine some tests in TestOzoneManagerHAWithData
to reuse mini-Clusters (#2593)
add 758b8ca HDDS-5689. chooseRandom in NetworkTopology should exclude the
affinity node too (#2590)
add 194de5b HDDS-5680. Fix bug for incomplete SCM ratis log (#2586)
add cdac8bc HDDS-5644. Speed up decommission tests using a background
Mini Cluster provider (#2554)
add fe42640 HDDS-5667. documentation page layout (#2604)
add 730cb66 HDDS-5706. Fix ReplicationManager zero metrics for inflight
actions. (#2605)
add ea1ff1e HDDS-5690. Speed up TestContainerReplication by removing
testSkipDemmissionAndMaintenanceNode (#2591)
add f71474a HDDS-5700. Improve LOG message of decommission progress.
(#2598)
add aa8cfcd HDDS-5709. do not call removeTransactionsFromDB if nothing to
remove (#2608)
add 7e2cdf0 HDDS-5432. Enable downgrade testing after 1.1.0 release.
(#2484)
add bbd0907 HDDS-5678. Handle unsecure SCM HA converted to secure SCM HA.
(#2596)
add 0decca8 HDDS-5716. Fix create key failure error log print (#2614)
add 737297e HDDS-5719. Reduce number of mini-clusters needed for
decommission tests (#2617)
add 2d188cc HDDS-5726. Skip remove for already removed pipeline. (#2624)
add b64080b HDDS-5638. Fix docker-compose to make Recon come up. (#2563)
add e9cce6e HDDS-5670. ContainerBalancer should get OzoneConfiguration
from ContainerBalancerConfiguration. (#2577)
add ebbcf0a HDDS-5711. support -1 for running balancer infinitely (#2621)
add 6239270 HDDS-5724. Add RaftpeerId when getting scm roles (#2622)
add 2b6e1ac HDDS-5718. Refactor TestXceiverClientManager to reuse
mini-clusters (#2616)
add ee993d0 HDDS-5723. Increase time limit of Ozone acceptance tests.
(#2620)
add e392c1a HDDS-5699. Added Log to show why a container was marked
UNHEALTHY. (#2627)
add 753a5b7 HDDS-5715. Make XceiverServerRatis#raftGids a thread-safe
set. (#2613)
add 6569040 HDDS-5502. [OFS] URI parser throws URISyntaxException when
path contains space (#2500)
add bb37a55 HDDS-5735. Prometheus HTTP API Reference cannot be displayed
normally (#2630)
add 67be640 HDDS-5496. Missing directory name in `start_k8s_env` log
(#2545)
add 9aebf6e HDDS-5720. Reuse mini-clusters in TestOzoneFileInterfaces
(#2619)
add e97ea36 HDDS-5679. Use more defensive sizeRequired for replication
manager for container replication. (#2585)
add 2d57e9f HDDS-5708. Skip sending container close command to unhealthy
replica. (#2610)
add 873c0d2 HDDS-5696. Close grpcReplicationClient on exception. (#2595)
add 6543c67 HDDS-5729. support setting maxSizeEnteringTarget and
maxSizeLeavingSource in command line (#2626)
add ec5c51f HDDS-5335. Method not found: allocateBlock - when tracing is
enabled (#2330)
add c5a012b HDDS-5744. Remove unnecessary duplicate semicolon in
PipelineStateMap (#2641)
add ee19b95 HDDS-5748. Reuse mini-clusters in
TestOzoneFSWithObjectStoreCreate (#2646)
add da18617 HDDS-5717. Refactor TestOzoneManagerListVolumes to reuse
mini-ozone clusters (#2615)
add ed30752 HDDS-5749. Reuse mini-clusters in TestOzoneFsHAURLs (#2647)
add d716bf4 HDDS-5402 Support list node based on NodeOperationalState and
NodeState options in printTopology CLI (#2645)
add fab8a12 HDDS-5728. ContainerBalancer should use remaining space to
calculate utilization. (#2625)
add f2c20ce HDDS-5751. Use Mini Cluster Provider to speed up
TestHDDSUpgrade (#2650)
add 5c972c4 HDDS-5753. Split parts of misc suite (#2654)
add 0b9e23e HDDS-5607. remove container manager v1 code (#2525)
add 2d58933 HDDS-5758. Speed up TestKeyInputStream and
TestChunkInputStream by combining some tests (#2656)
add c68468c HDDS-5761. should not shutdown om when setting a bigger
bucket quota than volume quota (#2659)
add 79da6b7 HDDS-5766. Speed up some OM tests by skipping SCM safemode
check (#2662)
add b8afbbc HDDS-5768. Skip safemode check in
TestOzoneManagerRocksDBLogging (#2665)
add 79716f0 HDDS-5765. Test cluster provider possibly returns null (#2663)
add 2c2d2f3 HDDS-5767. Unit check may timeout (#2664)
add b508b25 HDDS-5771. Speed up TestDatanodeHddsVolumeFailureToleration
by reducing dead interval (#2668)
add fc61be5 HDDS-5691. Restrict Recon NSSummaryEndpoint and
ContainerEndpoint to admins. (#2638)
add 3116fa2 HDDS-5773. Avoid code duplication for mini cluster without
datanodes (#2669)
add c25b6d7 HDDS-5759. Bump aspectj version (#2658)
add 87b4971 HDDS-5770. Silent failures of k3s install are difficult to
debug (#2667)
new 7f818ab Merge remote-tracking branch 'asf/master' into HDDS-4944
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/post-commit.yml | 6 +-
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 5 +
.../apache/hadoop/hdds/recon/ReconConfigKeys.java | 20 +
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 10 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 4 +-
.../org/apache/hadoop/hdds/scm/ha/SCMHAUtils.java | 2 +
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 24 +-
.../protocol/StorageContainerLocationProtocol.java | 4 +-
.../apache/hadoop/hdds/tracing/TraceAllMethod.java | 23 +-
.../apache/hadoop/hdds/tracing/TracingUtil.java | 6 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 -
.../upgrade/AbstractLayoutVersionManager.java | 158 +++--
.../common/src/main/resources/ozone-default.xml | 45 +-
.../hdds/scm/net/TestNetworkTopologyImpl.java | 33 +-
.../hadoop/hdds/tracing/TestTraceAllMethod.java | 68 ++
.../hadoop/hdds/tracing/TestTracingUtil.java} | 33 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 5 +-
.../container/common/helpers/ContainerUtils.java | 34 +-
.../ozone/container/common/impl/ContainerSet.java | 3 +
.../container/common/impl/HddsDispatcher.java | 1 +
.../common/interfaces/ContainerDispatcher.java | 7 +-
.../common/statemachine/DatanodeConfiguration.java | 21 +
.../common/statemachine/DatanodeStateMachine.java | 6 +-
.../CloseContainerCommandHandler.java | 2 +
.../ClosePipelineCommandHandler.java | 13 +-
.../commandhandler/DeleteBlocksCommandHandler.java | 4 +-
.../states/endpoint/VersionEndpointTask.java | 5 +-
.../server/ratis/ContainerStateMachine.java | 11 +-
.../transport/server/ratis/XceiverServerRatis.java | 3 +-
.../container/common/utils/HddsVolumeUtil.java | 72 +-
.../ozone/container/common/volume/HddsVolume.java | 2 +-
.../container/common/volume/MutableVolumeSet.java | 13 -
.../container/keyvalue/KeyValueContainer.java | 12 +-
.../container/keyvalue/KeyValueContainerCheck.java | 4 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 52 +-
.../container/keyvalue/helpers/ChunkUtils.java | 18 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 9 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 9 +-
.../container/ozoneimpl/ContainerController.java | 24 +-
.../ozone/container/ozoneimpl/ContainerReader.java | 161 ++---
.../ozone/container/ozoneimpl/OzoneContainer.java | 5 +-
.../replication/GrpcReplicationClient.java | 4 +-
.../container/replication/ReplicationServer.java | 7 +-
.../replication/SimpleContainerDownloader.java | 20 +-
.../upgrade/DataNodeUpgradeFinalizer.java | 5 +-
.../upgrade/DatanodeMetadataFeatures.java | 53 --
.../ScmHAFinalizeUpgradeActionDatanode.java | 129 ++++
.../ScmHAFirstUpgradeLayoutChangeAction.java | 45 --
.../upgrade/VersionedDatanodeFeatures.java | 145 ++++
.../hadoop/ozone/container/common/ScmTestMock.java | 9 +
.../ozone/container/common/TestContainerCache.java | 23 +-
.../common/TestKeyValueContainerData.java | 8 +-
.../common/impl/TestContainerDataYaml.java | 41 +-
.../TestCloseContainerCommandHandler.java | 33 +-
.../upgrade/TestDatanodeUpgradeToScmHA.java | 741 +++++++++++++++++++++
hadoop-hdds/docs/content/concept/OzoneManager.md | 2 +-
.../docs/content/concept/OzoneManager.zh.md | 2 +-
hadoop-hdds/docs/content/feature/SCM-HA.md | 4 +-
hadoop-hdds/docs/content/interface/ReconApi.md | 238 ++++++-
hadoop-hdds/docs/content/interface/ReconApi.zh.md | 2 +-
.../docs/themes/ozonedoc/layouts/index.html | 2 +
...inerLocationProtocolClientSideTranslatorPB.java | 19 +-
.../hadoop/hdds/server/http/HttpServer2.java | 17 +-
...equestLogAppender.java => TestHttpServer2.java} | 40 +-
.../src/main/proto/ScmAdminProtocol.proto | 2 +
.../src/main/proto/DatanodeClientProtocol.proto | 1 +
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 13 +-
.../scm/block/DeletedBlockLogStateManagerImpl.java | 12 +-
.../hdds/scm/block/SCMBlockDeletingService.java | 9 +-
.../container/AbstractContainerReportHandler.java | 6 +-
.../scm/container/CloseContainerEventHandler.java | 4 +-
.../hdds/scm/container/ContainerManager.java | 144 ++--
.../hdds/scm/container/ContainerManagerImpl.java | 73 +-
.../hdds/scm/container/ContainerManagerV2.java | 190 ------
.../hdds/scm/container/ContainerReportHandler.java | 6 +-
.../IncrementalContainerReportHandler.java | 2 +-
.../hdds/scm/container/ReplicationManager.java | 31 +-
.../hdds/scm/container/SCMContainerManager.java | 672 -------------------
.../scm/container/balancer/ContainerBalancer.java | 50 +-
.../balancer/ContainerBalancerConfiguration.java | 14 +-
.../ContainerBalancerSelectionCriteria.java | 14 +-
.../scm/container/balancer/FindTargetGreedy.java | 6 +-
.../container/placement/metrics/SCMNodeStat.java | 23 -
.../replication/ReplicationManagerMetrics.java | 44 +-
.../org/apache/hadoop/hdds/scm/ha/RatisUtil.java | 7 +-
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 3 +-
.../hdds/scm/node/DatanodeAdminMonitorImpl.java | 18 +
.../hadoop/hdds/scm/node/DatanodeUsageInfo.java | 49 +-
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 6 +-
.../hdds/scm/node/NodeDecommissionManager.java | 8 +-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 47 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 20 +
.../hdds/scm/pipeline/PipelineManagerImpl.java | 51 +-
.../hadoop/hdds/scm/pipeline/PipelineStateMap.java | 2 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 18 -
.../pipeline/WritableRatisContainerProvider.java | 105 ++-
...inerLocationProtocolServerSideTranslatorPB.java | 15 +-
.../hdds/scm/safemode/ContainerSafeModeRule.java | 10 +-
.../hdds/scm/safemode/SCMSafeModeManager.java | 10 +-
.../scm/server/OzoneStorageContainerManager.java | 4 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 23 +-
.../hadoop/hdds/scm/server/SCMConfigurator.java | 8 +-
.../hdds/scm/server/StorageContainerManager.java | 54 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 12 +-
.../hadoop/hdds/scm/block/TestBlockManager.java | 6 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 6 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 4 +-
.../container/TestCloseContainerEventHandler.java | 2 +-
.../scm/container/TestContainerManagerImpl.java | 2 +-
.../scm/container/TestContainerReportHandler.java | 4 +-
.../TestIncrementalContainerReportHandler.java | 4 +-
.../hdds/scm/container/TestReplicationManager.java | 33 +-
.../scm/container/TestSCMContainerManager.java | 324 ---------
.../scm/container/TestUnknownContainerReport.java | 4 +-
.../container/balancer/TestContainerBalancer.java | 19 +-
.../TestSCMContainerPlacementRackAware.java | 27 +
.../hdds/scm/node/TestContainerPlacement.java | 95 +--
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 8 +-
.../hdds/scm/pipeline/MockPipelineManager.java | 20 +
.../ozone/container/common/TestEndPoint.java | 4 +-
.../org/apache/hadoop/hdds/cli/OzoneAdmin.java | 7 +
.../scm/cli/ContainerBalancerStartSubcommand.java | 13 +-
.../hdds/scm/cli/ContainerOperationClient.java | 7 +-
.../hadoop/hdds/scm/cli/TopologySubcommand.java | 46 +-
.../datanode/TestContainerBalancerSubCommand.java | 6 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 8 +-
.../main/java/org/apache/hadoop/ozone/OFSPath.java | 45 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 36 +-
hadoop-ozone/dev-support/checks/_lib.sh | 8 +-
.../dist/src/main/compose/ozone-ha/test.sh | 2 +
.../src/main/compose/ozone-mr/hadoop27/test.sh | 2 +
.../src/main/compose/ozone-mr/hadoop31/test.sh | 2 +
.../src/main/compose/ozone-mr/hadoop32/test.sh | 2 +
.../src/main/compose/ozone-mr/hadoop33/test.sh | 2 +
.../dist/src/main/compose/ozone-mr/test.sh | 3 +
.../dist/src/main/compose/ozone/docker-config | 2 +
.../compose/ozonesecure-ha/docker-compose.yaml | 2 +
.../dist/src/main/compose/ozonesecure-ha/test.sh | 2 +
.../dist/src/main/compose/ozonesecure-mr/test.sh | 2 +
.../src/main/compose/ozonesecure/docker-config | 3 +-
hadoop-ozone/dist/src/main/compose/test-all.sh | 3 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 23 +-
.../dist/src/main/compose/upgrade/README.md | 35 +-
.../compose/upgrade/delete-and-regenerate-data.sh | 53 --
.../dist/src/main/compose/upgrade/delete-data.sh | 36 -
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 9 +-
.../manual-upgrade/0.5.0-1.1.0/callback.sh | 40 --
.../upgrades/manual-upgrade}/README.md | 9 +-
.../{1.0.0-1.1.0 => 1.1.0-1.2.0}/callback.sh | 17 +-
.../upgrade/upgrades/non-rolling-upgrade/driver.sh | 31 +-
.../dist/src/main/compose/versions/0.5.0.sh | 26 -
.../dist/src/main/compose/versions/1.0.0.sh | 26 -
.../dist/src/main/compose/versions/1.1.0.sh | 26 -
.../dist/src/main/compose/versions/README.md | 20 -
hadoop-ozone/dist/src/main/compose/xcompat/test.sh | 4 -
hadoop-ozone/dist/src/main/k8s/examples/testlib.sh | 6 +-
.../dist/src/main/smoketest/ozonefs/setup.robot | 2 +-
.../dist/src/main/smoketest/recon/recon-api.robot | 53 +-
.../dist/src/main/smoketest/topology/cli.robot | 6 +
hadoop-ozone/integration-test/pom.xml | 5 +
.../fs/ozone/TestOzoneFSWithObjectStoreCreate.java | 30 +-
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 69 +-
.../fs/ozone/TestOzoneFileInterfacesWithFSO.java | 8 +-
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 35 +-
.../hadoop/hdds/scm/TestSCMInstallSnapshot.java | 4 +-
.../apache/hadoop/hdds/scm/TestSCMSnapshot.java | 4 +-
.../TestContainerStateManagerIntegration.java | 140 ++--
.../metrics/TestSCMContainerManagerMetrics.java | 4 +-
.../hdds/scm/pipeline/TestNode2PipelineMap.java | 4 +-
.../hdds/scm/pipeline/TestPipelineClose.java | 4 +-
.../hadoop/hdds/scm/pipeline/TestSCMRestart.java | 6 +-
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 41 +-
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 19 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 2 +-
.../hadoop/ozone/MiniOzoneClusterProvider.java | 286 ++++++++
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 21 +-
.../ozone/TestContainerBalancerOperations.java | 8 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 11 +-
.../client/rpc/read/TestChunkInputStream.java | 13 +-
.../ozone/client/rpc/read/TestInputStreamBase.java | 44 --
.../ozone/client/rpc/read/TestKeyInputStream.java | 58 +-
.../ozone/container/TestContainerReplication.java | 47 --
.../apache/hadoop/ozone/container/TestHelper.java | 4 +-
.../container/ozoneimpl/TestOzoneContainer.java | 8 +-
.../ozone/dn/TestDatanodeLayoutUpgradeTool.java | 131 ----
.../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 4 +-
.../TestDatanodeHddsVolumeFailureToleration.java | 40 +-
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 3 -
.../org/apache/hadoop/ozone/om/TestOmMetrics.java | 28 +-
.../ozone/om/TestOzoneManagerConfiguration.java | 2 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 29 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 17 +
.../hadoop/ozone/om/TestOzoneManagerHAWithACL.java | 24 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 32 +-
.../ozone/om/TestOzoneManagerListVolumes.java | 117 ++--
.../ozone/om/TestOzoneManagerRocksDBLogging.java | 2 +-
.../apache/hadoop/ozone/om/TestScmSafeMode.java | 7 +-
.../hadoop/ozone/recon/TestReconAsPassiveScm.java | 10 +-
.../apache/hadoop/ozone/recon/TestReconTasks.java | 4 +-
.../org/apache/hadoop/ozone/scm/TestSCMMXBean.java | 4 +-
.../hadoop/ozone/scm/TestXceiverClientManager.java | 18 +-
.../scm/node/TestDecommissionAndMaintenance.java | 128 ++--
...TestScmAdminHA.java => TestNSSummaryAdmin.java} | 58 +-
hadoop-ozone/ozone-manager/pom.xml | 10 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 26 +-
.../request/bucket/OMBucketSetPropertyRequest.java | 9 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 3 +-
.../request/file/OMFileCreateRequestWithFSO.java | 5 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 7 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 3 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 7 +-
.../om/request/volume/OMVolumeSetQuotaRequest.java | 5 +-
.../ozone/om/upgrade/OMLayoutVersionManager.java | 3 +-
.../bucket/TestOMBucketSetPropertyRequest.java | 30 +-
.../om/request/key/TestOMKeyCreateRequest.java | 70 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 15 +-
.../ozone/om/request/key/TestOMKeyRequest.java | 7 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 28 +-
.../org/apache/hadoop/fs/ozone/OzoneFsShell.java | 4 +-
.../org/apache/hadoop/fs/ozone/TestOFSPath.java | 26 +
.../apache/hadoop/ozone/recon/ReconHttpServer.java | 5 +-
.../hadoop/ozone/recon/ReconRestServletModule.java | 144 ++--
.../org/apache/hadoop/ozone/recon/ReconServer.java | 26 +-
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 4 -
.../apache/hadoop/ozone/recon/api/AdminOnly.java | 23 +-
.../hadoop/ozone/recon/api/ContainerEndpoint.java | 1 +
.../hadoop/ozone/recon/api/NSSummaryEndpoint.java | 142 ++--
.../ozone/recon/api/filters/ReconAdminFilter.java | 113 ++++
.../ozone/recon/api/filters/ReconAuthFilter.java | 117 ++++
.../recon/api/{ => filters}/package-info.java | 5 +-
.../hadoop/ozone/recon/api/types/DUResponse.java | 14 +
.../ozone/recon/api/types/ResponseStatus.java | 1 +
.../ozone/recon/fsck/ContainerHealthTask.java | 6 +-
.../recon/scm/ReconContainerReportHandler.java | 4 +-
.../ozone/recon/scm/ReconDeadNodeHandler.java | 4 +-
.../ReconIncrementalContainerReportHandler.java | 4 +-
.../ozone/recon/scm/ReconPipelineManager.java | 12 +-
.../scm/ReconStorageContainerManagerFacade.java | 4 +-
.../webapps/recon/ozone-recon-web/api/db.json | 324 ++++++++-
.../webapps/recon/ozone-recon-web/api/routes.json | 22 +-
.../src/components/navBar/navBar.tsx | 5 +
.../src/components/rightDrawer/rightDrawer.tsx | 72 ++
.../src/constants/breadcrumbs.constants.tsx | 3 +-
.../webapps/recon/ozone-recon-web/src/routes.tsx | 5 +
.../insights.less => diskUsage/diskUsage.less} | 31 +-
.../src/views/diskUsage/diskUsage.tsx | 444 ++++++++++++
.../src/views/overview/overview.tsx | 3 +-
.../ozone/recon/api/TestNSSummaryEndpoint.java | 12 +-
.../ozone/recon/api/filters/TestAdminFilter.java | 161 +++++
.../ozone/recon/fsck/TestContainerHealthTask.java | 6 +-
.../ozone/admin/nssummary/DiskUsageSubCommand.java | 214 ++++++
.../admin/nssummary/FileSizeDistSubCommand.java | 129 ++++
.../ozone/admin/nssummary/NSSummaryAdmin.java | 137 ++++
.../ozone/admin/nssummary/NSSummaryCLIUtils.java | 174 +++++
.../admin/nssummary/QuotaUsageSubCommand.java | 117 ++++
.../ozone/admin/nssummary/SummarySubCommand.java | 115 ++++
.../admin/{om => nssummary}/package-info.java | 6 +-
.../admin/scm/GetScmRatisRolesSubcommand.java | 7 +-
.../apache/hadoop/ozone/debug/DatanodeLayout.java | 110 ---
.../ozone/shell/volume/CreateVolumeHandler.java | 4 +-
pom.xml | 5 +-
261 files changed, 6551 insertions(+), 3476 deletions(-)
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/tracing/TestTraceAllMethod.java
copy
hadoop-hdds/{framework/src/test/java/org/apache/hadoop/hdds/server/http/TestHttpRequestLogAppender.java
=> common/src/test/java/org/apache/hadoop/hdds/tracing/TestTracingUtil.java}
(50%)
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeMetadataFeatures.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/ScmHAFinalizeUpgradeActionDatanode.java
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/ScmHAFirstUpgradeLayoutChangeAction.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/VersionedDatanodeFeatures.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDatanodeUpgradeToScmHA.java
copy
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/http/{TestHttpRequestLogAppender.java
=> TestHttpServer2.java} (50%)
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManagerV2.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestSCMContainerManager.java
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/delete-and-regenerate-data.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/delete-data.sh
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/0.5.0-1.1.0/callback.sh
copy hadoop-ozone/dist/src/main/compose/{ozonesecure =>
upgrade/upgrades/manual-upgrade}/README.md (66%)
rename
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/non-rolling-upgrade/{1.0.0-1.1.0
=> 1.1.0-1.2.0}/callback.sh (86%)
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/0.5.0.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/1.0.0.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/1.1.0.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/versions/README.md
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterProvider.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/TestDatanodeLayoutUpgradeTool.java
copy
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/{TestScmAdminHA.java
=> TestNSSummaryAdmin.java} (56%)
copy
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditAction.java
=>
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/AdminOnly.java
(61%)
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/filters/ReconAdminFilter.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/filters/ReconAuthFilter.java
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/{ =>
filters}/package-info.java (87%)
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/components/rightDrawer/rightDrawer.tsx
copy
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/{insights/insights.less
=> diskUsage/diskUsage.less} (75%)
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/diskUsage/diskUsage.tsx
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/filters/TestAdminFilter.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/DiskUsageSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/FileSizeDistSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/NSSummaryAdmin.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/NSSummaryCLIUtils.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/QuotaUsageSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/nssummary/SummarySubCommand.java
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/{om =>
nssummary}/package-info.java (88%)
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DatanodeLayout.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
