This is an automated email from the ASF dual-hosted git repository.
sammichen pushed a change to branch HDDS-10685
in repository https://gitbox.apache.org/repos/asf/ozone.git
omit a102c43063 HDDS-11622. Support domain socket creation (#7397)
add db36e39ad0 HDDS-11822. Register subcommands in OzoneShell (#7513)
add 6ba309a359 HDDS-11831. Finer-grained interface for dynamically
registered subcommands (#7514)
add b60d897c90 HDDS-11820. Create test principals at test run time (#7507)
add 69538b0b85 HDDS-11687. Robot warning: replace "is not" with "!="
(#7516)
add 77ce962f34 HDDS-10568. When the ldb command is executed, it is output
by line (#7467)
add 9b619373b6 HDDS-11806. Add HttpFS and Recon in getting-started k8s
example (#7485)
add b27714f572 HDDS-11266. Update proto.lock for Ozone 1.4.1 (#7504)
add 9dd8bb9105 HDDS-11847. Mark
TestSnapshotDeletingServiceIntegrationTest#testParallelExcecutionOfKeyDeletionAndSnapshotDeletion
as flaky
add 1c5676bcf9 HDDS-11849. Mark
TestBlockOutputStreamWithFailures.test2DatanodesFailure as flaky
add 98e070e6ee HDDS-11728. Refactor subcommand layouts of ozone debug and
repair (#7489)
add 0bde3a293f HDDS-11410. Refactoring more tests from
TestContainerBalancerTask (#7156)
add 65df3080e3 HDDS-11794. Display HostName in OM / SCM Overview. (#7482)
add 9fcecc194a HDDS-11855. Mark
TestContainerBalancerDatanodeNodeLimit#checkIterationResultException as flaky
add 8a1967ea06 HDDS-11719. Remove dependency on server components from
ozonefs-common (#7438)
add faab1e8e5f HDDS-11656. Default native ACL limits to user and user's
primary group (#7455)
add d66c0888fb HDDS-11848. Serialization bug in Recon listKeys API (#7524)
add e17f92c4fc HDDS-10821. Ensure ChunkBuffer fully writes buffer to
FileChannel (#6652)
add cb0a402bea HDDS-11859. Remove mention of fuse from s3 interface docs
page (#7530)
add e8ad7ad27a HDDS-11782. ozone debug ldb --with-keys defaults to false
instead of true (#7521)
add fc6371039b HDDS-11833. Return NotImplemented for S3 put-object-acl
request. (#7531)
add 8526f2e81f HDDS-11826. Interactive mode for ozone shell. (#7515)
add d4c41e542e HDDS-11334. Improve EC xcompat acceptance test (#7492)
add 959a39d4d7 HDDS-11851. Finer-grained subcommand interface for
OzoneDebug and OzoneRepair. (#7526)
add 51c6ed65ad HDDS-11830. Subcommands should not extend GenericCli
.(#7537)
add af345b23df HDDS-11877. Enable Maven cache for more checks (#7538)
add 9854591537 HDDS-11882. Make BOM, not aggregate one (#7544)
add f7fe30a165 HDDS-11691. Support object tags in
ObjectEndpointStreaming#put (#7543)
add d0d82c5112 HDDS-11886. Bump license-maven-plugin to 2.5.0 (#7539)
add 055b13c8dd HDDS-11751. Use Java 21 in CI (#7458)
add 23197e26ac HDDS-11605. Directory deletion service should support
multiple threads (#7349)
add 80c6446a11 HDDS-8101. Add tool to repair broken FSO tree (#7368)
add 092b00087d HDDS-11890. Update project description in GitHub (#7547)
add a46153d221 HDDS-11872. Disable Apache snapshots repo (#7536)
add befd64e068 HDDS-11694. Safemode Improvement: Introduce factory class
to create safemode rules. (#7433)
add aa37ae8a8e HDDS-11885. Download Hadoop for S3A test from mirrors if
available (#7545)
add ec348a7e48 HDDS-11889. Include Maven dependencies for
hdds-rocks-native in cache (#7546)
add e8f3b25241 HDDS-11873. Skip old-only xcompat read tests (#7534)
add eea5600871 HDDS-11911. Return consistent error code when snapshot is
not found in the DB or Snapshot Chain. (#7557)
add 6b9cbe0e47 HDDS-11909. Intermittent timeout building Hadoop in s3a
test (#7559)
add 745ed1cd23 HDDS-11367. Improve ozone balancing status command output
(#7139)
add 85364907d6 HDDS-11367. Fix flaky balancer robot test (#7569)
add b5d04e2086 HDDS-11915. Netty OpenSsl not available in acceptance tests
on arm64 (#7570)
add 853d6572a8 HDDS-11893. Fix full snapshot diff fallback logic because
of DAG pruning (#7549)
add bf6f323109 HDDS-11413. PipelineManagerImpl lock optimization reduces
AllocateBlock latency (#7160)
add 16ba289fe3 Revert "HDDS-11413. PipelineManagerImpl lock optimization
reduces AllocateBlock latency (#7160)"
add 66ccc25280 HDDS-11908. Snapshot diff DAG traversal should not skip
node based on prefix presence (#7567)
add 1835326b16 HDDS-11927. Mark
testGetCurrentStatisticsWhileBalancingInProgress as flaky
add 7a460803f5 HDDS-11914. Snapshot diff should not filter SST Files based
by reading SST file reader (#7563)
add 008f9a61f2 HDDS-11852. Reduce duplication in some GenericCli
subclasses (#7553)
add 202b0c731b HDDS-11860. Improve BufferUtils.writeFully. (#7564)
add 1453fd922b HDDS-11935. Bump develocity-maven-extension to 1.23 (#7577)
add bef2415dbb HDDS-11940. Bump jline to 3.28.0 (#7576)
add f5e5493d01 HDDS-11927. Fix flaky
TestContainerBalancerStatusInfo.testGetCurrentStatisticsWhileBalancingInProgress
(#7579)
add daf2f9fce0 HDDS-11863. Speed up TestFSORepairTool (#7561)
add c523825f02 HDDS-11926 - Rename bucket name for bucket info/ls for
linked buckets (#7581)
add 54f0272468 HDDS-11807. Make callId different for each request in
openKeyCleanupService (#7551)
add 3648b592a9 HDDS-11906. Add sortpom dependency, sort root POM. (#7555)
add 8bb0587d37 HDDS-11712. Process other DeletedBlocksTransaction before
retrying failed one. (#7532)
add dae388b7ee HDDS-11907. OzoneSecretKey does not need to implement
Writable (#7574)
add c28e16ec68 HDDS-11950. Enable sortpom in dev-support module. (#7586)
add 976e45f334 HDDS-11711. Add SCM metrics for delete commands sent and
response received per datanode (#7522)
add a27e4ec1ac HDDS-11779. Add DN metrics to show deletion progress (#7552)
add bde8cf444d HDDS-11759. Remove LegacyReplicationManager (#7580)
add 1f29e05ea1 HDDS-11934. Split compat suite to old/new (#7578)
add 6e0c753d78 HDDS-11509. logging improvements for directory deletion
(#7314)
add 3a0e3b5364 HDDS-11845. Extract k8s definitions for HttpFS and Recon
from getting-started example (#7523)
add f5ff2f05a5 HDDS-11868. Enable OM Ratis in TestQuotaRepairTask (#7593)
add a523e17dac HDDS-11846. [Recon] Recon Schema version_number column is
always set as -1. (#7554)
add 4fe166d9dc HDDS-11957. Make breadcrumb scrollable for long path names
in DiskUsage page (#7590)
add 0066526dce HDDS-11869. Enable OM Ratis in
TestOzoneDelegationTokenSecretManager (#7594)
add 0a0cffcc6e HDDS-11964. Allow ozone sh bucket create to take obs and
fso as options. (#7599)
add 0306d97400 HDDS-11961. Improve existing repair tests (#7595)
add 7af38a9864 HDDS-11963. Add parent interface of component and layout
versions for use in request validator (#7598)
add e41ba9b81b HDDS-11982. Bump jersey2 to 2.46 (#7604)
add e25e6bf12a HDDS-11983. Bump junit to 5.11.4 (#7605)
add ed9ac7add5 HDDS-11984. Bump reload4j to 1.2.26 (#7608)
add be18a496ab HDDS-11985. Bump assertj-core to 3.27.0 (#7607)
add 16322fdfa5 Add flush to buffer (#7609)
add 5b27f6d938 HDDS-11941. Include fork timeout in integration check
summary (#7582)
add f34cf34d85 HDDS-11857. Freon log flooded by HSync message (#7613)
add f125363700 HDDS-11878. Use CommandSpec to find top-level command.
(#7575)
add ddd5433374 HDDS-11986. Enable SCM Ratis in TestSCMNodeManager (#7624)
add 345c46850f HDDS-11995. Acceptance Test test-all script fails to delete
old result directories. (#7622)
add c3003fd870 HDDS-11992. Replace GenericCli#createOzoneConfiguration
calls with getOzoneConf. (#7623)
add ad108c86b1 HDDS-11997. Duplicate snapshot purge request causes NPE
(#7627)
add f57cf017c7 HDDS-11993. Move OzoneAdmin to ozone-tools (#7619)
add 14756bf4aa HDDS-11865. Remove tests for non-Ratis OM (#7535)
add f2e6d38ad7 HDDS-11994. Convert Freon to pluggable model (#7620)
add 0d1a0ce1f0 HDDS-11880. Intermediate subcommands do not need to
implement Callable (#7617)
add 9aae7a5c0b HDDS-9791. Add tests for Datanodes page (#7626)
add 9b5cedd0f9 HDDS-11951. Enable sortpom in hadoop-hdds sub-modules :
annotations, client, common & config. (#7631)
add c282d91b78 HDDS-8175. getFileChecksum() throws exception in debug
mode. (#7611)
add 4121aa21f6 HDDS-11726. Add leader readiness state to OM UI (#7628)
add 47c1b3584b HDDS-12001. Create parent class for repair tools (#7633)
add bf112aa36b HDDS-11990. Use arity in decommission subcommands (#7618)
add fbb792d46d HDDS-11945. Improve startup message for ozone repair
commands (#7591)
add 69206e9681 HDDS-11902. Remove upgrade tests for non-ha and om-ha
(#7610)
add d013188032 HDDS-11727. Block `ozone repair om fso-tree` if OM is
running (#7589)
add 99fd5fcbb5 HDDS-11801. Logs missing if kubernetes check fails before
tests (#7496)
add 60a7fdb628 HDDS-11819. Improve mock datanode version handling in
MiniOzoneCluster (#7632)
add ce6fb458e3 HDDS-11999. Remove `ozone repair ldb command` and move its
subcommands to `ozone repair om` (#7634)
add 2dd8a71cfa HDDS-11731. ContainerSafeModeRule Refactor (#7587)
add 91d95371dc HDDS-12013. Bump sqlite-jdbc to 3.47.2.0 (#7641)
add df6e119f4c HDDS-12006. Enable sortpom in hdds-container-service,
hdds-crypto-api, hdds-crypto-default, hdds-docs (#7638)
add de79c37c61 HDDS-12014. Bump assertj-core to 3.27.1 (#7642)
add 138cbe3157 HDDS-11998. BlockDataStreamOutput should
decrPendingContainerOpsMetrics (#7636)
add 53fc590878 HDDS-12019. Bump zstd-jni to 1.5.6-9 (#7643)
add 24a721182e HDDS-7188. Read chunk files using netty ChunkedNioFile.
(#7625)
add 6280c6643d HDDS-11952. Enable sortpom in hadoop-ozone. (#7646)
add f9bd055667 HDDS-12020. Enable sortpom in hdds-erasurecode,
hdds-server-framework, hdds-hadoop-dependency-client. (#7644)
add 71de2a200a HDDS-10469. Ozone Manager should continue to work when S3
secret storage is unavailable (#6339)
add 8f2689dfe5 HDDS-11864. Remove config from OM for disabling Ratis
(#7640)
add ab161dd72a HDDS-11949. Ozone Recon - Update Recon OM Sync default
configs and docker configs. (#7600)
add 3d35b01e62 HDDS-12011. Show PID of running service. (#7648)
add 2ec05cb122 HDDS-11987. Remove duplicate Quota In Bytes field from DU
metadata (#7649)
add 5354cec5d4 HDDS-12027. Mark
TestBlockDataStreamOutput#testMultiBlockWrite as flaky
add ae9a56fc47 HDDS-11991. Use picocli built-in for missing subcommand of
GenericCli (#7635)
add 6b8b844dfa HDDS-12009. Merge FSORepairTool and FSORepairCLI (#7639)
add 8a774a57df HDDS-11989. Enable SCM Ratis in tests related to
DeletedBlockLog (#7615)
add 44ba9a3f5d HDDS-12023. Enable SCM Ratis in TestContainerCommandsEC
(#7650)
add e8d96f422e HDDS-12022. Enable SCM Ratis in TestStorageContainerManager
(#7651)
add 3dfd2410a0 HDDS-11753. Deprecate file per chunk layout from datanode
code. (#7654)
add 984027cc25 HDDS-12035. Enable sortpom in hdds-hadoop-dependency-server
and -test (#7659)
add 2fc9c6e5f1 HDDS-12034. Enable sortpom in hdds-interface-admin, -client
and -server. (#7660)
add 36a430db45 HDDS-7307. Move S3 Gateway web content to separate port
(#6898)
add f8394cf5b7 HDDS-11975. wrap TermIndex in ExecutionContext (#7602)
add 14bba1ee7a HDDS-5045. Create acceptance test for using rclone with s3
protocol (#7571)
add a1324b66a7 HDDS-11511. Introduce metrics in deletion services of OM
(#7377)
add ab29a55eae HDDS-7990. Add acceptance test for HA Proxy with secure
Ozone S3 Gateway (#7562)
add ae1843f0f5 HDDS-12045. S3 secret admin test fails with HAProxy (#7668)
add 400e94d543 HDDS-11959. Remove tests for non-Ratis SCM (#7612)
add e21e724d75 HDDS-12003. Reduce code duplication related to tracing init
(#7658)
add f1f0ec323d HDDS-12007. BlockDataStreamOutput should only send one
PutBlock during close. (#7645)
add 80dc87a652 HDDS-12039. Move quota subcommands under ozone repair om
(#7662)
add a4f48211b3 HDDS-11969. getFilechecksum() API fails if checksum type is
NONE. (#7656)
add 990b5bf8c9 HDDS-12038. Bump maven-remote-resources-plugin to 3.3.0
(#7661)
add 93dab91667 HDDS-11816. Ozone stream to support Hsync,Hflush. (#7592)
add 9670965428 HDDS-11699. Remove unnecessary information about parts when
downloading multipart files. (#7558)
add 76ac396611 HDDS-12036. Add storage indicators when reaching capacity
(#7663)
add 49efad97a7 HDDS-12012. Defer ozone repair prompt after subcommand
validation (#7653)
add 7c13de865e HDDS-12041. Add ozone repair scm cert command and its
subcommand (#7671)
add 9c391bc29e HDDS-12049. Rename OM ID and OM Service ID to Ozone Service
ID. (#7679)
add da8fa24791 HDDS-12054. Move ozone debug prefix to ozone debug om
prefix (#7674)
add 62f24bd3bd HDDS-12066. Bump jetty to 9.4.57.v20241219 (#7683)
add 9b8ef74573 HDDS-12043. Mark fixed column with disabled checkbox (#7667)
add 75bed82a76 HDDS-12067. Bump assertj-core to 3.27.2 (#7684)
add 1d95f67e8d HDDS-12055. Move ozone debug container to ozone debug
datanode container (#7675)
add 19c8136afe HDDS-11962. [Docs] Hive Integration (#7596)
add 468c35d76c HDDS-11947. [Docs] Impala Integration (#7584)
add b89b6e021d HDDS-12058. Use CommandLine out/err in GenericCli
subclasses (#7673)
add 12def8a0f1 HDDS-11326. Speed up TestBlockOutputStreamWithFailures
(#7374)
add 67fdb88b95 HDDS-12068. Enable sortpom in remaining hdds modules (#7686)
add c387656530 HDDS-12063. Speed up TestLeaseRecovery (#7688)
add 2516ea6eba HDDS-12002. Move up out() and err() to AbstractSubcommand
(#7687)
new 508e2eaa50 HDDS-11622. Support domain socket creation (#7397)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a102c43063)
\
N -- N -- N refs/heads/HDDS-10685 (508e2eaa50)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 2 +-
.github/workflows/ci.yml | 32 +-
.github/workflows/populate-cache.yml | 2 +-
.mvn/extensions.xml | 2 +-
dev-support/ci/maven-settings.xml | 35 +
dev-support/pom.xml | 30 +-
hadoop-hdds/annotations/pom.xml | 15 +-
hadoop-hdds/client/pom.xml | 60 +-
.../apache/hadoop/hdds/scm/OzoneClientConfig.java | 12 +-
.../hdds/scm/storage/BlockDataStreamOutput.java | 25 +-
.../hdds/scm/storage/ByteBufferStreamOutput.java | 4 +-
.../hadoop/hdds/scm/storage/StreamBuffer.java | 4 +-
hadoop-hdds/common/pom.xml | 216 +-
.../org/apache/hadoop/hdds/ComponentVersion.java | 9 +-
.../hadoop/hdds/cli/ExtensibleParentCommand.java | 60 +
.../org/apache/hadoop/hdds/cli/GenericCli.java | 98 +-
.../hadoop/hdds/cli/GenericParentCommand.java | 3 +-
.../hdds/cli/MissingSubcommandException.java | 31 -
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 3 +
.../ContainerCommandResponseBuilders.java | 4 +-
.../apache/hadoop/hdds/utils/db/CodecBuffer.java | 6 +-
.../org/apache/hadoop/ozone/ClientVersion.java | 5 +-
.../apache/hadoop/ozone/OzoneManagerVersion.java | 3 +
.../Versioned.java} | 16 +-
.../apache/hadoop/ozone/common/ChecksumCache.java | 2 +-
.../apache/hadoop/ozone/common/ChunkBuffer.java | 55 +-
.../common/ChunkBufferImplWithByteBuffer.java | 3 +-
.../common/ChunkBufferImplWithByteBufferList.java | 6 +-
.../ozone/common/ChunkBufferToByteString.java | 93 +
.../common/ChunkBufferToByteStringByByteBufs.java | 101 +
.../ozone/common/IncrementalChunkBuffer.java | 7 +-
.../hadoop/ozone/common/utils/BufferUtils.java | 50 +
.../apache/hadoop/ozone/upgrade/LayoutFeature.java | 9 +-
.../common/src/main/resources/ozone-default.xml | 57 +-
.../hadoop/hdds/utils/MockGatheringChannel.java | 67 +-
hadoop-hdds/config/pom.xml | 12 +-
hadoop-hdds/container-service/pom.xml | 161 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 25 +-
.../helpers/BlockDeletingServiceMetrics.java | 32 +-
.../common/impl/BlockDeletingService.java | 9 +-
.../common/impl/ContainerLayoutVersion.java | 3 +-
.../commandhandler/DeleteBlocksCommandHandler.java | 8 +-
.../transport/server/GrpcXceiverService.java | 13 +-
.../server/ratis/ContainerStateMachine.java | 40 +-
.../transport/server/ratis/DispatcherContext.java | 30 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 27 +-
.../container/keyvalue/helpers/ChunkUtils.java | 40 +-
.../container/keyvalue/impl/BlockManagerImpl.java | 12 +-
.../keyvalue/impl/ChunkManagerDispatcher.java | 7 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 11 +-
.../keyvalue/impl/FilePerChunkStrategy.java | 10 +-
.../keyvalue/impl/KeyValueStreamDataChannel.java | 65 +-
.../keyvalue/interfaces/BlockManager.java | 3 +
.../keyvalue/interfaces/ChunkManager.java | 3 +-
.../common/impl/TestContainerPersistence.java | 6 +-
.../container/keyvalue/TestKeyValueHandler.java | 8 +
.../keyvalue/impl/TestChunkManagerDummyImpl.java | 4 +-
.../keyvalue/impl/TestFilePerBlockStrategy.java | 13 +-
.../impl/TestKeyValueStreamDataChannel.java | 65 +-
.../upgrade/TestDatanodeUpgradeToScmHA.java | 604 ----
hadoop-hdds/crypto-api/pom.xml | 35 +-
hadoop-hdds/crypto-default/pom.xml | 35 +-
hadoop-hdds/docs/content/feature/OM-HA.md | 8 -
hadoop-hdds/docs/content/feature/OM-HA.zh.md | 9 -
hadoop-hdds/docs/content/integration/Hive.md | 169 +
hadoop-hdds/docs/content/integration/Impala.md | 101 +
hadoop-hdds/docs/content/integration/_index.md | 26 +
hadoop-hdds/docs/content/interface/CSI.md | 2 +-
hadoop-hdds/docs/content/interface/S3.md | 7 -
hadoop-hdds/docs/content/interface/S3.zh.md | 7 -
hadoop-hdds/docs/pom.xml | 12 +-
hadoop-hdds/erasurecode/pom.xml | 29 +-
hadoop-hdds/framework/pom.xml | 279 +-
.../hadoop/hdds/security/OzoneSecretKey.java | 53 +-
.../hadoop/hdds/server/http/BaseHttpServer.java | 22 +-
.../hadoop/hdds/server/http/HttpServer2.java | 47 +-
hadoop-hdds/hadoop-dependency-client/pom.xml | 231 +-
hadoop-hdds/hadoop-dependency-server/pom.xml | 123 +-
hadoop-hdds/hadoop-dependency-test/pom.xml | 29 +-
hadoop-hdds/interface-admin/pom.xml | 20 +-
.../src/main/proto/ScmAdminProtocol.proto | 21 +-
hadoop-hdds/interface-client/pom.xml | 77 +-
.../interface-client/src/main/resources/proto.lock | 12 +
hadoop-hdds/interface-server/pom.xml | 57 +-
.../interface-server/src/main/resources/proto.lock | 24 +
hadoop-hdds/managed-rocksdb/pom.xml | 31 +-
.../hdds/utils/db/managed/ManagedRocksDB.java | 13 +
hadoop-hdds/pom.xml | 106 +-
hadoop-hdds/rocks-native/pom.xml | 121 +-
hadoop-hdds/rocksdb-checkpoint-differ/pom.xml | 69 +-
.../ozone/compaction/log/CompactionFileInfo.java | 12 +
.../org/apache/ozone/rocksdiff/CompactionNode.java | 7 +
.../ozone/rocksdiff/RocksDBCheckpointDiffer.java | 203 +-
.../org/apache/ozone/rocksdiff/RocksDiffUtils.java | 96 +-
.../rocksdiff/TestRocksDBCheckpointDiffer.java | 299 +-
.../apache/ozone/rocksdiff/TestRocksDiffUtils.java | 121 +
hadoop-hdds/server-scm/pom.xml | 195 +-
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 51 +-
.../scm/block/DeletedBlockLogStateManagerImpl.java | 3 +-
.../hdds/scm/block/SCMBlockDeletingService.java | 7 +-
.../scm/block/ScmBlockDeletingServiceMetrics.java | 143 +-
.../hdds/scm/container/ContainerManager.java | 3 +
.../hdds/scm/container/ContainerManagerImpl.java | 7 +
.../hdds/scm/container/ContainerStateManager.java | 7 +
.../scm/container/ContainerStateManagerImpl.java | 8 +
.../balancer/AbstractFindTargetGreedy.java | 5 +
.../balancer/ContainerBalancerMetrics.java | 52 +-
.../ContainerBalancerSelectionCriteria.java | 16 +-
.../balancer/ContainerBalancerStatusInfo.java | 19 +
.../container/balancer/ContainerBalancerTask.java | 268 +-
.../ContainerBalancerTaskIterationStatusInfo.java | 166 +-
.../scm/container/balancer/ContainerMoveInfo.java | 60 +
.../hdds/scm/container/balancer/DataMoveInfo.java | 60 +
.../scm/container/balancer/FindSourceGreedy.java | 5 +
.../scm/container/balancer/FindSourceStrategy.java | 9 +
.../scm/container/balancer/FindTargetStrategy.java | 9 +
.../hdds/scm/container/balancer/IterationInfo.java | 39 +-
.../replication/ContainerReplicaCount.java | 2 +-
.../LegacyRatisContainerReplicaCount.java | 142 -
.../replication/LegacyReplicationManager.java | 2581 ---------------
.../replication/RatisContainerReplicaCount.java | 6 +-
.../container/replication/ReplicationManager.java | 96 +-
.../replication/ReplicationManagerMetrics.java | 50 +-
.../scm/container/states/ContainerStateMap.java | 2 +-
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 2 -
.../hdds/scm/node/DatanodeAdminMonitorImpl.java | 18 +-
.../hdds/scm/safemode/ContainerSafeModeRule.java | 163 +-
.../hdds/scm/safemode/SCMSafeModeManager.java | 45 +-
.../hadoop/hdds/scm/safemode/SafeModeExitRule.java | 12 +
.../hdds/scm/safemode/SafeModeRuleFactory.java | 131 +
.../hdds/scm/server/SCMClientProtocolServer.java | 45 +-
.../apache/hadoop/hdds/scm/server/SCMMXBean.java | 7 +
.../hdds/scm/server/StorageContainerManager.java | 18 +-
.../scm/server/StorageContainerManagerStarter.java | 9 +-
.../hadoop/hdds/scm/block/TestBlockManager.java | 3 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 71 +-
.../balancer/ContainerBalancerConfigBuilder.java | 46 +
.../hdds/scm/container/balancer/MockedSCM.java | 37 +-
.../TestContainerBalancerDatanodeNodeLimit.java | 306 +-
.../balancer/TestContainerBalancerStatusInfo.java | 191 +-
.../balancer/TestContainerBalancerTask.java | 285 +-
.../replication/TestLegacyReplicationManager.java | 3487 --------------------
.../replication/TestReplicationManager.java | 3 -
.../replication/TestReplicationManagerMetrics.java | 6 -
.../TestReplicationManagerScenarios.java | 3 -
...onfiguration.java => TestSCMConfiguration.java} | 73 +-
.../ha/TestStatefulServiceStateManagerImpl.java | 2 -
.../hdds/scm/node/TestDatanodeAdminMonitor.java | 147 -
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 5 +-
.../hdds/scm/pipeline/TestPipelineManagerImpl.java | 7 +-
.../safemode/TestHealthyPipelineSafeModeRule.java | 15 +-
.../TestOneReplicaPipelineSafeModeRule.java | 8 +-
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 49 +-
.../hdds/scm/safemode/TestSafeModeRuleFactory.java | 85 +
.../TestSCMHAUnfinalizedStateValidationAction.java | 54 +-
hadoop-hdds/test-utils/pom.xml | 47 +-
.../org/apache/ozone/test/GenericTestUtils.java | 127 +-
.../main/java/org/apache/ozone/test/IntLambda.java | 29 +-
hadoop-hdds/tools/pom.xml | 139 +-
.../org/apache/hadoop/hdds/cli/AbstractMixin.java | 42 +-
.../apache/hadoop/hdds/cli/AbstractSubcommand.java | 89 +
.../apache/hadoop/hdds/cli/AdminSubcommand.java | 9 +-
.../apache/hadoop/hdds/cli/DebugSubcommand.java | 9 +-
.../org/apache/hadoop/hdds/cli/OzoneAdmin.java | 82 -
.../apache/hadoop/hdds/cli/RepairSubcommand.java | 9 +-
.../hdds/scm/cli/ContainerBalancerCommands.java | 26 +-
.../scm/cli/ContainerBalancerStatusSubcommand.java | 71 +-
.../hdds/scm/cli/ReplicationManagerCommands.java | 26 +-
.../hadoop/hdds/scm/cli/SafeModeCommands.java | 25 +-
.../org/apache/hadoop/hdds/scm/cli/ScmOption.java | 17 +-
.../apache/hadoop/hdds/scm/cli/ScmSubcommand.java | 3 +-
.../hadoop/hdds/scm/cli/TopologySubcommand.java | 12 +-
.../hadoop/hdds/scm/cli/cert/CertCommands.java | 26 +-
.../hdds/scm/cli/container/ContainerCommands.java | 33 +-
.../hdds/scm/cli/container/InfoSubcommand.java | 11 +-
.../hdds/scm/cli/container/ListSubcommand.java | 6 +-
.../hdds/scm/cli/container/UpgradeSubcommand.java | 15 +-
.../hdds/scm/cli/datanode/DatanodeCommands.java | 26 +-
.../scm/cli/datanode/DecommissionSubCommand.java | 54 +-
.../hdds/scm/cli/datanode/HostNameParameters.java | 53 +
.../scm/cli/datanode/MaintenanceSubCommand.java | 50 +-
.../scm/cli/datanode/RecommissionSubCommand.java | 48 +-
.../hdds/scm/cli/datanode/StatusSubCommand.java | 21 +-
.../hdds/scm/cli/pipeline/PipelineCommands.java | 25 +-
.../org/apache/hadoop/hdds/util/DurationUtil.java | 52 +
.../org/apache/hadoop/hdds/util}/package-info.java | 12 +-
.../datanode/TestContainerBalancerSubCommand.java | 419 ++-
.../apache/hadoop/hdds/util/TestDurationUtil.java | 92 +
hadoop-ozone/client/pom.xml | 1 +
.../client/checksum/BaseFileChecksumHelper.java | 35 +-
.../client/checksum/ECFileChecksumHelper.java | 29 -
.../checksum/ReplicatedFileChecksumHelper.java | 46 -
.../client/io/BlockDataStreamOutputEntry.java | 12 +
.../client/io/BlockDataStreamOutputEntryPool.java | 29 +
.../ozone/client/io/KeyDataStreamOutput.java | 20 +-
.../ozone/client/io/OzoneDataStreamOutput.java | 70 +-
.../hadoop/ozone/client/io/OzoneOutputStream.java | 6 +
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 145 +-
hadoop-ozone/common/pom.xml | 1 +
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 3 +-
.../java/org/apache/hadoop/ozone/OzoneAcl.java | 11 +
.../ozone/client/io/SelectorOutputStream.java | 17 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 9 +-
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 11 +-
.../hadoop/ozone/om/helpers/OzoneAclUtil.java | 51 +-
.../hadoop/ozone/om/helpers/OzoneFSUtils.java | 2 +-
...OzoneManagerProtocolClientSideTranslatorPB.java | 32 +-
.../hadoop/ozone/security/acl/OzoneAclConfig.java | 28 +-
.../hadoop/ozone/om/helpers/TestOzoneAclUtil.java | 4 +-
hadoop-ozone/csi/pom.xml | 1 +
.../org/apache/hadoop/ozone/csi/CsiServer.java | 2 +-
hadoop-ozone/datanode/pom.xml | 1 +
hadoop-ozone/dev-support/checks/_lib.sh | 2 +-
.../dev-support/checks/_mvn_unit_report.sh | 10 +-
.../intellij/runConfigurations/ScmRoles.xml | 2 +-
.../k8s/regenerate-examples.sh} | 16 +-
hadoop-ozone/dist/pom.xml | 5 +-
.../dist/src/main/compose/common/init-kdc.sh | 3 +-
.../src/main/compose/common/s3-haproxy-secure.yaml | 57 +
.../dist/src/main/compose/common/s3-haproxy.cfg | 10 +
.../dist/src/main/compose/common/s3a-test.sh | 2 +-
.../dist/src/main/compose/compatibility/test.sh | 2 +-
.../src/main/compose/ozone-balancer/docker-config | 4 +-
.../dist/src/main/compose/ozone-ha/docker-config | 1 -
.../src/main/compose/ozone-om-ha/docker-config | 1 -
.../main/compose/ozone-om-prepare/docker-config | 1 -
.../src/main/compose/ozone/docker-compose.yaml | 2 +
.../dist/src/main/compose/ozone/prometheus.yml | 4 +-
.../src/main/compose/ozonesecure-ha/docker-config | 1 -
.../test.sh => ozonesecure/test-haproxy-s3g.sh} | 33 +-
hadoop-ozone/dist/src/main/compose/test-all.sh | 4 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 36 +-
.../dist/src/main/compose/upgrade/README.md | 18 +-
.../main/compose/upgrade/compose/ha/docker-config | 1 -
.../upgrade/compose/non-ha/docker-compose.yaml | 155 -
.../compose/upgrade/compose/non-ha/docker-config | 49 -
.../upgrade/compose/om-ha/docker-compose.yaml | 178 -
.../compose/upgrade/compose/om-ha/docker-config | 65 -
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 2 +-
.../upgrade/upgrades/manual-upgrade/README.md | 19 -
.../upgrade/upgrades/manual-upgrade/driver.sh | 54 -
.../dist/src/main/compose/xcompat/clients.yaml | 89 +-
hadoop-ozone/dist/src/main/compose/xcompat/lib.sh | 114 +
.../compose/om-ha/load.sh => xcompat/test-new.sh} | 17 +-
.../compose/om-ha/.env => xcompat/test-old.sh} | 25 +-
hadoop-ozone/dist/src/main/compose/xcompat/test.sh | 157 -
.../src/main/k8s/definitions/ozone/config.yaml | 18 +-
.../ozone/httpfs-ss-service.yaml} | 23 +-
.../definitions/ozone/httpfs-ss.yaml} | 41 +-
.../ozone/recon-ss-service.yaml} | 23 +-
.../definitions/ozone/recon-ss.yaml} | 51 +-
hadoop-ozone/dist/src/main/k8s/examples/README.md | 22 +
.../examples/getting-started/config-configmap.yaml | 16 +-
.../getting-started/httpfs-public-service.yaml} | 18 +-
.../httpfs-service.yaml} | 22 +-
.../getting-started/httpfs-statefulset.yaml | 55 +
.../examples/getting-started/kustomization.yaml | 6 +
.../recon-public-service.yaml} | 22 +-
.../recon-service.yaml} | 22 +-
.../getting-started/recon-statefulset.yaml | 63 +
.../k8s/examples/minikube/config-configmap.yaml | 16 +-
.../httpfs-public-service.yaml} | 22 +-
.../httpfs-service.yaml} | 22 +-
.../k8s/examples/minikube/httpfs-statefulset.yaml | 55 +
.../main/k8s/examples/minikube/kustomization.yaml | 6 +
.../recon-public-service.yaml} | 22 +-
.../recon-service.yaml} | 22 +-
.../k8s/examples/minikube/recon-statefulset.yaml | 63 +
.../k8s/examples/ozone-dev/config-configmap.yaml | 16 +-
.../httpfs-public-service.yaml} | 22 +-
.../httpfs-service.yaml} | 22 +-
.../k8s/examples/ozone-dev/httpfs-statefulset.yaml | 62 +
.../main/k8s/examples/ozone-dev/kustomization.yaml | 6 +
.../recon-public-service.yaml} | 22 +-
.../recon-service.yaml} | 22 +-
.../k8s/examples/ozone-dev/recon-statefulset.yaml | 69 +
.../k8s/examples/ozone-ha/config-configmap.yaml | 14 +-
.../httpfs-service.yaml} | 22 +-
.../k8s/examples/ozone-ha/httpfs-statefulset.yaml | 61 +
.../main/k8s/examples/ozone-ha/kustomization.yaml | 4 +
.../recon-service.yaml} | 22 +-
.../k8s/examples/ozone-ha/recon-statefulset.yaml | 70 +
.../main/k8s/examples/ozone/config-configmap.yaml | 16 +-
.../{kustomization.yaml => httpfs-service.yaml} | 22 +-
.../k8s/examples/ozone/httpfs-statefulset.yaml | 61 +
.../src/main/k8s/examples/ozone/kustomization.yaml | 4 +
.../{kustomization.yaml => recon-service.yaml} | 22 +-
.../main/k8s/examples/ozone/recon-statefulset.yaml | 70 +
.../dist/src/main/k8s/examples/test-all.sh | 21 +-
hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 2 +
hadoop-ozone/dist/src/main/license/jar-report.txt | 2 +
.../dist/src/main/smoketest/admincli/admin.robot | 2 +-
.../src/main/smoketest/admincli/container.robot | 2 +-
.../src/main/smoketest/admincli/datanode.robot | 2 +-
.../src/main/smoketest/admincli/pipeline.robot | 2 +-
.../smoketest/admincli/replicationmanager.robot | 2 +-
.../src/main/smoketest/admincli/safemode.robot | 2 +-
.../src/main/smoketest/balancer/testBalancer.robot | 50 +-
.../smoketest/compatibility/dn-one-rocksdb.robot | 2 +-
.../src/main/smoketest/compatibility/read.robot | 132 +-
.../src/main/smoketest/compatibility/setup.robot | 9 +-
.../src/main/smoketest/compatibility/write.robot | 42 +-
.../dist/src/main/smoketest/createmrenv.robot | 2 +-
.../src/main/smoketest/debug/ozone-debug-ldb.robot | 12 +-
.../src/main/smoketest/ec/backward-compat.robot | 107 -
.../dist/src/main/smoketest/ec/lib.resource | 12 -
.../dist/src/main/smoketest/ozone-lib/shell.robot | 14 +-
.../dist/src/main/smoketest/recon/recon-api.robot | 12 +
.../dist/src/main/smoketest/s3/bucketcreate.robot | 21 +-
.../dist/src/main/smoketest/s3/commonawslib.robot | 14 +
.../dist/src/main/smoketest/s3/objectputget.robot | 39 +-
.../dist/src/main/smoketest/s3/rclone.robot | 46 +
.../src/main/smoketest/s3/s3_compatbility_check.sh | 1 +
.../src/main/smoketest/s3/secretgenerate.robot | 2 +-
.../dist/src/main/smoketest/s3/secretrevoke.robot | 3 +-
.../dist/src/main/smoketest/s3/webui.robot | 13 +-
.../dist/src/main/smoketest/spnego/web.robot | 2 +-
hadoop-ozone/dist/src/shell/ozone/ozone | 24 +-
hadoop-ozone/fault-injection-test/pom.xml | 4 +
hadoop-ozone/httpfsgateway/pom.xml | 1 +
hadoop-ozone/insight/pom.xml | 1 +
.../ozone/insight/ConfigurationSubCommand.java | 2 +-
.../org/apache/hadoop/ozone/insight/Insight.java | 4 -
.../apache/hadoop/ozone/insight/LogSubcommand.java | 2 +-
.../hadoop/ozone/insight/MetricsSubCommand.java | 2 +-
hadoop-ozone/integration-test/pom.xml | 4 +
.../fs/ozone/AbstractOzoneFileSystemTest.java | 82 +-
.../ozone/AbstractOzoneFileSystemTestWithFSO.java | 4 +-
.../ozone/AbstractRootedOzoneFileSystemTest.java | 24 +-
.../AbstractRootedOzoneFileSystemTestWithFSO.java | 4 +-
.../ozone/TestDirectoryDeletingServiceWithFSO.java | 82 +-
.../java/org/apache/hadoop/fs/ozone/TestHSync.java | 97 +-
.../apache/hadoop/fs/ozone/TestHSyncUpgrade.java | 2 -
.../apache/hadoop/fs/ozone/TestLeaseRecovery.java | 239 +-
.../java/org/apache/hadoop/fs/ozone/TestO3FS.java | 2 +-
.../apache/hadoop/fs/ozone/TestO3FSWithFSO.java | 2 +-
.../hadoop/fs/ozone/TestO3FSWithFSOAndOMRatis.java | 27 -
.../hadoop/fs/ozone/TestO3FSWithFSPaths.java | 2 +-
.../fs/ozone/TestO3FSWithFSPathsAndOMRatis.java | 28 -
.../hadoop/fs/ozone/TestO3FSWithOMRatis.java | 28 -
.../java/org/apache/hadoop/fs/ozone/TestOFS.java | 2 +-
.../hadoop/fs/ozone/TestOFSWithCacheOnly.java | 28 -
.../org/apache/hadoop/fs/ozone/TestOFSWithFSO.java | 2 +-
.../fs/ozone/TestOFSWithFSOAndCacheOnly.java | 27 -
.../hadoop/fs/ozone/TestOFSWithFSOAndOMRatis.java | 27 -
.../TestOFSWithFSOAndOMRatisAndCacheOnly.java | 27 -
.../apache/hadoop/fs/ozone/TestOFSWithFSPaths.java | 2 +-
.../fs/ozone/TestOFSWithFSPathsAndOMRatis.java | 28 -
.../ozone/TestOFSWithFSPathsAndOMRatisAndACL.java | 28 -
.../TestOFSWithFSPathsAndOMRatisAndCacheOnly.java | 28 -
.../apache/hadoop/fs/ozone/TestOFSWithOMRatis.java | 28 -
.../fs/ozone/TestOzoneFileSystemPrefixParser.java | 2 +-
.../fs/ozone/TestOzoneFileSystemWithStreaming.java | 8 +-
.../hadoop/fs/ozone/TestRootedDDSWithFSO.java | 1 -
.../hadoop/hdds/scm/TestFailoverWithSCMHA.java | 91 -
.../hadoop/hdds/scm/TestSCMInstallSnapshot.java | 3 +-
.../apache/hadoop/hdds/scm/TestSCMSnapshot.java | 1 -
.../hdds/scm/TestStorageContainerManager.java | 171 +-
.../metrics/TestSCMContainerManagerMetrics.java | 1 +
.../hdds/scm/storage/TestContainerCommandsEC.java | 7 +-
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 28 -
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 45 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 3 -
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 33 +
.../apache/hadoop/ozone/TestBlockTokensCLI.java | 8 +-
.../hadoop/ozone/TestMultipartObjectGet.java | 2 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 2 +
.../hadoop/ozone/TestSecureOzoneCluster.java | 43 +-
.../hadoop/ozone/UniformDatanodesFactory.java | 26 +
.../ozone/client/rpc/OzoneRpcClientTests.java | 112 +-
.../client/rpc/TestBlockDataStreamOutput.java | 94 +-
.../ozone/client/rpc/TestBlockOutputStream.java | 7 +-
.../rpc/TestBlockOutputStreamWithFailures.java | 30 +-
.../client/rpc/TestContainerStateMachine.java | 2 +
.../ozone/client/rpc/TestDatanodeVersion.java | 143 -
.../client/rpc/TestDeleteWithInAdequateDN.java | 3 +
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 157 +-
.../ozone/client/rpc/TestOzoneRpcClient.java | 4 +-
.../client/rpc/TestOzoneRpcClientWithRatis.java | 55 -
.../ozone/container/TestContainerReplication.java | 1 -
.../commandhandler/TestBlockDeletion.java | 59 +-
.../commandhandler/TestDeleteContainerHandler.java | 6 +
.../org/apache/hadoop/ozone/debug/TestLDBCli.java | 90 +-
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +-
.../hadoop/ozone/freon/TestOMSnapshotDAG.java | 15 +-
.../ozone/om/TestBucketLayoutWithOlderClient.java | 4 +
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 108 +-
.../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 -
.../hadoop/ozone/om/TestOMDbCheckpointServlet.java | 8 -
.../hadoop/ozone/om/TestOMEpochForNonRatis.java | 215 --
.../ozone/om/TestOzoneManagerConfiguration.java | 1 -
.../hadoop/ozone/om/TestRecursiveAclWithFSO.java | 79 +
.../om/ratis/TestOzoneManagerRatisRequest.java | 18 +-
.../hadoop/ozone/om/snapshot/TestOmSnapshot.java | 9 +-
.../snapshot/TestOzoneManagerSnapshotProvider.java | 1 -
...TestSnapshotDeletingServiceIntegrationTest.java | 11 +-
.../ozone/parser/TestOzoneHARatisLogParser.java | 4 +-
.../TestReconInsightsForDeletedDirectories.java | 4 -
.../hadoop/ozone/recon/TestReconScmHASnapshot.java | 65 -
.../ozone/recon/TestReconScmNonHASnapshot.java | 64 -
.../apache/hadoop/ozone/recon/TestReconTasks.java | 1 +
.../ozone/recon/TestReconWithOzoneManagerHA.java | 2 -
.../hadoop/ozone/repair/om/TestFSORepairTool.java | 534 +++
.../ozone/s3/awssdk/v1/AbstractS3SDKV1Tests.java | 105 +
.../hadoop/ozone/s3/awssdk/v1/TestS3SDKV1.java | 4 +-
...tis.java => TestS3SDKV1WithRatisStreaming.java} | 12 +-
.../ozone/shell/TestDeletedBlocksTxnShell.java | 2 -
.../hadoop/ozone/shell/TestNSSummaryAdmin.java | 8 +-
.../hadoop/ozone/shell/TestOzoneDebugShell.java | 10 +-
.../hadoop/ozone/shell/TestOzoneRepairShell.java | 109 +-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 4 +-
.../hadoop/ozone/shell/TestOzoneTenantShell.java | 48 +-
.../hadoop/ozone/shell/TestReconfigShell.java | 6 +-
.../apache/hadoop/ozone/shell/TestScmAdminHA.java | 2 +-
.../ozone/shell/TestTransferLeadershipShell.java | 8 +-
hadoop-ozone/interface-client/pom.xml | 1 +
.../interface-client/src/main/resources/proto.lock | 136 +-
hadoop-ozone/interface-storage/pom.xml | 1 +
hadoop-ozone/ozone-manager/pom.xml | 1 +
.../hadoop/ozone/om/DeletingServiceMetrics.java | 163 +
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 32 +-
.../java/org/apache/hadoop/ozone/om/OMMXBean.java | 6 +
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 7 +-
.../apache/hadoop/ozone/om/OmSnapshotManager.java | 7 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 24 +-
.../hadoop/ozone/om/OzoneManagerStarter.java | 5 +-
.../execution/flowcontrol/ExecutionContext.java} | 42 +-
.../om/execution/flowcontrol/package-info.java} | 7 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 4 +-
.../hadoop/ozone/om/request/OMClientRequest.java | 14 +-
.../om/request/bucket/OMBucketCreateRequest.java | 19 +-
.../om/request/bucket/OMBucketDeleteRequest.java | 6 +-
.../om/request/bucket/OMBucketSetOwnerRequest.java | 6 +-
.../request/bucket/OMBucketSetPropertyRequest.java | 6 +-
.../om/request/bucket/acl/OMBucketAclRequest.java | 6 +-
.../request/bucket/acl/OMBucketAddAclRequest.java | 6 +-
.../bucket/acl/OMBucketRemoveAclRequest.java | 6 +-
.../request/bucket/acl/OMBucketSetAclRequest.java | 6 +-
.../om/request/file/OMDirectoryCreateRequest.java | 136 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 95 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 11 +-
.../request/file/OMFileCreateRequestWithFSO.java | 13 +-
.../om/request/file/OMRecoverLeaseRequest.java | 6 +-
.../om/request/key/OMAllocateBlockRequest.java | 6 +-
.../request/key/OMAllocateBlockRequestWithFSO.java | 6 +-
.../key/OMDirectoriesPurgeRequestWithFSO.java | 25 +-
.../ozone/om/request/key/OMKeyCommitRequest.java | 6 +-
.../om/request/key/OMKeyCommitRequestWithFSO.java | 6 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 14 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 14 +-
.../ozone/om/request/key/OMKeyDeleteRequest.java | 6 +-
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 6 +-
.../ozone/om/request/key/OMKeyPurgeRequest.java | 16 +-
.../ozone/om/request/key/OMKeyRenameRequest.java | 6 +-
.../om/request/key/OMKeyRenameRequestWithFSO.java | 6 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 250 +-
.../ozone/om/request/key/OMKeySetTimesRequest.java | 6 +-
.../request/key/OMKeySetTimesRequestWithFSO.java | 6 +-
.../ozone/om/request/key/OMKeysDeleteRequest.java | 6 +-
.../ozone/om/request/key/OMKeysRenameRequest.java | 6 +-
.../om/request/key/OMOpenKeysDeleteRequest.java | 6 +-
.../ozone/om/request/key/acl/OMKeyAclRequest.java | 6 +-
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 6 +-
.../om/request/key/acl/OMKeyAddAclRequest.java | 6 +-
.../request/key/acl/OMKeyAddAclRequestWithFSO.java | 6 +-
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 6 +-
.../key/acl/OMKeyRemoveAclRequestWithFSO.java | 6 +-
.../om/request/key/acl/OMKeySetAclRequest.java | 6 +-
.../request/key/acl/OMKeySetAclRequestWithFSO.java | 6 +-
.../request/key/acl/prefix/OMPrefixAclRequest.java | 6 +-
.../S3ExpiredMultipartUploadsAbortRequest.java | 6 +-
.../S3InitiateMultipartUploadRequest.java | 8 +-
.../S3InitiateMultipartUploadRequestWithFSO.java | 12 +-
.../multipart/S3MultipartUploadAbortRequest.java | 6 +-
.../S3MultipartUploadCommitPartRequest.java | 6 +-
.../S3MultipartUploadCompleteRequest.java | 12 +-
.../om/request/s3/security/OMSetSecretRequest.java | 6 +-
.../om/request/s3/security/S3GetSecretRequest.java | 14 +-
.../request/s3/security/S3RevokeSecretRequest.java | 4 +-
.../s3/tagging/S3DeleteObjectTaggingRequest.java | 6 +-
.../S3DeleteObjectTaggingRequestWithFSO.java | 6 +-
.../s3/tagging/S3PutObjectTaggingRequest.java | 6 +-
.../tagging/S3PutObjectTaggingRequestWithFSO.java | 6 +-
.../tenant/OMSetRangerServiceVersionRequest.java | 6 +-
.../s3/tenant/OMTenantAssignAdminRequest.java | 6 +-
.../tenant/OMTenantAssignUserAccessIdRequest.java | 6 +-
.../request/s3/tenant/OMTenantCreateRequest.java | 6 +-
.../request/s3/tenant/OMTenantDeleteRequest.java | 6 +-
.../s3/tenant/OMTenantRevokeAdminRequest.java | 6 +-
.../tenant/OMTenantRevokeUserAccessIdRequest.java | 6 +-
.../security/OMCancelDelegationTokenRequest.java | 6 +-
.../security/OMGetDelegationTokenRequest.java | 6 +-
.../security/OMRenewDelegationTokenRequest.java | 6 +-
.../request/snapshot/OMSnapshotCreateRequest.java | 8 +-
.../request/snapshot/OMSnapshotDeleteRequest.java | 6 +-
.../snapshot/OMSnapshotMoveDeletedKeysRequest.java | 12 +-
.../snapshot/OMSnapshotMoveTableKeysRequest.java | 12 +-
.../request/snapshot/OMSnapshotPurgeRequest.java | 14 +-
.../request/snapshot/OMSnapshotRenameRequest.java | 9 +-
.../snapshot/OMSnapshotSetPropertyRequest.java | 6 +-
.../om/request/upgrade/OMCancelPrepareRequest.java | 8 +-
.../request/upgrade/OMFinalizeUpgradeRequest.java | 6 +-
.../ozone/om/request/upgrade/OMPrepareRequest.java | 10 +-
.../om/request/util/OMEchoRPCWriteRequest.java | 4 +-
.../om/request/validation/VersionExtractor.java | 63 +
.../om/request/volume/OMQuotaRepairRequest.java | 6 +-
.../om/request/volume/OMVolumeCreateRequest.java | 23 +-
.../om/request/volume/OMVolumeDeleteRequest.java | 6 +-
.../om/request/volume/OMVolumeSetOwnerRequest.java | 6 +-
.../om/request/volume/OMVolumeSetQuotaRequest.java | 6 +-
.../om/request/volume/acl/OMVolumeAclRequest.java | 6 +-
.../request/volume/acl/OMVolumeAddAclRequest.java | 6 +-
.../volume/acl/OMVolumeRemoveAclRequest.java | 6 +-
.../request/volume/acl/OMVolumeSetAclRequest.java | 6 +-
.../response/s3/security/S3GetSecretResponse.java | 9 +-
.../response/snapshot/OMSnapshotPurgeResponse.java | 6 +
.../om/service/AbstractKeyDeletingService.java | 30 +-
.../ozone/om/service/DirectoryDeletingService.java | 227 +-
.../ozone/om/service/KeyDeletingService.java | 5 +
.../ozone/om/service/OpenKeyCleanupService.java | 52 +-
.../ozone/om/snapshot/SnapshotDiffManager.java | 44 +-
.../hadoop/ozone/om/snapshot/SnapshotUtils.java | 8 +-
...OzoneManagerProtocolServerSideTranslatorPB.java | 19 +-
.../protocolPB/OzoneManagerRequestHandler.java | 12 +-
.../hadoop/ozone/protocolPB/RequestHandler.java | 14 +-
.../OzoneDelegationTokenSecretManager.java | 5 +-
.../webapps/ozoneManager/om-overview.html | 3 +
.../hadoop/ozone/om/TestOmSnapshotManager.java | 25 +
...TestOzoneManagerDoubleBufferWithOMResponse.java | 23 +-
.../request/bucket/TestOMBucketCreateRequest.java | 7 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 7 +-
.../request/file/TestOMDirectoryCreateRequest.java | 12 +-
.../file/TestOMDirectoryCreateRequestWithFSO.java | 14 +-
.../om/request/file/TestOMFileCreateRequest.java | 15 +-
.../file/TestOMFileCreateRequestWithFSO.java | 8 +-
.../om/request/key/TestOMKeyCreateRequest.java | 25 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 13 +-
.../ozone/om/request/key/TestOMKeyRequest.java | 4 +
.../TestS3ExpiredMultipartUploadsAbortRequest.java | 2 +
.../TestS3InitiateMultipartUploadRequest.java | 2 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 9 +-
.../s3/multipart/TestS3MultipartRequest.java | 20 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 7 +-
.../TestS3MultipartUploadCommitPartRequest.java | 2 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 13 +-
...estS3MultipartUploadCompleteRequestWithFSO.java | 19 +-
.../s3/security/TestS3GetSecretRequest.java | 28 +
.../TestOMSnapshotPurgeRequestAndResponse.java | 37 +
.../request/validation/TestVersionExtractor.java | 66 +
.../om/request/volume/TestOMVolumeRequest.java | 1 +
.../om/service/TestDirectoryDeletingService.java | 12 +-
.../om/service/TestOpenKeyCleanupService.java | 24 +-
.../ozone/om/service/TestQuotaRepairTask.java | 23 +-
.../ozone/om/snapshot/TestSnapshotDiffManager.java | 60 +-
.../TestOzoneDelegationTokenSecretManager.java | 20 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 3 +-
.../hadoop/ozone/security/acl/TestParentAcl.java | 3 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 6 +-
hadoop-ozone/ozonefs-common/pom.xml | 26 +-
.../fs/ozone/BasicOzoneClientAdapterImpl.java | 8 +-
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 7 +-
.../ozone/BasicRootedOzoneClientAdapterImpl.java | 8 +-
.../fs/ozone/BasicRootedOzoneFileSystem.java | 7 +-
.../fs/ozone/CapableOzoneFSDataStreamOutput.java | 63 +
.../hadoop/fs/ozone/OzoneFSDataStreamOutput.java | 16 +
.../apache/hadoop/fs/ozone/TestOzoneFsShell.java | 9 +-
hadoop-ozone/ozonefs-hadoop2/pom.xml | 1 +
hadoop-ozone/ozonefs-hadoop3-client/pom.xml | 1 +
hadoop-ozone/ozonefs-hadoop3/pom.xml | 1 +
.../apache/hadoop/fs/ozone/OzoneFileSystem.java | 6 +
.../hadoop/fs/ozone/RootedOzoneFileSystem.java | 6 +
hadoop-ozone/ozonefs-shaded/pom.xml | 1 +
hadoop-ozone/ozonefs/pom.xml | 1 +
.../apache/hadoop/fs/ozone/OzoneFileSystem.java | 6 +
.../hadoop/fs/ozone/RootedOzoneFileSystem.java | 6 +
hadoop-ozone/pom.xml | 247 +-
hadoop-ozone/recon-codegen/pom.xml | 1 +
.../org/hadoop/ozone/recon/codegen/SqlDbUtils.java | 22 +
.../recon/schema/SchemaVersionTableDefinition.java | 52 +-
hadoop-ozone/recon/pom.xml | 1 +
.../hadoop/ozone/recon/ReconSchemaManager.java | 51 +-
.../org/apache/hadoop/ozone/recon/ReconServer.java | 5 +-
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 6 +-
.../recon/api/types/KeyEntityInfoProtoWrapper.java | 2 +-
.../spi/impl/OzoneManagerServiceProviderImpl.java | 12 +-
.../ozone/recon/upgrade/ReconLayoutFeature.java | 12 +
.../recon/upgrade/ReconLayoutVersionManager.java | 2 +-
.../webapps/recon/ozone-recon-web/package.json | 1 +
.../webapps/recon/ozone-recon-web/pnpm-lock.yaml | 12 +
.../src/__tests__/datanodes/Datanodes.test.tsx | 191 ++
.../__tests__/datanodes/DatanodesTable.test.tsx | 151 +
.../src/__tests__/locators/locators.ts | 13 +-
.../mocks/datanodeMocks/datanodeResponseMocks.ts | 212 ++
.../mocks/datanodeMocks/datanodeServer.ts | 72 +
.../src/__tests__/utils/datanodes.utils.tsx} | 11 +-
.../src/components/overviewCard/overviewCard.tsx | 2 +-
.../components/duBreadcrumbNav/duBreadcrumbNav.tsx | 14 +-
.../src/v2/components/duMetadata/duMetadata.tsx | 5 -
.../overviewCard/overviewStorageCard.tsx | 6 +-
.../src/v2/components/search/search.tsx | 6 +-
.../src/v2/components/select/multiSelect.tsx | 6 +-
.../src/v2/components/storageBar/storageBar.less | 29 +-
.../src/v2/components/storageBar/storageBar.tsx | 46 +-
.../src/v2/components/tables/datanodesTable.tsx | 8 +-
.../src/v2/pages/datanodes/datanodes.tsx | 13 +-
.../src/v2/pages/diskUsage/diskUsage.less | 29 +-
.../src/v2/pages/overview/overview.tsx | 2 +-
.../src/views/overview/overview.tsx | 2 +-
.../recon/persistence/AbstractReconSqlDBTest.java | 4 +
.../TestSchemaVersionTableDefinition.java | 192 +-
hadoop-ozone/s3-secret-store/pom.xml | 1 +
.../ozone/s3/remote/vault/VaultS3SecretStore.java | 8 +-
hadoop-ozone/s3gateway/pom.xml | 1 +
.../org/apache/hadoop/ozone/audit/S3GAction.java | 3 +-
.../hadoop/ozone/s3/AuthorizationFilter.java | 6 -
.../java/org/apache/hadoop/ozone/s3/Gateway.java | 11 +-
.../apache/hadoop/ozone/s3/GatewayApplication.java | 2 +-
.../hadoop/ozone/s3/RootPageDisplayFilter.java | 64 -
.../hadoop/ozone/s3/S3GatewayConfigKeys.java | 14 +
.../hadoop/ozone/s3/S3GatewayHttpServer.java | 80 +-
...ttpServer.java => S3GatewayWebAdminServer.java} | 59 +-
.../hadoop/ozone/s3/VirtualHostStyleFilter.java | 7 -
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 12 +-
.../ozone/s3/endpoint/ObjectEndpointStreaming.java | 6 +-
.../hadoop/ozone/s3/metrics/S3GatewayMetrics.java | 22 +
.../Application.java} | 8 +-
.../ozone/s3secret/S3SecretManagementEndpoint.java | 2 +-
.../webapps/{s3gateway => s3g-web}/WEB-INF/web.xml | 30 +-
.../webapps/{static => s3g-web}/images/ozone.ico | Bin
.../webapps/{static => s3g-web}/index.html | 24 +-
.../resources/webapps/s3gateway/WEB-INF/web.xml | 11 +-
.../hadoop/ozone/client/OzoneBucketStub.java | 10 +
.../ozone/client/OzoneDataStreamOutputStub.java | 2 +-
.../hadoop/ozone/s3/TestAuthorizationFilter.java | 22 -
.../ozone/s3/TestVirtualHostStyleFilter.java | 23 -
.../hadoop/ozone/s3/endpoint/TestListParts.java | 6 +-
.../s3/endpoint/TestMultipartUploadComplete.java | 2 +-
.../s3/endpoint/TestMultipartUploadWithCopy.java | 6 +-
.../hadoop/ozone/s3/endpoint/TestObjectGet.java | 4 +-
.../hadoop/ozone/s3/endpoint/TestObjectPut.java | 66 +-
.../ozone/s3/endpoint/TestObjectTaggingDelete.java | 2 +-
.../ozone/s3/endpoint/TestObjectTaggingGet.java | 2 +-
.../ozone/s3/endpoint/TestObjectTaggingPut.java | 17 +-
.../hadoop/ozone/s3/endpoint/TestPartUpload.java | 14 +-
.../s3/endpoint/TestPartUploadWithStream.java | 8 +-
.../ozone/s3/endpoint/TestPermissionCheck.java | 4 +-
.../ozone/s3/endpoint/TestUploadWithStream.java | 4 +-
.../ozone/s3/metrics/TestS3GatewayMetrics.java | 32 +-
hadoop-ozone/tools/pom.xml | 28 +-
.../OzoneAdmin.java} | 34 +-
.../ozone/admin/nssummary/NSSummaryAdmin.java | 27 +-
.../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 25 +-
.../reconfig/AbstractReconfigureSubCommand.java | 1 +
.../ozone/admin/reconfig/ReconfigureCommands.java | 27 +-
.../ozone/admin/scm/DeletedBlocksTxnCommands.java | 13 +-
.../apache/hadoop/ozone/admin/scm/ScmAdmin.java | 26 +-
.../org/apache/hadoop/ozone/conf/OzoneGetConf.java | 26 +-
.../ozone/debug/CompactionLogDagPrinter.java | 11 +-
.../hadoop/ozone/debug/FindMissingPadding.java | 14 +-
.../apache/hadoop/ozone/debug/LeaseRecoverer.java | 14 +-
.../org/apache/hadoop/ozone/debug/OzoneDebug.java | 40 +-
.../apache/hadoop/ozone/debug/ReadReplicas.java | 11 +-
.../apache/hadoop/ozone/debug/VersionDebug.java | 11 +-
.../debug/{ => chunk}/ChunkDataNodeDetails.java | 2 +-
.../ozone/debug/{ => chunk}/ChunkDetails.java | 2 +-
.../ozone/debug/{ => chunk}/ChunkKeyHandler.java | 15 +-
.../hadoop/ozone/debug/{ => chunk}/ChunkType.java | 2 +-
.../debug/{ => chunk}/ContainerChunkInfo.java | 2 +-
.../quota => debug/chunk}/package-info.java | 4 +-
.../datanode/DatanodeDebug.java} | 20 +-
.../container/ContainerCommands.java | 37 +-
.../{ => datanode}/container/ExportSubcommand.java | 4 +-
.../{ => datanode}/container/InfoSubcommand.java | 6 +-
.../container/InspectSubcommand.java | 9 +-
.../{ => datanode}/container/ListSubcommand.java | 6 +-
.../{ => datanode}/container/package-info.java | 2 +-
.../quota => debug/datanode}/package-info.java | 4 +-
.../hadoop/ozone/debug/{ => ldb}/DBScanner.java | 84 +-
.../hadoop/ozone/debug/{ => ldb}/DropTable.java | 11 +-
.../hadoop/ozone/debug/{ => ldb}/ListTables.java | 13 +-
.../hadoop/ozone/debug/{ => ldb}/RDBParser.java | 33 +-
.../hadoop/ozone/debug/{ => ldb}/ValueSchema.java | 37 +-
.../{repair/quota => debug/ldb}/package-info.java | 4 +-
.../package-info.java => debug/om/OMDebug.java} | 19 +-
.../hadoop/ozone/debug/{ => om}/PrefixParser.java | 17 +-
.../{repair/quota => debug/om}/package-info.java | 4 +-
.../{ => debug}/segmentparser/BaseLogParser.java | 2 +-
.../segmentparser/DatanodeRatisLogParser.java | 2 +-
.../segmentparser/GenericRatisLogParser.java | 2 +-
.../segmentparser/OMRatisLogParser.java | 2 +-
.../{ => debug}/segmentparser/RatisLogParser.java | 19 +-
.../segmentparser/SCMRatisLogParser.java | 2 +-
.../{ => debug}/segmentparser/package-info.java | 2 +-
.../hadoop/ozone/freon/BaseFreonGenerator.java | 5 +-
.../hadoop/ozone/freon/ChunkManagerDiskWrite.java | 2 +
.../ozone/freon/ClosedContainerReplicator.java | 2 +
.../hadoop/ozone/freon/DNRPCLoadGenerator.java | 6 +-
.../hadoop/ozone/freon/DatanodeBlockPutter.java | 2 +
.../hadoop/ozone/freon/DatanodeChunkGenerator.java | 2 +
.../hadoop/ozone/freon/DatanodeChunkValidator.java | 2 +
.../hadoop/ozone/freon/DatanodeSimulator.java | 6 +-
.../freon/FollowerAppendLogEntryGenerator.java | 2 +
.../java/org/apache/hadoop/ozone/freon/Freon.java | 55 +-
.../FreonSubcommand.java} | 9 +-
.../hadoop/ozone/freon/HadoopDirTreeGenerator.java | 2 +
.../hadoop/ozone/freon/HadoopFsGenerator.java | 2 +
.../hadoop/ozone/freon/HadoopFsValidator.java | 2 +
.../ozone/freon/HadoopNestedDirGenerator.java | 2 +
.../apache/hadoop/ozone/freon/HsyncGenerator.java | 4 +-
.../ozone/freon/LeaderAppendLogEntryGenerator.java | 2 +
.../hadoop/ozone/freon/OmBucketGenerator.java | 2 +
.../ozone/freon/OmBucketReadWriteFileOps.java | 3 +-
.../ozone/freon/OmBucketReadWriteKeyOps.java | 3 +-
.../apache/hadoop/ozone/freon/OmBucketRemover.java | 2 +
.../apache/hadoop/ozone/freon/OmKeyGenerator.java | 5 +-
.../hadoop/ozone/freon/OmMetadataGenerator.java | 5 +-
.../hadoop/ozone/freon/OmRPCLoadGenerator.java | 2 +
.../hadoop/ozone/freon/OzoneClientCreator.java | 2 +
.../ozone/freon/OzoneClientKeyGenerator.java | 2 +
.../freon/OzoneClientKeyReadWriteListOps.java | 2 +
.../hadoop/ozone/freon/OzoneClientKeyRemover.java | 2 +
.../ozone/freon/OzoneClientKeyValidator.java | 2 +
.../hadoop/ozone/freon/RandomKeyGenerator.java | 8 +-
.../hadoop/ozone/freon/RangeKeysGenerator.java | 2 +
.../hadoop/ozone/freon/S3BucketGenerator.java | 2 +
.../apache/hadoop/ozone/freon/S3KeyGenerator.java | 2 +
.../hadoop/ozone/freon/SCMThroughputBenchmark.java | 9 +-
.../apache/hadoop/ozone/freon/SameKeyReader.java | 2 +
.../hadoop/ozone/freon/StreamingGenerator.java | 2 +
.../containergenerator/GeneratorDatanode.java | 3 +
.../freon/containergenerator/GeneratorOm.java | 4 +
.../freon/containergenerator/GeneratorScm.java | 4 +
.../GenerateOzoneRequiredConfigurations.java | 48 +-
.../apache/hadoop/ozone/repair/OzoneRepair.java | 65 +-
.../org/apache/hadoop/ozone/repair/RepairTool.java | 104 +
.../hadoop/ozone/repair/om/FSORepairTool.java | 744 +++++
.../{quota/package-info.java => om/OMRepair.java} | 22 +-
...napshotRepair.java => SnapshotChainRepair.java} | 80 +-
.../hadoop/ozone/repair/om/SnapshotRepair.java | 164 +-
.../repair/{ => om}/TransactionInfoRepair.java | 58 +-
.../ozone/repair/{ => om}/quota/QuotaRepair.java | 41 +-
.../ozone/repair/{ => om}/quota/QuotaStatus.java | 19 +-
.../ozone/repair/{ => om}/quota/QuotaTrigger.java | 40 +-
.../ozone/repair/{ => om}/quota/package-info.java | 4 +-
.../package-info.java => scm/SCMRepair.java} | 20 +-
.../package-info.java => scm/cert/CertRepair.java} | 17 +-
.../{ => scm/cert}/RecoverSCMCertificate.java | 59 +-
.../repair/{quota => scm/cert}/package-info.java | 4 +-
.../ozone/repair/{quota => scm}/package-info.java | 4 +-
.../org/apache/hadoop/ozone/shell/Handler.java | 39 +-
.../apache/hadoop/ozone/shell/OzoneAddress.java | 4 +-
.../org/apache/hadoop/ozone/shell/OzoneRatis.java | 14 +-
.../org/apache/hadoop/ozone/shell/OzoneShell.java | 35 +-
.../java/org/apache/hadoop/ozone/shell/REPL.java | 90 +
.../java/org/apache/hadoop/ozone/shell/Shell.java | 49 +-
.../apache/hadoop/ozone/shell/acl/AclOption.java | 8 +-
.../hadoop/ozone/shell/bucket/BucketCommands.java | 38 +-
.../ozone/shell/bucket/CreateBucketHandler.java | 33 +-
.../ozone/shell/bucket/InfoBucketHandler.java | 8 +-
.../ozone/shell/checknative/CheckNative.java | 11 +-
.../hadoop/ozone/shell/keys/KeyCommands.java | 37 +-
.../hadoop/ozone/shell/prefix/PrefixCommands.java | 37 +-
.../org/apache/hadoop/ozone/shell/s3/S3Shell.java | 14 -
.../ozone/shell/snapshot/SnapshotCommands.java | 37 +-
.../ozone/shell/snapshot/SnapshotDiffHandler.java | 12 +-
.../ozone/shell/tenant/GetUserInfoHandler.java | 10 -
.../hadoop/ozone/shell/tenant/TenantShell.java | 15 +-
.../ozone/shell/tenant/TenantUserCommands.java | 36 +-
.../hadoop/ozone/shell/token/TokenCommands.java | 37 +-
.../hadoop/ozone/shell/volume/VolumeCommands.java | 37 +-
.../hadoop/ozone/conf/TestGetConfOptions.java | 76 +-
.../hadoop/ozone/repair/TestOzoneRepair.java | 37 +-
.../ozone/repair/TestTransactionInfoRepair.java | 146 -
.../ozone/repair/om/TestTransactionInfoRepair.java | 147 +
.../ozone/scm/TestDecommissionScmSubcommand.java | 2 +-
pom.xml | 1914 ++++++-----
776 files changed, 15229 insertions(+), 17835 deletions(-)
create mode 100644 dev-support/ci/maven-settings.xml
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/ExtensibleParentCommand.java
delete mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/MissingSubcommandException.java
rename
hadoop-hdds/common/src/main/java/org/apache/hadoop/{hdds/cli/SubcommandWithParent.java
=> ozone/Versioned.java} (78%)
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChunkBufferToByteString.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChunkBufferToByteStringByByteBufs.java
delete mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDatanodeUpgradeToScmHA.java
create mode 100644 hadoop-hdds/docs/content/integration/Hive.md
create mode 100644 hadoop-hdds/docs/content/integration/Impala.md
create mode 100644 hadoop-hdds/docs/content/integration/_index.md
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerMoveInfo.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/DataMoveInfo.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/ChunkDetails.java
=>
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/IterationInfo.java
(55%)
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/LegacyRatisContainerReplicaCount.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/LegacyReplicationManager.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/SafeModeRuleFactory.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerConfigBuilder.java
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestLegacyReplicationManager.java
rename
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/ha/{TestSCMHAConfiguration.java
=> TestSCMConfiguration.java} (80%)
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/safemode/TestSafeModeRuleFactory.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/quota/package-info.java
=> hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/IntLambda.java
(53%)
rename
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RDBRepair.java
=>
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/cli/AbstractMixin.java
(51%)
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/cli/AbstractSubcommand.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/quota/package-info.java
=>
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/cli/AdminSubcommand.java
(83%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/quota/package-info.java
=>
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/cli/DebugSubcommand.java
(83%)
delete mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/cli/OzoneAdmin.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/quota/package-info.java
=>
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/cli/RepairSubcommand.java
(83%)
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/HostNameParameters.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/util/DurationUtil.java
copy {hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/segmentparser
=>
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/util}/package-info.java
(83%)
create mode 100644
hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/util/TestDurationUtil.java
copy hadoop-ozone/{dist/src/main/compose/upgrade/compose/non-ha/load.sh =>
dev-support/k8s/regenerate-examples.sh} (71%)
create mode 100644
hadoop-ozone/dist/src/main/compose/common/s3-haproxy-secure.yaml
copy hadoop-ozone/dist/src/main/compose/{compatibility/test.sh =>
ozonesecure/test-haproxy-s3g.sh} (56%)
delete mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
delete mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
delete mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml
delete mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config
delete mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/xcompat/lib.sh
rename hadoop-ozone/dist/src/main/compose/{upgrade/compose/om-ha/load.sh =>
xcompat/test-new.sh} (67%)
rename hadoop-ozone/dist/src/main/compose/{upgrade/compose/om-ha/.env =>
xcompat/test-old.sh} (61%)
mode change 100644 => 100755
delete mode 100755 hadoop-ozone/dist/src/main/compose/xcompat/test.sh
copy hadoop-ozone/dist/src/main/k8s/{examples/ozone/kustomization.yaml =>
definitions/ozone/httpfs-ss-service.yaml} (78%)
copy hadoop-ozone/dist/src/main/{compose/upgrade/compose/non-ha/load.sh =>
k8s/definitions/ozone/httpfs-ss.yaml} (57%)
mode change 100755 => 100644
copy hadoop-ozone/dist/src/main/k8s/{examples/ozone/kustomization.yaml =>
definitions/ozone/recon-ss-service.yaml} (78%)
rename hadoop-ozone/dist/src/main/{compose/upgrade/compose/non-ha/load.sh =>
k8s/definitions/ozone/recon-ss.yaml} (50%)
mode change 100755 => 100644
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/README.md
rename hadoop-ozone/dist/src/main/{compose/upgrade/compose/non-ha/.env =>
k8s/examples/getting-started/httpfs-public-service.yaml} (77%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
getting-started/httpfs-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/getting-started/httpfs-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
getting-started/recon-public-service.yaml} (78%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
getting-started/recon-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/getting-started/recon-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
minikube/httpfs-public-service.yaml} (78%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
minikube/httpfs-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/minikube/httpfs-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
minikube/recon-public-service.yaml} (78%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
minikube/recon-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/minikube/recon-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
ozone-dev/httpfs-public-service.yaml} (78%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
ozone-dev/httpfs-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/ozone-dev/httpfs-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
ozone-dev/recon-public-service.yaml} (78%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
ozone-dev/recon-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/ozone-dev/recon-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
ozone-ha/httpfs-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/ozone-ha/httpfs-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone/kustomization.yaml =>
ozone-ha/recon-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/ozone-ha/recon-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/ozone/{kustomization.yaml =>
httpfs-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/ozone/httpfs-statefulset.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/ozone/{kustomization.yaml =>
recon-service.yaml} (78%)
create mode 100644
hadoop-ozone/dist/src/main/k8s/examples/ozone/recon-statefulset.yaml
delete mode 100644
hadoop-ozone/dist/src/main/smoketest/ec/backward-compat.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/rclone.robot
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestO3FSWithFSOAndOMRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestO3FSWithFSPathsAndOMRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestO3FSWithOMRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithCacheOnly.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithFSOAndCacheOnly.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithFSOAndOMRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithFSOAndOMRatisAndCacheOnly.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithFSPathsAndOMRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithFSPathsAndOMRatisAndACL.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithFSPathsAndOMRatisAndCacheOnly.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOFSWithOMRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestDatanodeVersion.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMEpochForNonRatis.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmHASnapshot.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmNonHASnapshot.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/repair/om/TestFSORepairTool.java
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/s3/awssdk/v1/{TestS3SDKV1WithRatis.java
=> TestS3SDKV1WithRatisStreaming.java} (75%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DeletingServiceMetrics.java
copy
hadoop-ozone/{tools/src/main/java/org/apache/hadoop/ozone/debug/ChunkDetails.java
=>
ozone-manager/src/main/java/org/apache/hadoop/ozone/om/execution/flowcontrol/ExecutionContext.java}
(55%)
rename hadoop-ozone/{s3gateway/src/main/resources/webapps/static/s3g.js =>
ozone-manager/src/main/java/org/apache/hadoop/ozone/om/execution/flowcontrol/package-info.java}
(85%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/VersionExtractor.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/validation/TestVersionExtractor.java
create mode 100644
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/CapableOzoneFSDataStreamOutput.java
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/__tests__/datanodes/Datanodes.test.tsx
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/__tests__/datanodes/DatanodesTable.test.tsx
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/__tests__/mocks/datanodeMocks/datanodeResponseMocks.ts
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/__tests__/mocks/datanodeMocks/datanodeServer.ts
copy
hadoop-ozone/{tools/src/main/java/org/apache/hadoop/ozone/segmentparser/package-info.java
=>
recon/src/main/resources/webapps/recon/ozone-recon-web/src/__tests__/utils/datanodes.utils.tsx}
(83%)
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/RootPageDisplayFilter.java
copy
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/{S3GatewayHttpServer.java
=> S3GatewayWebAdminServer.java} (70%)
copy
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/{s3/GatewayApplication.java
=> s3secret/Application.java} (81%)
copy hadoop-ozone/s3gateway/src/main/resources/webapps/{s3gateway =>
s3g-web}/WEB-INF/web.xml (60%)
rename hadoop-ozone/s3gateway/src/main/resources/webapps/{static =>
s3g-web}/images/ozone.ico (100%)
rename hadoop-ozone/s3gateway/src/main/resources/webapps/{static =>
s3g-web}/index.html (74%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{segmentparser/GenericRatisLogParser.java
=> admin/OzoneAdmin.java} (57%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
chunk}/ChunkDataNodeDetails.java (96%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
chunk}/ChunkDetails.java (96%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
chunk}/ChunkKeyHandler.java (97%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
chunk}/ChunkType.java (95%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
chunk}/ContainerChunkInfo.java (98%)
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota =>
debug/chunk}/package-info.java (90%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota/package-info.java
=> debug/datanode/DatanodeDebug.java} (60%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
datanode}/container/ContainerCommands.java (88%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
datanode}/container/ExportSubcommand.java (96%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
datanode}/container/InfoSubcommand.java (88%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
datanode}/container/InspectSubcommand.java (89%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
datanode}/container/ListSubcommand.java (87%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
datanode}/container/package-info.java (93%)
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota =>
debug/datanode}/package-info.java (90%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
ldb}/DBScanner.java (94%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
ldb}/DropTable.java (90%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
ldb}/ListTables.java (82%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
ldb}/RDBParser.java (67%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
ldb}/ValueSchema.java (83%)
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota =>
debug/ldb}/package-info.java (91%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota/package-info.java
=> debug/om/OMDebug.java} (65%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/{ =>
om}/PrefixParser.java (94%)
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota =>
debug/om}/package-info.java (91%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/BaseLogParser.java (97%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/DatanodeRatisLogParser.java (97%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/GenericRatisLogParser.java (96%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/OMRatisLogParser.java (96%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/RatisLogParser.java (73%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/SCMRatisLogParser.java (96%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{ =>
debug}/segmentparser/package-info.java (94%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/{repair/quota/package-info.java
=> freon/FreonSubcommand.java} (82%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/RepairTool.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/FSORepairTool.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{quota/package-info.java
=> om/OMRepair.java} (58%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/{SnapshotRepair.java
=> SnapshotChainRepair.java} (73%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{ =>
om}/TransactionInfoRepair.java (74%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{ =>
om}/quota/QuotaRepair.java (81%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{ =>
om}/quota/QuotaStatus.java (81%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{ =>
om}/quota/QuotaTrigger.java (72%)
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{ =>
om}/quota/package-info.java (91%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{quota/package-info.java
=> scm/SCMRepair.java} (62%)
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{quota/package-info.java
=> scm/cert/CertRepair.java} (69%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{ =>
scm/cert}/RecoverSCMCertificate.java (86%)
copy hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{quota =>
scm/cert}/package-info.java (91%)
rename hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/{quota
=> scm}/package-info.java (91%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/REPL.java
delete mode 100644
hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/repair/TestTransactionInfoRepair.java
create mode 100644
hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/repair/om/TestTransactionInfoRepair.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
