This is an automated email from the ASF dual-hosted git repository.
msingh pushed a change to branch HDDS-3630
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from cbecfe6 HDDS-5920. Allow to add ColumnFamilyOptions to tables. (#2790)
new e314ccb HDDS-5863. Error message having null fields on volume
creation (#2752)
new 04b4603 HDDS-5885. OM Auth Validate for read requests and handle non
ratis enabled code. (#2779)
new 88db753 HDDS-5908. MPU getKey can fail, if completeMPU result is
still in cache. (#2787)
new 82ed79c HDDS-5883 Change S3G client to set S3 Auth per req (#2775)
new f54a110 HDDS-5864. Retry when DN connection issue during
getBlock/ReadChunk call during Ozone key Read (#2746)
new 4526820 HDDS-5736. Update navbar from Hadoop to Ozone on doc page
(#2792)
new c110da3 HDDS-3983 Ozone RocksDB Iterator wrapper should not expose
key() and value() API. (#2402)
new 57ad3ca Support more detailed error log when handleFlush in Ozone
client (#2795)
new 7af8b98 HDDS-5931. Broken link in dist/README (#2799)
new 78ddb68 HDDS-5933. Ozone Filesystem shaded jars include unnecessary
dependencies (#2798)
new 249fb30 HDDS-3369. Cleanup old write-path of volume in OM. (#2780)
new 57ee2d7 HDDS-5925. Fix SecuringOzoneHTTP doc (#2796)
new bcca116 HDDS-5943. Shutdown ResultHandlerExecutorService for
StorageVolumeChecker. (#2806)
new 0948456 HDDS-5944. Fix typo in OzoneManagerServiceProviderImpl.java
(#2807)
new 90a2076 HDDS-5945. bump rocksdb version to 6.25.3 (#2809)
new c0f387e HDDS-5935. Bump Spring to 5.2.18 (#2801)
new 45fd4e1 HDDS-5922. ignore deletetransaction when container is not
found (#2793)
new 09227e5 HDDS-5839. Make sure buckets created from OFS are in
FILE_SYSTEM_OPTIMIZED layout (#2730)
new e851c8a HDDS-5947. Remove unused mina-core and sshd-core dependencies
(#2811)
new bd04099 HDDS-5910 Add additional verification for S3 Auth. (#2817)
new a6f96e9 HDDS-5956. Speed up
TestOzoneRpcClientAbstract#testZReadKeyWithUnhealthyContainerReplica (#2820)
new 93f0fb0 HDDS-5929. Make FSO and OBS bucket layouts independent of
normalization config flag (#2819)
new f3d1b9a HDDS-5960. Change option name to bucketlayout instead of type
(#2825)
new 2eea2eb HDDS-5970. Remove OMKeyRequest#getBucketLayout overridden
method in subclasses (#2830)
new 23bae65 HDDS-5820. Intermittent failure in
TestPipelinePlacementPolicy#testPickLowestLoadAnchor (#2757)
new 1cef2be HDDS-5958. Refine container scrub log message. (#2823)
new a5127b7 HDDS-5937. Inaccurate bucket info returned from bucket list
command in Shell (#2832)
new 3475993 HDDS-5872. Do not failover on some RpcExceptions (#2772)
new ea19c6b HDDS-5981. Refactor usage of bucket type. (#2836)
new 3ceaf8a HDDS-5982. Remove isBucketFSOptimized flag. (#2838)
new 9bc29ed HDDS-5906. Fix DBScanner to support Datanode DB (#2781)
new 2f52203 HDDS-5983. Prefix Parser tool should only work for FSO
buckets. (#2839)
new c9592d3 HDDS-5957. Bump proto-backwards-compatibility from 1.0.5 to
1.0.7 (#2821)
new 2e2f7f0 HDDS-5891. OFS mkdir -p does not work as expected for bucket
creation when volume exists due to volume create ACL check (#2814)
new 8f3fa92 HDDS-5964. [S3-TDE] Range Reads not working on encrypted
buckets (#2827)
new f134072 HDDS-5848. Introduce more replication metrics (#2758)
new 23fe5b1 HDDS-5980. Get on a key which is created/overwrite through
MPU on a TDE bucket, the data does not match with uploaded data. (#2844)
new b518a47 HDDS-5969. TestRootedOzoneFileSystem parameterized test is
not initialized properly (#2843)
new 47b5be5 HDDS-5870. Make Datanode usageinfo command output more
readable (#2816)
new 3d92b3b HDDS-5534. Verify config is updated on all OMs before
proceeding with Bootstrap (#2491)
new 0baa42e HDDS-5898 S3G in secure mode checks OM version. (#2822)
new 2585ece HDDS-5517. Support multiple container moves from a source
datanode in one balance iteration (#2808)
new 40e0473 HDDS-5976. add raftlog IOException notifier for SCM HA (#2840)
new 8a6b5e4 [FSO] Allow existing/old buckets with any layout during OM
startup (#2571)
new a6f0bec HDDS-6019. Fix s3g create key error in cluster with
replication one. (#2855)
new a9ecaf0 HDDS-6003. Update Ozone master branch version after 1.2.0
release (#2863)
new b887b89 HDDS-6043. Buckets created via link command do not mirror
layout of s… (#2867)
new a1a3bd9 HDDS-5897. Support configuration for including/excluding
datanodes for balancing (#2786)
new 58ceddf HDDS-5989. Binary content printed for failed chunk write
(#2846)
new d8f32a6 HDDS-6011. Freon datanode chunk generator fails with NPE in
secure cluster (#2853)
new 0db7930 HDDS-5497. Save log from init containers (#2547)
new c8b3000 HDDS-5804. SCMContainerPlacementCapacity to choose lower
utilized node (#2784)
new ccf68ea HDDS-5917. S3 acceptance test failure due to too wide
assertion (#2789)
new 432c146 HDDS-5702. SCM remote benchmark tool. (#2600)
new aad2bd0 HDDS-5936. Duplicate Test Ozone Shell invocation (#2803)
new 5091bdd HDDS-6054. Fix resource leak due to OzoneManager.createOm
(#2876)
new 0f54c90 HDDS-6000. Freon datanode chunk validator fails checksum
validation (#2850)
new 1186262 HDDS-6055. Refactor isFileSystemOptimized usage in
OzoneManagerUtils (#2877)
new 1ddac15 HDDS-5959. Handles bucket layout validation logic in ofs/o3fs
client. (#2837)
new 6a00d27 HDDS-5999. Freon datanode chunk validator does not find
pipeline from param (#2879)
new 53818cc HDDS-6014. Add proto.lock files from ozone-1.2 release branch
to master (#2865)
new 57d17c1 HDDS-5962. Limit grpc threads in XceiverServerGrpc. (#2824)
new 5be0778 HDDS-5997 Centralize string based replication config
validation via ReplicationConfigValidator (#2871)
new 15c0c5a HDDS-6058. Bump aws-java-sdk to latest 1.12 (#2881)
new ea50f88 HDDS-6041. [FSO] Revisit robot test cases related to
prefix/simple me… (#2885)
new 4869766 HDDS-6061. Peer datanode cannot add group for pipeline in
secure env (#2882)
new 90be5f7 HDDS-5965. Recon should be able to distinguish between
containers that have no replicas and those have all replicas as UNHEALTHY.
(#2874)
new c743865 HDDS-5973. Changed sequence of steps during pipeline close.
(#2847)
new fbd6b59 HDDS-5903. Add Support for Bucket Owner Acls (#2826)
new d66ad70 HDDS-6059. Change Cannot close container log message from
WARN to DEBUG (#2890)
new 841d3ec HDDS-3227. Ensure eviction of stateMachineData from cache
only when both followers catch up (#2704)
new 95865b0 HDDS-6067. Improve Debugging around unhealthy container state
on Datanode. (#2888)
new 5229537 HDDS-6071. ResourceLimitCache leaks permits (#2895)
new 0470930 HDDS-5385. [FSO] Remove ozone.om.metadata.layout config in OM
(#2887)
new aa0c69d HDDS-6073. Document: Fix images alignment in the doc pages
(#2898)
new d08011a HDDS-6069: Fix XmlRootElement for CopyObjectResponse (#2891)
new 5445334 HDDS-6046. Multi-threaded validator for freon
RandomKeyGenerator. (#2869)
new 1fd6101 HDDS-6078. Move BucketManagerImpl#resolveLinkBucketLayout
into utilit… (#2902)
new ff63767 HDDS-5974. Remove OMClientResponse#getBucketLayout override
in subclasses - part1 (#2894)
new ef5cf76 HDDS-6085. Update log4j to 2.15 (#2906)
new 458b654 HDDS-6066. Cleanup OmService injection. (#2896)
new 1b76294 HDDS-6090. Collect output.log from acceptance tests (#2908)
new 2a5d7a7 HDDS-6096. S3 copy object fails for source key with special
char (#2912)
new a9b4467 HDDS-6018 Closing state to quasiclosed (#2854)
new b1c23fa HDDS-6104. Update log4j version to 2.16.0 version. (#2920)
new 6624532 HDDS-6053. Fix too short container scrubber data scan
interval. (#2875)
new 75c72df HDDS-6080. Simplify the logic in
SimpleContainerDownloader.getContainerDataFromReplicas (#2903)
new 310cdec HDDS-6107. Exclude hugo lock file from source tarball (#2923)
new bda049b HDDS-6089. Remove OMClientResponse#getBucketLayout override
in subclasses - part2 (#2915)
new f129671 HDDS-6072. Fix increased integration test execution time
(#2900)
new ca4ddd8 HDDS-6105. remove db cache size config (#2922)
new 73c6201 HDDS-5912. Add description of hdds.datanode.dir in Getting
Started (#2783)
new c923508 HDDS-6110. Add hugo lock file to gitignore (#2928)
new 5455fac HDDS-5795. [FSO] Add unit test case with mpu open key during
abort. (#2931)
new 8c0ec9d HDDS-5490. Remove (Decommission) an OM node from HA ring
(#2886)
new 1df5931 HDDS-6124. Update log4j version to 2.17.0 version (#2933)
new 1740598 HDDS-6125. Support Hugo 0.91.0. (#2934)
new 860f904 HDDS-5602. make it configurable to choose the nearest one as
the target in the candidates according to networkTopology (#2756)
new 13ad5db HDDS-6101. use FileUtils.moveDirectory instead of Files.move
when installing snapshot (#2918)
new b80319c HDDS-5975. Serve url-encoded key and prefix name for
ListObjectResponse (#2834)
new 8f1d5b2 HDDS-5807. [FSO] Merge HDDS-4653 changes to support TDE for
MPU Keys in FSO bucket layout. (#2941)
new d3c6b4a HDDS-6129. OM has thread unsafe issues that can cause OM
shutdown. (#2938)
new ecc49d7 HDDS-6070. ContainerBalancerConfig doesn't read config from
ozone-site.xml (#2893)
new 0825e35 HDDS-6121. DatanodeAdminMonitor should log detailed
information for a limited number of containers (#2932)
new f7b9c36 HDDS-5993. Disable flaky
TestRandomKeyGenerator#cleanObjectsTest
new 0dc4905 HDDS-5319 Intermittent failure in
TestSCMUpdateServiceGrpcServer (#2558)
new 600bda6 HDDS-6143. Update log4j version to 2.17.1 (#2952)
new 4e6be1f HDDS-6142. Remove unused dependency: jdom (#2951)
new 9d0328f HDDS-6119. Reset the property name to configure the datanode
kerberos keytab (#2949)
new 6004246 HDDS-6118. Documentation missing settings to secure S3g with
Kerberos (#2950)
new 4043682 HDDS-6108. Add slf4j and log4j.properties for tests in
hadoop-ozone/client (#2925)
new 799c5f0 HDDS-6146. TestDefaultCAServer#testIntermediaryCA failure
(#2955)
new 14a036e HDDS-6103. Avoid refresh pipeline for key override in
Key/File Create. (#2926)
new 8d586a2 HDDS-6145. Include docker-compose output in acceptance test
output.log (#2958)
new ce0a81f HDDS-6123. Disable system exit in integration tests (#2959)
new bcd8fd0 HDDS-6149. Remove unused keytabs (#2960)
new af3e985 HDDS-6094. Some unit tests are skipped due to using JUnit4
runner (#2909)
new 529b596 HDDS-6075. OzoneConfiguration constructor overrides input
configuration keys. (#2921)
new 90f5c8c HDDS-4177. SCM Container DB bootstrap on Recon startup (#2942)
new 5728511 HDDS-6086. Compute MD5MD5CRC file checksum using chunk
checksums from DataNodes (#2919)
new 1f3fa6b HDDS-6148. Validate ContainerBalancerConfiguration before
start ContainerBalancer (#2957)
new 65722cc HDDS-6161. SCM StateMachine failing to reinitialize doesn't
terminate the process. (#2971)
new caeeed8 HDDS-6134. Move replication-specific config to
ReplicationServer (#2943)
new b838906 HDDS-4010. S3G startup fails when multiple service ids are
configured. (#2976)
new d037b68 HDDS-6170. Add metrics to replication manager to track
container health states (#2975)
new 6333ddb HDDS-3231. Cleanup KeyManagerImpl (#2961)
new 20a9b1c HDDS-5927. Improve defaults in ContainerBalancerConfiguration
(#2892)
new e2249e8 HDDS-6157. More consistent synchronization in InputStreams
(#2965)
new 41be11a HDDS-4743. [FSO] Add FSO variant of ITestOzoneContractDistcp.
(#2980)
new 1b8c570 HDDS-6114. Intermittent error due to Failed to init RocksDB
(#2947)
new a9dac96 HDDS-6175. Use s3Auth during proxy during decrypt in
RpcClient. (#2981)
new d63d398 HDDS-5740. Enable ratis by default for SCM. (#2637)
new 57e24aa HDDS-6183. Intermittent failure in
TestKeyDeletingService.checkIfDeleteServiceWithFailingSCM. (#2991)
new d3e8617 HDDS-4190. Intermittent failure in
TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest. (#2982)
new da819e5 HDDS-6120. Compute block checksum using chunk checksums
(#2930)
new 2aacf6c HDDS-6147. Add ability in OM to get limited delta updates
(#2956)
new 3017c8d HDDS-6195. Remove unused jmh-core dependency. (#2997)
new a182eb9 HDDS-6167. Update ozone-runner version to 20211202-1 (#2969)
new e5e5abc HDDS-6171. Create an API to change Bucket Owner (#2988)
new f061a42 HDDS-6163. Fix PATH in docker image (#2967)
new 408e7a8 HDDS-6202. Avoid using jmh-generator-annprocess since it is
GPL2.0. (#2998)
new 0032f6e HDDS-6135. SCM Container DB bootstrap on Recon startup for
SCM HA. (#2972)
new ce6054a HDDS-6109. Preserve the underlying exception raised in client
lib. (#2989)
new 6fc732c HDDS-3408. Rename ChunkLayOutVersion to
ContainerLayoutVersion. (#2983)
new 94f59e5 HDDS-6203. CleanUp incomplete gz files during Container move
(#3000)
new 834cb1a HDDS-6216. Move OMOpenKeysDeleteRequest to package
om.request.key (#3011)
new fae9b2a HDDS-6191. Intermittent failure in TestDeleteWithSlowFollower
(#3015)
new 066b79d HDDS-6128. CLI tool that downloads all the block replicas and
creates a manifest file (#2987)
new 46eff76 HDDS-6177. Extend container info command to include replica
details (#2995)
new 572251b HDDS-6211. [Docs] Image styling on deployed site does not
replicate local builds. (#3007)
new 7369ead HDDS-6219. Switch to RATIS ReplicationType from STAND_ALONE
in our tests. (#3014)
new 25e99a0 HDDS-6192. feature/Observability.md translated to Chinese
(#2994)
new 0833ee3 HDDS-6205. Add CLI command to display the latest Replication
Manager report (#3013)
new ae1d283 HDDS-6227. Test helpers should observe naming conditions
(#3020)
new b1f8f63 HDDS-6239. ozonesecure-mr failing with No URLs in mirrorlist
(#3029)
new 4994a39 HDDS-6201. Fix NPE for DataScanner with scanned container
deleted by others. (#3005)
new dbfb1f9 HDDS-5529. For any IOexception from @Replicated method we
should throw it (#2788)
new a911276 HDDS-6181. Change SCMHAInvocationHandler#invokeRatis()
logging to TRACE (#2992)
new 37f5c4e HDDS-6206. Application errors must not flood system log
(#3001)
new a4636cf HDDS-6245. Add BucketLayout logging to Audit Logs (#3040)
new d13381b HDDS-6238 Reduce memory requirements for list keys. (#3032)
new ccfc01e HDDS-2919. Intermittent failure in TestRDBStore (#3028)
new 0576d59 HDDS-6253. Unnecessary duplicate smoketest after defaulting
to FSO (#3036)
new 50d49ba HDDS-6204. Cleanup handling malformed authorization header
(#2999)
new faa1577 HDDS-6169. Selective checks: skip junit tests on ozone-runner
image update (#2974)
new 79e9046 HDDS-6270. Use a dedicated file instead of /etc/passwd for
xcompat acceptance test (#3050)
new d4ea29e HDDS-6273. Amend doc SecuringTDE.md (#3047)
new 17db563 HDDS-6140. Selective checks: skip unit check for
integration-test changes (#2948)
new 8585e8d HDDS-6215. Recon get limited delta updates from OM (#3009)
new 1c60c12 HDDS-6226. Run tests for selective CI checks in CI (#3019)
new a441f04 HDDS-6247. Avoid logging stack trace for user input problems
(#3039)
new 85816df HDDS-6208. New checkstyle: WhitespaceAround (#3003)
new 58a0cac HDDS-6289. Upgrade acceptance test log flooded with parse
error (#3063)
new 53e6aea HDDS-6291. Do not let slf4j-log4j12 leak into Ozone FS shaded
jar (#3064)
new d9eced2 HDDS-6285. ozonesecure-mr intermittently failing with timeout
downloading packages (#3057)
new e12ad2d HDDS-6278 Improve memory profile for listStatus API call.
(#3053)
new 1e2a819 HDDS-6257. Wrong stack trace for S3 errors (#3066)
new e3cc9d1 HDDS-6088. Implement O3FS/OFS getFileChecksum() using file
checksum helpers. (#2935)
new a016b52 HDDS-6088. Implement O3FS/OFS getFileChecksum() using file
checksum helpers - addendum: fix checkstyle
new 3ecfcb0 HDDS-4126. Freon key generator should support >2GB files.
(#3054)
new 7613306 HDDS-6293. Allow using custom ozone-runner image (#3072)
new dbab846 HDDS-6309. Update ozone-runner version to 20220212-1 (#3079)
new 834949a HDDS-6304. Add enforcer to make sure ozone.version equals
project.version (#3075)
new 67821c0 HDDS-6190. Cleanup unnecessary datanode id path checks.
(#2993)
new 5eb8d9e HDDS-6310. Update json-smart to 2.4.7. (#3080)
new fbbfacc HDDS-6186. Selective checks: skip integration check for unit
test changes (#3061)
new 7f027a1 HDDS-6313. Remove replicas in ContainerStateMap when a
container is deleted (#3086)
new 3ce111d HDDS-5913. Avoid integer overflow when setting
dfs.container.ratis.lo… (#2785)
new 97692e3 HDDS-6322. Fix Recon getting inccorrect sequenceNumber from
OM (#3090)
new 4a50c86 HDDS-6325. Fix interface ClientProtocol methods typo
setThreadLocalS3Auth and clearThreadLocalS3Auth (#3093)
new a849fd7 HDDS-6311. Fix number of keys displayed in Recon Overview.
(#3081)
new 5222300 HDDS-6305: Add metrics - number of FSO bucket creates (#3077)
new 9ae9a57 HDDS-6330. Remove unnecessary duplicate semicolons (#3097)
new 8a326a8 HDDS-6331. Remove toString in debug log parameters within
SCMCommonPlacementPolicy (#3098)
new 5b1b276 HDDS-6283. Change ContainerStateMachine ContainerOpExecutor
name (#3055)
new 56ea9b4 HDDS-6324. Do not trigger CI by reopening PR (#3092)
new fd6b2c6 HDDS-6284. Add BlockDeletingService worker size config (#3056)
new cf06445 HDDS-6314. ConcurrentModificationException getting
SCMContainerMetrics (#3101)
new a33c31d HDDS-6290. operational-state and node-state options in
datanode list CLI not working correctly (#3105)
new cbb51c1 HDDS-6262. ozone insight log stops working after OM DBUpdates
message (#3044)
new b4ac55b HDDS-6234. Repair containers affected by incorrect used bytes
and block count. (#3042)
new 8cb93b5 HDDS-6212. SCM Container DB bootstrap on Recon startup for
secure cluster (#3027)
new 742b0bd HDDS-6292. Ensure immutable ContainerReplica set is returned
from ContainerStateManagerImpl (#3071)
new 9629f0b HDDS-6329. New checkstyle: AvoidStarImport (#3096)
new 86b8a9b HDDS-6350. Rename TestStandardOutputUtil to observe naming
conventions (#3114)
new 1aefd7e HDDS-6328. Add metrics - number of FSO bucket deletes (#3107)
new 7de53c4 HDDS-6327. Upgrade acceptance test doesn't collect logs when
the test fails (#3106)
new 9cb1191 HDDS-6351. [Docs] Update prefix-based FSO documentation.
(#3115)
new aebdb6b HDDS-5267. Full Container Report can remove replicas added by
an Incremental Report (#2963)
new 07a3eb4 HDDS-6363. Update checkstyle version to 9.3 (#3123)
new 7ce1b55 HDDS-6221. Update versions covered by cross-compatibility
test (#3017)
new 7b6baaa HDDS-5359. Incorrect BLOCKCOUNT and BYTESUSED in container
DB. (#3034)
new 1197e6d HDDS-6349. IncompleteReadError on get MPU key from TDE bucket
(#3116)
new 192ef6f HDDS-6235. Empty KeyValueContainers are replicated without
chunks directory. (#3052)
new 9b6a0e8 HDDS-5194. Provide kustomize descriptors for Ozone kubernetes
examples (#2221)
new 84a2a67 HDDS-6334. Remove ContainerID to Proto to ContainerID
conversion in ContainerStateManagerImpl (#3110)
new d2b832e HDDS-6361. Modify docs build flow to replace image tags with
shortcodes. (#3122)
new ff06807 HDDS-6369. PARTIAL_RENAME does not update OM DB (#3126)
new 1e74cb1 HDDS-6374. Fix incorrect queueTime metrics of ReplicationTask
(#3135)
new 71b0471 HDDS-6337. [FSO] Disable recursive access check flag for
directories with no children. (#3134)
The 220 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/post-commit.yml | 8 +-
.gitignore | 4 +-
CONTRIBUTING.md | 3 +-
dev-support/ci/selective_ci_checks.bats | 70 +-
dev-support/ci/selective_ci_checks.sh | 76 +-
hadoop-hdds/client/pom.xml | 4 +-
.../apache/hadoop/hdds/scm/OzoneClientConfig.java | 38 +
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 46 +-
.../hadoop/hdds/scm/client/HddsClientUtils.java | 4 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 54 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 10 +-
.../apache/hadoop/hdds/scm/storage/BufferPool.java | 2 +-
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 2 +-
.../hdds/scm/storage/RatisBlockOutputStream.java | 2 +-
.../storage/DummyBlockInputStreamWithRetry.java | 14 +-
.../hdds/scm/storage/TestBlockInputStream.java | 32 +-
hadoop-hdds/common/pom.xml | 19 +-
.../apache/hadoop/hdds/DFSConfigKeysLegacy.java | 4 +
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 2 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 124 ++-
.../java/org/apache/hadoop/hdds/NodeDetails.java | 0
.../java/org/apache/hadoop/hdds/StringUtils.java | 2 +-
.../hadoop/hdds/annotation/InterfaceAudience.java | 6 +-
.../org/apache/hadoop/hdds/client/OzoneQuota.java | 2 +-
.../org/apache/hadoop/hdds/client/QuotaList.java | 11 +-
.../hadoop/hdds/client/RatisReplicationConfig.java | 27 +-
.../hdds/client/ReplicatedReplicationConfig.java} | 23 +-
.../hadoop/hdds/client/ReplicationConfig.java | 138 +--
.../hdds/client/ReplicationConfigValidator.java | 5 +-
.../hadoop/hdds/client/ReplicationFactor.java | 2 +-
.../hdds/client/StandaloneReplicationConfig.java | 23 +-
.../hadoop/hdds/conf/OzoneConfiguration.java | 12 +-
.../hadoop/hdds/fs/CachingSpaceUsageSource.java | 2 +-
.../hadoop/hdds/protocol/DatanodeDetails.java | 4 +-
.../org/apache/hadoop/hdds/ratis/RatisHelper.java | 3 +-
.../RequestTypeDependentRetryPolicyCreator.java | 6 +-
.../org/apache/hadoop/hdds/recon/ReconConfig.java | 2 +-
.../hadoop/hdds/scm/ByteStringConversion.java | 2 +-
.../java/org/apache/hadoop/hdds/scm/ScmConfig.java | 2 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 12 +-
.../apache/hadoop/hdds/scm/XceiverClientSpi.java | 6 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 24 +-
.../hadoop/hdds/scm/container/ContainerInfo.java | 3 +-
.../hdds/scm/container/ContainerReplicaInfo.java | 129 +++
.../scm/container/ReplicationManagerReport.java | 283 ++++++
.../org/apache/hadoop/hdds/scm/ha/SCMHAUtils.java | 40 +-
.../apache/hadoop/hdds/scm/net/InnerNodeImpl.java | 16 +-
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 28 +-
.../org/apache/hadoop/hdds/scm/net/NodeSchema.java | 4 +-
.../hadoop/hdds/scm/net/NodeSchemaLoader.java | 8 +-
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 10 +-
.../protocol/StorageContainerLocationProtocol.java | 24 +-
.../hdds/scm/storage/ContainerProtocolCalls.java | 13 +-
.../apache/hadoop/hdds/utils/HddsVersionInfo.java | 2 +-
.../hadoop/hdds/utils/ResourceLimitCache.java | 7 +-
.../hadoop/hdds/utils/ResourceSemaphore.java | 10 +-
.../org/apache/hadoop/hdds/utils/UniqueId.java | 2 +-
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 17 +
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 13 +-
.../hadoop/ozone/audit/AuditEventStatus.java | 2 +-
.../org/apache/hadoop/ozone/audit/AuditLogger.java | 4 +-
.../apache/hadoop/ozone/audit/AuditLoggerType.java | 2 +-
.../org/apache/hadoop/ozone/audit/AuditMarker.java | 4 +-
.../apache/hadoop/ozone/audit/AuditMessage.java | 10 +-
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +-
.../hadoop/ozone/common/ChecksumByteBuffer.java | 2 +-
.../ozone/common/IncrementalChunkBuffer.java | 13 +-
.../apache/hadoop/ozone/common/StorageInfo.java | 8 +-
.../ozone/common/ha/ratis/RatisSnapshotInfo.java | 2 +-
.../ozone/container/common/helpers/BlockData.java | 4 +-
.../helpers/ContainerCommandRequestPBHelper.java | 10 +-
.../java/org/apache/hadoop/ozone/lease/Lease.java | 8 +-
.../hadoop/ozone/lease/LeaseCallbackExecutor.java | 2 +-
.../hadoop/ozone/util/ShutdownHookManager.java | 2 +-
.../common/src/main/resources/ozone-default.xml | 166 ++--
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 4 +-
.../hadoop/hdds/client/TestReplicationConfig.java | 280 +++---
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 28 +
.../java/org/apache/hadoop/hdds/fs/TestDU.java | 2 +-
.../ratis/TestContainerCommandRequestMessage.java | 6 +-
.../scm/container/TestContainerReplicaInfo.java | 59 ++
.../container/TestReplicationManagerReport.java | 162 ++++
.../hadoop/hdds/scm/container}/package-info.java | 8 +-
.../apache/hadoop/hdds/scm/ha/TestSCMNodeInfo.java | 4 +-
.../hdds/scm/net/TestNetworkTopologyImpl.java | 20 +-
.../hadoop/hdds/scm/pipeline/MockPipeline.java | 2 +-
.../hadoop/hdds/tracing/TestStringCodec.java | 18 +-
.../hadoop/hdds/utils/MockGatheringChannel.java | 2 +-
.../hadoop/hdds/utils/TestResourceLimitCache.java | 46 +-
.../hadoop/hdds/utils/TestResourceSemaphore.java | 6 +-
.../org/apache/hadoop/ozone/audit/DummyEntity.java | 2 +-
.../hadoop/ozone/audit/TestOzoneAuditLogger.java | 4 +-
.../apache/hadoop/ozone/common/TestChecksum.java | 2 +-
.../hadoop/ozone/common/TestChunkBuffer.java | 20 +-
.../hadoop/ozone/common/TestStateMachine.java | 4 +-
.../ozone/container/ContainerTestHelper.java | 70 +-
.../hadoop/ozone/lease/TestLeaseManager.java | 2 +-
.../TestDefaultUpgradeFinalizationExecutor.java | 3 +-
...TestUpgradeUtils.java => UpgradeTestUtils.java} | 4 +-
hadoop-hdds/config/pom.xml | 4 +-
hadoop-hdds/container-service/pom.xml | 4 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 33 +-
.../container/common/helpers/ContainerMetrics.java | 6 +-
.../container/common/helpers/ContainerUtils.java | 70 +-
.../common/helpers/DatanodeVersionFile.java | 8 +-
.../ozone/container/common/impl/ContainerData.java | 65 +-
.../container/common/impl/ContainerDataYaml.java | 11 +-
...OutVersion.java => ContainerLayoutVersion.java} | 34 +-
.../ozone/container/common/impl/ContainerSet.java | 16 +-
.../container/common/impl/HddsDispatcher.java | 19 +-
.../common/impl/OpenContainerBlockMap.java | 2 +-
.../common/interfaces/ContainerInspector.java | 72 ++
.../ozone/container/common/interfaces/Handler.java | 14 +-
.../common/report/ContainerReportPublisher.java | 5 +-
.../common/report/IncrementalReportSender.java} | 26 +-
.../common/statemachine/DatanodeConfiguration.java | 63 +-
.../common/statemachine/DatanodeStateMachine.java | 21 +-
.../common/statemachine/StateContext.java | 143 +--
.../CloseContainerCommandHandler.java | 17 +-
.../commandhandler/CommandDispatcher.java | 2 +-
.../SetNodeOperationalStateCommandHandler.java | 11 -
.../common/states/datanode/InitDatanodeState.java | 9 +-
.../states/datanode/RunningDatanodeState.java | 8 +-
.../states/endpoint/HeartbeatEndpointTask.java | 10 +-
.../states/endpoint/VersionEndpointTask.java | 2 +-
.../common/transport/server/XceiverServerGrpc.java | 52 +-
.../common/transport/server/ratis/CSMMetrics.java | 2 +-
.../server/ratis/ContainerStateMachine.java | 84 +-
.../transport/server/ratis/XceiverServerRatis.java | 11 +-
.../common/utils/ContainerInspectorUtil.java | 87 ++
.../container/common/utils/HddsVolumeUtil.java | 2 +-
.../container/common/volume/AbstractFuture.java | 13 +-
.../container/common/volume/MutableVolumeSet.java | 2 +-
.../container/common/volume/StorageVolume.java | 2 +-
.../common/volume/StorageVolumeChecker.java | 1 +
.../ozone/container/common/volume/VolumeUsage.java | 2 +-
.../container/keyvalue/KeyValueContainer.java | 86 +-
.../container/keyvalue/KeyValueContainerCheck.java | 18 +-
.../container/keyvalue/KeyValueContainerData.java | 20 +-
.../KeyValueContainerMetadataInspector.java | 463 +++++++++
.../ozone/container/keyvalue/KeyValueHandler.java | 107 +-
.../container/keyvalue/TarContainerPacker.java | 55 +-
.../container/keyvalue/helpers/ChunkUtils.java | 4 +
.../helpers/KeyValueContainerLocationUtil.java | 2 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 23 +-
.../container/keyvalue/impl/BlockManagerImpl.java | 110 ++-
.../keyvalue/impl/ChunkManagerDispatcher.java | 36 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 4 +-
.../keyvalue/impl/FilePerChunkStrategy.java | 4 +-
.../background/BlockDeletingService.java | 112 ++-
.../container/metadata/AbstractDatanodeStore.java | 8 +-
.../metadata/DatanodeSchemaOneDBDefinition.java | 6 +-
.../container/ozoneimpl/ContainerController.java | 14 +-
.../ozoneimpl/ContainerDataScrubberMetrics.java | 6 +-
.../ozoneimpl/ContainerMetadataScanner.java | 2 +-
.../ContainerMetadataScrubberMetrics.java | 2 +-
.../ozone/container/ozoneimpl/ContainerReader.java | 6 +-
.../ozoneimpl/ContainerScrubberConfiguration.java | 65 +-
.../ozone/container/ozoneimpl/OzoneContainer.java | 51 +-
.../container/replication/ContainerDownloader.java | 3 +-
.../replication/DownloadAndImportReplicator.java | 19 +-
.../replication/GrpcReplicationClient.java | 17 +-
.../container/replication/MeasuredReplicator.java | 37 +-
.../container/replication/ReplicationServer.java | 56 +-
.../replication/ReplicationSupervisor.java | 9 +-
.../replication/ReplicationSupervisorMetrics.java | 1 -
.../replication/SimpleContainerDownloader.java | 39 +-
.../container/stream/DirstreamClientHandler.java | 2 +-
.../upgrade/DataNodeUpgradeFinalizer.java | 2 +-
.../upgrade/VersionedDatanodeFeatures.java | 2 +-
.../ozone/protocol/commands/ReregisterCommand.java | 2 +-
.../hadoop/ozone/TestHddsSecureDatanodeInit.java | 2 +-
.../ozone/container/common/ContainerTestUtils.java | 4 +-
.../hadoop/ozone/container/common/ScmTestMock.java | 42 +-
.../container/common/TestBlockDeletingService.java | 26 +-
.../ozone/container/common/TestContainerCache.java | 2 +-
...ersion.java => TestContainerLayoutVersion.java} | 12 +-
.../container/common/TestDatanodeStateMachine.java | 5 -
.../common/TestKeyValueContainerData.java | 16 +-
.../TestSchemaOneBackwardsCompatibility.java | 38 +-
.../container/common/helpers/TestBlockData.java | 6 +-
.../common/helpers/TestContainerUtils.java | 2 +-
.../common/helpers/TestDatanodeVersionFile.java | 14 +-
.../common/impl/TestContainerDataYaml.java | 18 +-
.../impl/TestContainerDeletionChoosingPolicy.java | 10 +-
.../common/impl/TestContainerPersistence.java | 55 +-
.../container/common/impl/TestContainerSet.java | 20 +-
.../container/common/impl/TestHddsDispatcher.java | 15 +-
.../container/common/interfaces/TestHandler.java | 2 +-
.../common/report/TestReportPublisher.java | 2 +-
.../statemachine/TestDatanodeConfiguration.java | 11 -
.../common/statemachine/TestStateContext.java | 71 +-
.../TestCloseContainerCommandHandler.java | 18 +-
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 2 +-
.../common/volume/TestStorageVolumeChecker.java | 24 +-
.../container/common/volume/TestVolumeSet.java | 2 +-
...tTestInfo.java => ContainerLayoutTestInfo.java} | 24 +-
.../keyvalue/TestKeyValueBlockIterator.java | 34 +-
.../container/keyvalue/TestKeyValueContainer.java | 85 +-
.../keyvalue/TestKeyValueContainerCheck.java | 159 +--
...a => TestKeyValueContainerIntegrityChecks.java} | 163 +---
.../TestKeyValueContainerMarkUnhealthy.java | 8 +-
.../TestKeyValueContainerMetadataInspector.java | 360 +++++++
.../container/keyvalue/TestKeyValueHandler.java | 31 +-
.../TestKeyValueHandlerWithUnhealthyContainer.java | 14 -
.../container/keyvalue/TestTarContainerPacker.java | 8 +-
.../keyvalue/impl/AbstractTestChunkManager.java | 8 +-
.../keyvalue/impl/CommonChunkManagerTestCases.java | 4 +-
.../keyvalue/impl/TestBlockManagerImpl.java | 67 +-
.../keyvalue/impl/TestChunkManagerDummyImpl.java | 6 +-
.../keyvalue/impl/TestFilePerBlockStrategy.java | 6 +-
.../keyvalue/impl/TestFilePerChunkStrategy.java | 12 +-
.../container/ozoneimpl/TestContainerReader.java | 18 +-
.../TestContainerScrubberConfiguration.java | 97 ++
.../container/ozoneimpl/TestOzoneContainer.java | 16 +-
.../replication/TestGrpcOutputStream.java | 8 +-
.../replication/TestMeasuredReplicator.java | 49 +-
.../replication/TestReplicationConfig.java | 75 ++
.../replication/TestReplicationSupervisor.java | 16 +-
.../replication/TestSimpleContainerDownloader.java | 13 +-
.../testutils/BlockDeletingServiceTestImpl.java | 2 +-
.../upgrade/TestDataNodeStartupSlvLessThanMlv.java | 4 +-
.../upgrade/TestDatanodeUpgradeToScmHA.java | 6 +-
hadoop-hdds/dev-support/checkstyle/checkstyle.xml | 2 +
hadoop-hdds/docs/config.yaml | 5 +
hadoop-hdds/docs/content/concept/Containers.md | 1 -
hadoop-hdds/docs/content/concept/Datanodes.md | 4 +-
hadoop-hdds/docs/content/concept/OzoneManager.md | 4 +-
hadoop-hdds/docs/content/concept/Recon.md | 9 +-
hadoop-hdds/docs/content/feature/OM-HA.md | 5 +-
hadoop-hdds/docs/content/feature/Observability.md | 2 +-
.../{Observability.md => Observability.zh.md} | 63 +-
hadoop-hdds/docs/content/feature/PrefixFSO.md | 70 +-
hadoop-hdds/docs/content/feature/SCM-HA.md | 2 +-
hadoop-hdds/docs/content/feature/Topology.md | 2 +-
hadoop-hdds/docs/content/security/SecureOzone.md | 10 +-
.../docs/content/security/SecureOzone.zh.md | 34 +-
.../docs/content/security/SecuringDatanodes.md | 2 +-
.../docs/content/security/SecuringDatanodes.zh.md | 2 +-
.../docs/content/security/SecuringOzoneHTTP.md | 20 +-
hadoop-hdds/docs/content/security/SecuringTDE.md | 48 +-
.../docs/content/security/SecuringTDE.zh.md | 4 +-
hadoop-hdds/docs/content/start/FromSource.md | 2 +-
hadoop-hdds/docs/content/start/OnPrem.md | 1 +
hadoop-hdds/docs/content/start/OnPrem.zh.md | 2 +
hadoop-hdds/docs/content/tools/TestTools.md | 129 +--
hadoop-hdds/docs/content/tools/TestTools.zh.md | 129 +--
hadoop-hdds/docs/content/tools/_index.md | 1 -
hadoop-hdds/docs/content/tools/_index.zh.md | 1 -
hadoop-hdds/docs/dev-support/bin/generate-site.sh | 12 +-
.../docs/dev-support/bin/make_images_responsive.py | 57 ++
hadoop-hdds/docs/pom.xml | 4 +-
.../themes/ozonedoc/layouts/partials/navbar.html | 6 +-
.../themes/ozonedoc/layouts/partials/sidebar.html | 4 +-
.../themes/ozonedoc/layouts/shortcodes/image.html | 19 +
hadoop-hdds/framework/pom.xml | 4 +-
.../java/org/apache/hadoop/hdds/ExitManager.java | 16 +-
.../hadoop/hdds/protocol/SCMSecurityProtocol.java | 12 +
.../SCMSecurityProtocolClientSideTranslatorPB.java | 39 +-
.../scm/protocol/ScmBlockLocationProtocol.java | 2 +-
...inerLocationProtocolClientSideTranslatorPB.java | 90 +-
.../scm/update/client/CRLClientUpdateHandler.java | 2 +-
.../hdds/scm/update/client/ClientCRLStore.java | 4 +-
.../update/client/SCMUpdateServiceGrpcClient.java | 2 +-
.../scm/update/client/UpdateServiceConfig.java | 5 +-
.../x509/certificate/authority/BaseApprover.java | 2 +-
.../certificate/authority/DefaultCAServer.java | 6 +-
.../authority/PKIProfiles/DefaultCAProfile.java | 2 +-
.../authority/PKIProfiles/DefaultProfile.java | 32 +-
...ateClient.java => CommonCertificateClient.java} | 116 +--
.../client/DefaultCertificateClient.java | 32 +-
.../certificate/client/OMCertificateClient.java | 83 +-
.../certificate/client/ReconCertificateClient.java | 37 +-
.../certificates/utils/CertificateSignRequest.java | 10 +-
.../hadoop/hdds/security/x509/crl/CRLInfo.java | 2 +-
.../hdds/security/x509/crl/CRLInfoCodec.java | 2 +-
...va => FixedThreadPoolWithAffinityExecutor.java} | 70 +-
.../hadoop/hdds/server/http/ProfileServlet.java | 2 +-
.../hadoop/hdds/utils/DBCheckpointMetrics.java | 2 +-
.../java/org/apache/hadoop/hdds/utils/HAUtils.java | 4 +-
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 5 +-
.../hadoop/hdds/utils/MetadataKeyFilters.java | 2 +-
.../apache/hadoop/hdds/utils/TransactionInfo.java | 2 +-
.../hadoop/hdds/utils/db/DBConfigFromFile.java | 4 +-
.../org/apache/hadoop/hdds/utils/db/DBStore.java | 10 +
.../hadoop/hdds/utils/db/DBStoreBuilder.java | 4 +-
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 11 +
.../hadoop/hdds/utils/db/RDBStoreIterator.java | 19 +-
.../apache/hadoop/hdds/utils/db/TableIterator.java | 12 -
.../apache/hadoop/hdds/utils/db/TypedTable.java | 20 +-
.../hadoop/hdds/utils/db/cache/CacheKey.java | 2 +-
.../hadoop/hdds/utils/db/cache/EpochEntry.java | 2 +-
.../x509/certificate/authority/MockCAStore.java | 2 +-
.../certificate/authority/TestDefaultCAServer.java | 6 +-
.../client/TestDefaultCertificateClient.java | 2 +-
.../x509/certificate/utils/TestCRLCodec.java | 2 +-
.../certificates/TestCertificateSignRequest.java | 4 +-
.../x509/certificates/TestRootCertificate.java | 2 +-
.../security/x509/keys/TestHDDSKeyGenerator.java | 2 +-
.../hadoop/hdds/server/events/TestEventQueue.java | 35 +-
.../hadoop/hdds/server/http/TestHtmlQuoting.java | 5 +-
.../hadoop/hdds/utils/db/TestDBStoreBuilder.java | 10 +-
.../apache/hadoop/hdds/utils/db/TestRDBStore.java | 48 +-
.../hadoop/hdds/utils/db/TestRDBStoreIterator.java | 21 +-
.../hadoop/hdds/utils/db/TestRDBTableStore.java | 4 +-
.../hdds/utils/db/TestTypedRDBTableStore.java | 2 +-
.../hadoop/hdds/utils/db/cache/TestTableCache.java | 20 +-
hadoop-hdds/hadoop-dependency-client/pom.xml | 4 +-
hadoop-hdds/hadoop-dependency-server/pom.xml | 4 +-
hadoop-hdds/hadoop-dependency-test/pom.xml | 4 +-
hadoop-hdds/interface-admin/pom.xml | 4 +-
.../src/main/proto/ScmAdminProtocol.proto | 40 +-
.../interface-admin/src/main/resources/proto.lock | 504 ++++++++++
hadoop-hdds/interface-client/pom.xml | 4 +-
.../src/main/proto/DatanodeClientProtocol.proto | 4 +-
.../interface-client/src/main/proto/hdds.proto | 43 +
.../interface-client/src/main/resources/proto.lock | 408 +++++++-
hadoop-hdds/interface-server/pom.xml | 4 +-
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 2 +-
.../src/main/proto/ScmServerSecurityProtocol.proto | 8 +-
.../interface-server/src/main/resources/proto.lock | 1024 +++++++++++++++++++
hadoop-hdds/pom.xml | 4 +-
hadoop-hdds/server-scm/pom.xml | 14 +-
.../hadoop/hdds/scm/SCMCommonPlacementPolicy.java | 2 +-
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 2 +-
.../scm/block/DeletedBlockLogStateManagerImpl.java | 10 -
.../scm/container/CloseContainerEventHandler.java | 12 +-
.../hdds/scm/container/ContainerManager.java | 7 +
.../hdds/scm/container/ContainerManagerImpl.java | 78 +-
.../hdds/scm/container/ContainerReplicaCount.java | 28 +-
.../hdds/scm/container/ContainerReportHandler.java | 2 +-
.../hdds/scm/container/ContainerStateManager.java | 10 +-
.../scm/container/ContainerStateManagerImpl.java | 29 +-
.../hdds/scm/container/ReplicationManager.java | 131 ++-
.../balancer/AbstractFindTargetGreedy.java | 245 +++++
.../scm/container/balancer/ContainerBalancer.java | 285 +++---
.../balancer/ContainerBalancerConfiguration.java | 286 +++---
.../ContainerBalancerSelectionCriteria.java | 22 +-
.../scm/container/balancer/FindSourceGreedy.java | 158 +++
.../scm/container/balancer/FindSourceStrategy.java | 67 ++
.../scm/container/balancer/FindTargetGreedy.java | 135 ---
.../FindTargetGreedyByNetworkTopology.java | 79 ++
.../balancer/FindTargetGreedyByUsageInfo.java | 52 +
.../scm/container/balancer/FindTargetStrategy.java | 32 +-
.../ContainerPlacementPolicyFactory.java | 2 +-
.../algorithms/SCMContainerPlacementCapacity.java | 2 +-
.../algorithms/SCMContainerPlacementMetrics.java | 2 +-
.../algorithms/SCMContainerPlacementRackAware.java | 12 +-
.../container/placement/metrics/SCMMetrics.java | 2 +-
.../replication/ReplicationManagerMetrics.java | 83 ++
.../scm/container/states/ContainerStateMap.java | 30 +-
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 2 +-
.../apache/hadoop/hdds/scm/ha/HASecurityUtils.java | 2 +-
.../hadoop/hdds/scm/ha/InterSCMGrpcClient.java | 4 +-
.../org/apache/hadoop/hdds/scm/ha/RatisUtil.java | 6 +-
.../org/apache/hadoop/hdds/scm/ha/SCMContext.java | 23 +-
.../hdds/scm/ha/SCMDBCheckpointProvider.java | 2 +-
.../hadoop/hdds/scm/ha/SCMHAInvocationHandler.java | 9 +-
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 35 +-
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 3 +-
.../hadoop/hdds/scm/ha/SCMServiceManager.java | 3 +-
.../apache/hadoop/hdds/scm/ha/SCMStateMachine.java | 13 +-
.../apache/hadoop/hdds/scm/ha/io/CodecFactory.java | 2 +-
.../hdds/scm/metadata/SCMMetadataStoreImpl.java | 4 +-
.../hdds/scm/metadata/X509CertificateCodec.java | 2 +-
.../apache/hadoop/hdds/scm/node/CommandQueue.java | 2 +-
.../hdds/scm/node/DatanodeAdminMonitorImpl.java | 33 +-
.../hdds/scm/node/NodeDecommissionManager.java | 32 +-
.../apache/hadoop/hdds/scm/node/NodeManager.java | 2 +-
.../hadoop/hdds/scm/node/NodeStateManager.java | 2 +-
.../apache/hadoop/hdds/scm/node/NodeStatus.java | 4 +-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 8 +-
.../hadoop/hdds/scm/node/SCMNodeMetrics.java | 8 +-
.../hdds/scm/node/SCMNodeStorageStatMap.java | 2 +-
.../hdds/scm/node/states/Node2ObjectsMap.java | 2 +-
.../hadoop/hdds/scm/node/states/NodeStateMap.java | 2 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 2 +-
.../hdds/scm/pipeline/PipelineManagerImpl.java | 27 +-
.../hdds/scm/pipeline/PipelinePlacementPolicy.java | 24 +-
.../hdds/scm/pipeline/PipelineReportHandler.java | 4 +-
.../scm/pipeline/PipelineStateManagerImpl.java | 7 +-
.../hadoop/hdds/scm/pipeline/PipelineStateMap.java | 11 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 2 +-
.../scm/pipeline/WritableContainerFactory.java | 2 +-
.../SCMSecurityProtocolServerSideTranslatorPB.java | 28 +
...lockLocationProtocolServerSideTranslatorPB.java | 2 +-
...inerLocationProtocolServerSideTranslatorPB.java | 86 +-
.../hdds/scm/safemode/ContainerSafeModeRule.java | 4 +-
.../hdds/scm/safemode/DataNodeSafeModeRule.java | 2 +-
.../hdds/scm/server/SCMBlockProtocolServer.java | 16 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 96 +-
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 20 +
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 12 +-
.../hadoop/hdds/scm/server/SCMPolicyProvider.java | 5 +-
.../hdds/scm/server/SCMSecurityProtocolServer.java | 16 +-
.../hadoop/hdds/scm/server/SCMStorageConfig.java | 2 +-
.../hdds/scm/server/StorageContainerManager.java | 58 +-
.../server/StorageContainerManagerHttpServer.java | 4 +-
.../scm/server/StorageContainerManagerStarter.java | 2 +-
.../scm/server/upgrade/SCMUpgradeFinalizer.java | 5 +-
.../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 692 ++++++++++++-
.../apache/hadoop/hdds/scm/TestHddsServerUtil.java | 2 +-
.../hadoop/hdds/scm/TestHddsServerUtils.java | 37 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 699 -------------
.../hadoop/hdds/scm/block/TestBlockManager.java | 22 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 4 +-
.../command/TestCommandStatusReportHandler.java | 4 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 18 +-
.../hdds/scm/container/SimpleMockNodeManager.java | 8 +-
.../container/TestCloseContainerEventHandler.java | 4 +-
.../scm/container/TestContainerManagerImpl.java | 20 +-
.../scm/container/TestContainerReportHandler.java | 32 +-
.../scm/container/TestContainerStateManager.java | 6 +-
.../TestIncrementalContainerReportHandler.java | 34 +-
.../hdds/scm/container/TestReplicationManager.java | 309 ++++--
.../scm/container/TestUnknownContainerReport.java | 2 +-
.../container/balancer/TestContainerBalancer.java | 211 ++--
.../container/balancer/TestFindTargetStrategy.java | 196 ++++
.../algorithms/TestContainerPlacementFactory.java | 12 +-
.../TestSCMContainerPlacementCapacity.java | 18 +-
.../TestSCMContainerPlacementRackAware.java | 24 +-
.../TestSCMContainerPlacementRandom.java | 16 +-
.../replication/TestReplicationManagerMetrics.java | 97 ++
.../states/TestContainerReplicaCount.java | 15 +-
.../hdds/scm/crl/TestCRLStatusReportHandler.java | 4 +-
.../hdds/scm/ha/TestReplicationAnnotation.java | 10 +-
.../hadoop/hdds/scm/ha/TestSCMHAConfiguration.java | 26 +-
.../hadoop/hdds/scm/ha/TestSCMRatisRequest.java | 2 +-
.../hdds/scm/metadata/TestPipelineIDCodec.java | 2 +-
.../hdds/scm/node/TestContainerPlacement.java | 10 +-
.../hdds/scm/node/TestDatanodeAdminMonitor.java | 11 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 49 +-
.../hdds/scm/node/TestNodeDecommissionManager.java | 16 +-
.../hdds/scm/node/TestNodeReportHandler.java | 10 +-
.../hadoop/hdds/scm/node/TestNodeStateManager.java | 2 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 96 +-
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 20 +-
.../hadoop/hdds/scm/node/TestStatisticsUpdate.java | 16 +-
.../hdds/scm/node/states/TestNodeStateMap.java | 4 +-
.../scm/pipeline/MockRatisPipelineProvider.java | 2 +-
.../TestPipelineDatanodesIntersection.java | 4 +-
.../hdds/scm/pipeline/TestPipelineManagerImpl.java | 64 +-
.../scm/pipeline/TestPipelinePlacementPolicy.java | 16 +-
.../scm/pipeline/TestPipelineStateManagerImpl.java | 8 +-
.../scm/pipeline/TestRatisPipelineProvider.java | 4 +-
.../TestOneReplicaPipelineSafeModeRule.java | 8 +-
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 8 +-
.../scm/server/TestSCMBlockProtocolServer.java | 4 +-
.../hadoop/hdds/scm/server/TestSCMCertStore.java | 6 +-
.../server/TestSCMDatanodeHeartbeatDispatcher.java | 54 +-
.../server/TestStorageContainerManagerStarter.java | 6 +-
.../server/TestSCMUpdateServiceGrpcServer.java | 28 +-
.../TestSCMHAUnfinalizedStateValidationAction.java | 5 +-
.../scm/upgrade/TestScmStartupSlvLessThanMlv.java | 6 +-
.../ozone/container/common/TestEndPoint.java | 23 +-
.../placement/TestContainerPlacement.java | 4 +-
.../testutils/ReplicationNodeManagerMock.java | 2 +-
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 11 +-
hadoop-hdds/test-utils/pom.xml | 4 +-
.../org/apache/ozone/test/GenericTestUtils.java | 18 +-
.../org/apache/ozone/test/LambdaTestUtils.java | 4 +-
hadoop-hdds/tools/pom.xml | 13 +-
.../org/apache/hadoop/hdds/cli/OzoneAdmin.java | 11 +
.../hdds/scm/cli/ContainerBalancerCommands.java | 19 +-
.../scm/cli/ContainerBalancerStartSubcommand.java | 50 +-
.../scm/cli/ContainerBalancerStatusSubcommand.java | 2 +-
.../hdds/scm/cli/ContainerOperationClient.java | 34 +-
.../cli/ReplicationManagerStatusSubcommand.java | 2 +-
.../hdds/scm/cli/SafeModeCheckSubcommand.java | 2 +-
.../hdds/scm/cli/SafeModeExitSubcommand.java | 2 +-
.../hadoop/hdds/scm/cli/cert/CertCommands.java | 4 +-
.../hdds/scm/cli/container/ContainerCommands.java | 3 +-
.../hdds/scm/cli/container/InfoSubcommand.java | 61 +-
.../hdds/scm/cli/container/ReportSubcommand.java | 116 +++
.../scm/cli/datanode/DecommissionSubCommand.java | 2 +-
.../hdds/scm/cli/datanode/ListInfoSubcommand.java | 38 +-
.../scm/cli/datanode/MaintenanceSubCommand.java | 4 +-
.../scm/cli/datanode/RecommissionSubCommand.java | 2 +-
.../hdds/scm/cli/datanode/UsageInfoSubcommand.java | 41 +-
.../hdds/scm/cli/container/TestInfoSubCommand.java | 249 +++++
.../scm/cli/container/TestReportSubCommand.java | 159 +++
.../datanode/TestContainerBalancerSubCommand.java | 8 +-
.../scm/cli/datanode/TestListInfoSubcommand.java | 6 +-
hadoop-ozone/client/pom.xml | 19 +-
.../org/apache/hadoop/ozone/client/BucketArgs.java | 29 +-
.../apache/hadoop/ozone/client/ObjectStore.java | 6 +-
.../apache/hadoop/ozone/client/OzoneBucket.java | 61 +-
.../hadoop/ozone/client/OzoneClientFactory.java | 2 +-
.../org/apache/hadoop/ozone/client/OzoneKey.java | 2 +-
.../apache/hadoop/ozone/client/OzoneVolume.java | 2 +-
.../checksum/AbstractBlockChecksumComputer.java} | 36 +-
.../client/checksum/BaseFileChecksumHelper.java | 193 ++++
.../checksum/ReplicatedBlockChecksumComputer.java | 72 ++
.../checksum/ReplicatedFileChecksumHelper.java | 187 ++++
.../ozone/client/checksum}/package-info.java | 10 +-
.../ozone/client/io/BlockOutputStreamEntry.java | 6 +-
.../client/io/BlockOutputStreamEntryPool.java | 1 -
.../hadoop/ozone/client/io/KeyInputStream.java | 8 +-
.../client/io/MultipartCryptoKeyInputStream.java | 4 +
.../hadoop/ozone/client/io/OzoneInputStream.java | 33 +-
.../ozone/client/protocol/ClientProtocol.java | 47 +
.../hadoop/ozone/client/rpc/OzoneKMSUtil.java | 4 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 224 ++++-
.../hadoop/ozone/client/TestHddsClientUtils.java | 4 +-
.../TestReplicatedBlockChecksumComputer.java | 68 ++
.../checksum/TestReplicatedFileChecksumHelper.java | 323 ++++++
.../ozone/client/checksum}/package-info.java | 10 +-
.../hadoop/ozone/client/rpc/RpcClientTest.java | 217 +++++
.../hadoop/ozone/client/rpc/TestOzoneKMSUtil.java | 3 +-
hadoop-ozone/common/pom.xml | 8 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 147 ++-
.../java/org/apache/hadoop/ozone/OzoneAcl.java | 8 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 32 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 14 +-
.../hadoop/ozone/om/helpers/BucketLayout.java | 17 +
.../hadoop/ozone/om/helpers/OMNodeDetails.java | 73 +-
.../hadoop/ozone/om/helpers/OmBucketArgs.java | 46 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 41 +-
.../hadoop/ozone/om/helpers/OmDirectoryInfo.java | 7 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 15 +-
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 19 +-
.../ozone/om/helpers/OmKeyLocationInfoGroup.java | 4 +-
.../ozone/om/helpers/OmMultipartKeyInfo.java | 2 +-
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 6 +-
.../hadoop/ozone/om/helpers/OzoneAclUtil.java | 4 +-
.../hadoop/ozone/om/helpers/OzoneFSUtils.java | 26 +-
.../hadoop/ozone/om/helpers/RepeatedOmKeyInfo.java | 10 +-
.../hadoop/ozone/om/helpers/ServiceInfo.java | 43 +-
.../hadoop/ozone/om/helpers/WithObjectID.java | 2 +-
.../hadoop/ozone/om/protocol/OMAdminProtocol.java} | 35 +-
.../hadoop/ozone/om/protocol/OMConfiguration.java | 104 ++
.../ozone/om/protocol/OzoneManagerProtocol.java | 12 +
.../apache/hadoop/ozone/om/protocol/S3Auth.java} | 39 +-
.../protocolPB/OMAdminProtocolClientSideImpl.java | 224 +++++
.../ozone/om/protocolPB/OMAdminProtocolPB.java} | 42 +-
.../om/protocolPB/OzoneManagerClientProtocol.java | 27 +-
...OzoneManagerProtocolClientSideTranslatorPB.java | 104 +-
.../apache/hadoop/ozone/protocolPB/OMPBHelper.java | 51 +-
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 3 +-
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 2 +-
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 2 +-
.../org/apache/hadoop/ozone/util/RadixTree.java | 2 +-
.../apache/hadoop/ozone/web/utils/OzoneUtils.java | 18 +-
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 10 +-
.../ozone/om/ha/TestOMFailoverProxyProvider.java | 2 +-
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 41 +-
.../hadoop/ozone/om/lock/TestOzoneManagerLock.java | 2 +-
.../ozone/security/TestGDPRSymmetricKey.java | 2 +-
.../ozone/security/acl/TestOzoneObjInfo.java | 9 +-
.../apache/hadoop/ozone/util/TestRadixTree.java | 4 +-
hadoop-ozone/csi/pom.xml | 4 +-
hadoop-ozone/datanode/pom.xml | 4 +-
hadoop-ozone/dev-support/checks/_lib.sh | 2 +-
hadoop-ozone/dev-support/checks/acceptance.sh | 5 +-
hadoop-ozone/dev-support/checks/bats.sh | 8 +-
hadoop-ozone/dev-support/checks/checkstyle.sh | 3 +
hadoop-ozone/dev-support/checks/coverage.sh | 1 -
hadoop-ozone/dev-support/checks/kubernetes.sh | 4 +-
hadoop-ozone/dist/README.md | 6 +-
hadoop-ozone/dist/pom.xml | 6 +-
.../dist/src/main/assemblies/ozone-src.xml | 1 +
.../dist/src/main/compose/compatibility/.env | 1 +
.../main/compose/compatibility/docker-compose.yaml | 2 +-
hadoop-ozone/dist/src/main/compose/ozone-csi/.env | 1 +
.../src/main/compose/ozone-csi/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozone-csi/docker-config | 1 +
hadoop-ozone/dist/src/main/compose/ozone-ha/.env | 1 +
.../src/main/compose/ozone-ha/docker-compose.yaml | 2 +-
.../dist/src/main/compose/ozone-ha/docker-config | 4 +
.../dist/src/main/compose/ozone-mr/hadoop27/.env | 1 +
.../compose/ozone-mr/hadoop27/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozone-mr/hadoop31/.env | 1 +
.../compose/ozone-mr/hadoop31/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozone-mr/hadoop32/.env | 1 +
.../compose/ozone-mr/hadoop32/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozone-mr/hadoop33/.env | 1 +
.../compose/ozone-mr/hadoop33/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozone-om-ha/.env | 1 +
.../dist/src/main/compose/ozone-om-ha/Dockerfile | 3 +-
.../dist/src/main/compose/ozone-om-prepare/.env | 6 +-
.../compose/ozone-om-prepare/docker-compose.yaml | 2 +-
.../dist/src/main/compose/ozone-topology/.env | 1 +
.../compose/ozone-topology/docker-compose.yaml | 16 +-
hadoop-ozone/dist/src/main/compose/ozone/.env | 1 +
hadoop-ozone/dist/src/main/compose/ozone/README.md | 6 -
.../src/main/compose/ozone/docker-compose.yaml | 12 +-
.../dist/src/main/compose/ozone/docker-config | 1 +
.../dist/src/main/compose/ozone/freon-ockg.yaml | 2 +-
.../dist/src/main/compose/ozone/freon-rk.yaml | 2 +-
hadoop-ozone/dist/src/main/compose/ozone/run.sh | 4 -
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 23 +-
.../dist/src/main/compose/ozoneblockade/.env | 1 +
.../main/compose/ozoneblockade/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozones3-haproxy/.env | 1 +
.../compose/ozones3-haproxy/docker-compose.yaml | 12 +-
.../dist/src/main/compose/ozonescripts/.env | 1 +
.../dist/src/main/compose/ozonescripts/Dockerfile | 3 +-
.../main/compose/ozonescripts/docker-compose.yaml | 3 +
.../dist/src/main/compose/ozonesecure-ha/.env | 1 +
.../compose/ozonesecure-ha/docker-compose.yaml | 22 +-
.../src/main/compose/ozonesecure-ha/docker-config | 2 +-
.../compose/ozonesecure-ha/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/om.keytab | Bin 278 -> 0 bytes
.../compose/ozonesecure-ha/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure-ha/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure-ha/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../ozonesecure-ha/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../dist/src/main/compose/ozonesecure-mr/.env | 1 +
.../compose/ozonesecure-mr/docker-compose.yaml | 8 +-
.../src/main/compose/ozonesecure-mr/docker-config | 6 +-
.../compose/ozonesecure-mr/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/om.keytab | Bin 278 -> 0 bytes
.../compose/ozonesecure-mr/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure-mr/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure-mr/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../ozonesecure-mr/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../dist/src/main/compose/ozonesecure-mr/test.sh | 1 -
.../dist/src/main/compose/ozonesecure/.env | 1 +
.../main/compose/ozonesecure/docker-compose.yaml | 10 +-
.../src/main/compose/ozonesecure/docker-config | 8 +-
.../main/compose/ozonesecure/keytabs/HTTP.keytab | Bin 144 -> 0 bytes
.../src/main/compose/ozonesecure/keytabs/dn.keytab | Bin 278 -> 0 bytes
.../src/main/compose/ozonesecure/keytabs/om.keytab | Bin 278 -> 0 bytes
.../main/compose/ozonesecure/keytabs/recon.keytab | Bin 296 -> 0 bytes
.../main/compose/ozonesecure/keytabs/s3g.keytab | Bin 434 -> 0 bytes
.../main/compose/ozonesecure/keytabs/scm.keytab | Bin 586 -> 0 bytes
.../compose/ozonesecure/keytabs/testuser.keytab | Bin 152 -> 0 bytes
.../compose/ozonesecure/keytabs/testuser2.keytab | Bin 154 -> 0 bytes
.../dist/src/main/compose/ozonesecure/test.sh | 2 +-
hadoop-ozone/dist/src/main/compose/restart/.env | 2 +-
.../src/main/compose/restart/docker-compose.yaml | 2 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 3 +-
.../dist/src/main/compose/upgrade/compose/ha/.env | 1 +
.../main/compose/upgrade/compose/ha/docker-config | 4 +-
.../src/main/compose/upgrade/compose/non-ha/.env | 1 +
.../compose/upgrade/compose/non-ha/docker-config | 2 +-
.../dist/src/main/compose/upgrade/testlib.sh | 1 -
hadoop-ozone/dist/src/main/compose/xcompat/.env | 1 +
.../dist/src/main/compose/xcompat/clients.yaml | 20 +-
.../dist/src/main/compose/xcompat/docker-config | 1 +
.../dist/src/main/compose/xcompat/new-cluster.yaml | 2 +-
hadoop-ozone/dist/src/main/compose/xcompat/test.sh | 17 +-
hadoop-ozone/dist/src/main/docker/Dockerfile | 1 -
.../dist/src/main/dockerlibexec/entrypoint.sh | 2 +-
.../k8s/definitions/ozone-csi/csi-controller.yaml | 2 +-
.../main/k8s/examples/getting-started/Flekszible | 2 +
.../examples/getting-started/config-configmap.yaml | 1 +
.../examples/getting-started/kustomization.yaml} | 17 +-
.../dist/src/main/k8s/examples/minikube/Flekszible | 2 +
.../k8s/examples/minikube/config-configmap.yaml | 1 +
.../examples/minikube/kustomization.yaml} | 17 +-
.../src/main/k8s/examples/ozone-dev/Flekszible | 1 +
.../k8s/examples/ozone-dev/config-configmap.yaml | 1 +
.../examples/ozone-dev/kustomization.yaml} | 26 +-
.../dist/src/main/k8s/examples/ozone-ha/Flekszible | 3 +
.../examples/ozone-ha/kustomization.yaml} | 12 +-
.../dist/src/main/k8s/examples/ozone/Flekszible | 3 +-
.../main/k8s/examples/ozone/config-configmap.yaml | 1 +
.../.env => k8s/examples/ozone/kustomization.yaml} | 12 +-
hadoop-ozone/dist/src/main/k8s/examples/testlib.sh | 11 +-
.../main/smoketest/basic/ozone-shell-single.robot | 2 +-
.../src/main/smoketest/basic/ozone-shell.robot | 15 +-
.../src/main/smoketest/compatibility/read.robot | 9 +-
.../setup.robot} | 17 +-
.../src/main/smoketest/compatibility/write.robot | 7 +-
.../dist/src/main/smoketest/createmrenv.robot | 2 +-
.../dist/src/main/smoketest/freon/generate.robot | 9 +
.../dist/src/main/smoketest/freon/validate.robot | 9 +
.../dist/src/main/smoketest/mapreduce.robot | 1 +
.../src/main/smoketest/ozonefs/hadoopo3fs.robot | 1 +
.../dist/src/main/smoketest/ozonefs/setup.robot | 12 +-
.../src/main/smoketest/s3/MultipartUpload.robot | 21 +-
.../dist/src/main/smoketest/s3/awss3.robot | 8 +
.../dist/src/main/smoketest/s3/commonawslib.robot | 36 +
.../dist/src/main/smoketest/s3/objectcopy.robot | 20 +-
.../dist/src/main/smoketest/s3/objectdelete.robot | 34 +-
.../src/main/smoketest/s3/objectmultidelete.robot | 24 +-
.../dist/src/main/smoketest/s3/objectputget.robot | 42 +-
.../main/smoketest/security/ozone-secure-fs.robot | 12 +-
hadoop-ozone/dist/src/shell/ozone/ozone | 17 -
hadoop-ozone/dist/src/shell/upgrade/1.0.0.sh | 3 +-
.../fault-injection-test/mini-chaos-tests/pom.xml | 4 +-
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 49 +-
.../hadoop/ozone/MiniOzoneLoadGenerator.java | 2 +-
.../hadoop/ozone/TestAllMiniChaosOzoneCluster.java | 2 +-
.../hadoop/ozone/loadgenerators/LoadBucket.java | 2 +-
.../fault-injection-test/network-tests/pom.xml | 2 +-
hadoop-ozone/fault-injection-test/pom.xml | 4 +-
hadoop-ozone/insight/pom.xml | 5 +-
.../hadoop/ozone/insight/TestBaseInsightPoint.java | 7 +-
hadoop-ozone/integration-test/pom.xml | 14 +-
.../ozone/TestDirectoryDeletingServiceWithFSO.java | 13 +-
.../hadoop/fs/ozone/TestOzoneFSInputStream.java | 4 +
.../fs/ozone/TestOzoneFSWithObjectStoreCreate.java | 17 +-
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 50 +-
.../fs/ozone/TestOzoneFileInterfacesWithFSO.java | 25 +-
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 52 +-
.../fs/ozone/TestOzoneFileSystemMetrics.java | 3 +
.../fs/ozone/TestOzoneFileSystemMissingParent.java | 2 +
.../fs/ozone/TestOzoneFileSystemWithLinks.java | 252 +++++
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 13 +-
.../hadoop/fs/ozone/TestRootedOzoneFileSystem.java | 188 +++-
.../fs/ozone/TestRootedOzoneFileSystemWithFSO.java | 12 +-
.../contract/ITestOzoneContractDistCpWithFSO.java | 65 ++
.../fs/ozone/contract/ITestOzoneContractUtils.java | 4 +-
.../hadoop/fs/ozone/contract/OzoneContract.java | 23 +-
.../TestContainerStateManagerIntegration.java | 28 +-
.../metrics/TestSCMContainerManagerMetrics.java | 2 +
.../hdds/scm/pipeline/TestLeaderChoosePolicy.java | 2 +-
.../hdds/scm/pipeline/TestMultiRaftSetup.java | 7 +-
.../hdds/scm/pipeline/TestPipelineClose.java | 4 +-
.../TestRatisPipelineCreateAndDestroy.java | 2 +-
.../safemode/TestSCMSafeModeWithPipelineRules.java | 6 +-
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 10 +-
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 6 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 36 +-
.../hadoop/ozone/MiniOzoneClusterProvider.java | 4 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 518 ++++++----
.../hadoop/ozone/MiniOzoneOMHAClusterImpl.java | 116 ---
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 25 +-
.../hadoop/ozone/StandardOutputTestBase.java | 84 ++
.../ozone/TestContainerBalancerOperations.java | 23 +-
.../hadoop/ozone/TestContainerOperations.java | 2 +-
.../java/org/apache/hadoop/ozone/TestDataUtil.java | 2 +-
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 6 +-
.../hadoop/ozone/TestMiniOzoneOMHACluster.java | 8 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 8 +
.../hadoop/ozone/TestSecureOzoneCluster.java | 32 +-
.../hadoop/ozone/TestStorageContainerManager.java | 16 +-
.../ozone/client/CertificateClientTestImpl.java | 4 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 9 +-
.../hadoop/ozone/client/rpc/TestCommitWatcher.java | 2 +-
.../client/rpc/TestContainerStateMachine.java | 3 +
.../rpc/TestContainerStateMachineFailures.java | 238 ++++-
.../rpc/TestContainerStateMachineFlushDelay.java | 12 +-
.../client/rpc/TestDeleteWithSlowFollower.java | 11 +-
.../client/rpc/TestDiscardPreallocatedBlocks.java | 2 +-
.../client/rpc/TestFailureHandlingByClient.java | 116 ++-
.../rpc/TestFailureHandlingByClientFlushDelay.java | 5 +-
.../rpc/TestMultiBlockWritesWithDnFailures.java | 4 +-
.../client/rpc/TestOzoneAtRestEncryption.java | 88 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 63 +-
.../rpc/TestOzoneClientRetriesOnExceptions.java | 4 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 190 ++--
.../rpc/TestOzoneRpcClientForAclAuditLog.java | 21 +-
.../TestOzoneRpcClientWithKeyLatestVersion.java | 209 ++++
.../hadoop/ozone/client/rpc/TestReadRetries.java | 20 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 54 +-
.../ozone/client/rpc/TestWatchForCommit.java | 2 +-
.../client/rpc/read/TestChunkInputStream.java | 4 +-
.../ozone/client/rpc/read/TestInputStreamBase.java | 17 +-
.../ozone/client/rpc/read/TestKeyInputStream.java | 12 +-
.../apache/hadoop/ozone/container/TestHelper.java | 15 +-
.../commandhandler/TestBlockDeletion.java | 1 +
.../TestCloseContainerByPipeline.java | 2 +-
.../commandhandler/TestCloseContainerHandler.java | 4 +-
.../commandhandler/TestDeleteContainerHandler.java | 5 +-
.../transport/server/ratis/TestCSMMetrics.java | 10 +-
.../container/metrics/TestContainerMetrics.java | 5 +-
.../container/ozoneimpl/TestOzoneContainer.java | 27 +-
.../ozoneimpl/TestOzoneContainerWithTLS.java | 6 +-
.../ozoneimpl/TestSecureOzoneContainer.java | 2 +
.../container/server/TestContainerServer.java | 3 +-
.../server/TestSecureContainerServer.java | 20 +-
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +
.../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 7 +-
.../TestDatanodeHddsVolumeFailureDetection.java | 9 +-
.../ozone/freon/TestHadoopDirTreeGenerator.java | 5 +-
.../freon/TestHadoopDirTreeGeneratorWithFSO.java | 6 +-
.../ozone/freon/TestHadoopNestedDirGenerator.java | 29 +-
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 1 +
.../hadoop/ozone/fsck/TestContainerMapper.java | 16 +-
.../hadoop/ozone/om/TestAddRemoveOzoneManager.java | 423 ++++++++
.../apache/hadoop/ozone/om/TestBucketOwner.java | 253 +++++
.../ozone/om/TestContainerReportWithKeys.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 403 ++++----
.../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +-
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 11 +-
.../ozone/om/TestOMStartupWithBucketLayout.java | 161 +++
.../hadoop/ozone/om/TestOMUpgradeFinalization.java | 6 +-
.../apache/hadoop/ozone/om/TestObjectStore.java | 142 ++-
.../hadoop/ozone/om/TestObjectStoreWithFSO.java | 15 +-
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 6 +-
.../org/apache/hadoop/ozone/om/TestOmLDBCli.java | 18 +-
.../org/apache/hadoop/ozone/om/TestOmMetrics.java | 2 +-
.../ozone/om/TestOmStartupSlvLessThanMlv.java | 6 +-
.../hadoop/ozone/om/TestOzoneManagerBootstrap.java | 216 -----
.../ozone/om/TestOzoneManagerConfiguration.java | 4 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 12 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerHAWithACL.java | 8 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 4 +-
.../ozone/om/TestOzoneManagerHAWithFailover.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerPrepare.java | 2 +-
.../ozone/om/TestOzoneManagerRestInterface.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerRestart.java | 47 +-
.../hadoop/ozone/om/TestRecursiveAclWithFSO.java | 5 +-
.../hadoop/ozone/om/TestSecureOzoneManager.java | 3 +-
.../om/ratis/TestOzoneManagerRatisRequest.java | 4 +-
.../snapshot/TestOzoneManagerSnapshotProvider.java | 8 +-
.../hadoop/ozone/recon/TestReconScmHASnapshot.java | 70 ++
.../hadoop/ozone/recon/TestReconScmSnapshot.java | 133 +++
.../ozone/recon/TestReconWithOzoneManager.java | 15 +-
.../ozone/recon/TestReconWithOzoneManagerFSO.java | 20 +-
.../ozone/recon/TestReconWithOzoneManagerHA.java | 14 +-
.../hadoop/ozone/scm/TestAllocateContainer.java | 2 +-
.../hadoop/ozone/scm/TestCloseContainer.java | 4 +-
.../hadoop/ozone/scm/TestFailoverWithSCMHA.java | 2 +-
.../ozone/scm/TestSCMInstallSnapshotWithHA.java | 18 +-
.../org/apache/hadoop/ozone/scm/TestSCMMXBean.java | 6 +-
.../ozone/scm/TestStorageContainerManagerHA.java | 4 +-
.../hadoop/ozone/scm/TestXceiverClientGrpc.java | 10 +-
.../scm/node/TestDecommissionAndMaintenance.java | 20 +-
.../hadoop/ozone/scm/node/TestQueryNode.java | 2 +-
.../hadoop/ozone/shell/TestNSSummaryAdmin.java | 122 ++-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 28 +-
hadoop-ozone/interface-client/pom.xml | 4 +-
.../src/main/proto/OMAdminProtocol.proto | 84 ++
.../src/main/proto/OmClientProtocol.proto | 8 +-
.../interface-client/src/main/resources/proto.lock | 826 +++++++++++++++-
hadoop-ozone/interface-storage/pom.xml | 4 +-
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 5 +-
.../ozone/om/codec/RepeatedOmKeyInfoCodec.java | 2 +-
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 4 +-
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 40 +-
.../ozone/om/helpers/TestInstanceHelper.java | 68 --
.../hadoop/ozone/om/helpers/TestOmPrefixInfo.java | 39 +-
hadoop-ozone/ozone-manager/pom.xml | 4 +-
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 28 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 111 ---
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 1025 ++------------------
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 28 +-
.../apache/hadoop/ozone/om/OMPolicyProvider.java | 5 +-
.../apache/hadoop/ozone/om/OMStarterInterface.java | 2 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 26 +-
.../org/apache/hadoop/ozone/om/OzoneAclUtils.java | 125 +++
.../org/apache/hadoop/ozone/om/OzoneManager.java | 552 ++++++++---
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 4 +-
.../hadoop/ozone/om/OzoneManagerPrepareState.java | 4 +-
.../hadoop/ozone/om/OzoneManagerStarter.java | 61 +-
.../apache/hadoop/ozone/om/OzoneManagerUtils.java | 180 ++++
.../hadoop/ozone/om/OzonePrefixPathImpl.java | 18 +
.../hadoop/ozone/om/S3SecretManagerImpl.java | 2 +-
.../hadoop/ozone/om/TrashOzoneFileSystem.java | 17 +-
.../apache/hadoop/ozone/om/TrashPolicyOzone.java | 12 +-
.../org/apache/hadoop/ozone/om/VolumeManager.java | 40 +-
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 300 ------
.../hadoop/ozone/om/codec/OMDBDefinition.java | 10 +-
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 6 -
.../apache/hadoop/ozone/om/ha/OMHANodeDetails.java | 9 +-
.../ozone/om/ratis/OzoneManagerDoubleBuffer.java | 4 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 98 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 6 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 248 +----
.../hadoop/ozone/om/request/OMClientRequest.java | 116 ++-
.../ozone/om/request/OMKeyRequestFactory.java | 139 +++
.../om/request/bucket/OMBucketCreateRequest.java | 19 +-
.../om/request/bucket/OMBucketDeleteRequest.java | 9 +-
...tyRequest.java => OMBucketSetOwnerRequest.java} | 223 ++---
.../request/bucket/OMBucketSetPropertyRequest.java | 6 +-
.../request/bucket/acl/OMBucketSetAclRequest.java | 2 +-
.../om/request/file/OMDirectoryCreateRequest.java | 26 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 13 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 26 +-
.../request/file/OMFileCreateRequestWithFSO.java | 14 +-
.../ozone/om/request/file/OMFileRequest.java | 73 +-
.../om/request/key/OMAllocateBlockRequest.java | 33 +-
.../request/key/OMAllocateBlockRequestWithFSO.java | 14 +-
.../ozone/om/request/key/OMKeyCommitRequest.java | 31 +-
.../om/request/key/OMKeyCommitRequestWithFSO.java | 15 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 41 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 36 +-
.../ozone/om/request/key/OMKeyDeleteRequest.java | 29 +-
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 14 +-
.../ozone/om/request/key/OMKeyRenameRequest.java | 25 +-
.../om/request/key/OMKeyRenameRequestWithFSO.java | 19 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 62 +-
.../ozone/om/request/key/OMKeysDeleteRequest.java | 3 +-
.../key/OMOpenKeysDeleteRequest.java | 6 +-
.../om/request/key/OMPathsPurgeRequestWithFSO.java | 9 +-
.../om/request/key/OMTrashRecoverRequest.java | 2 +-
.../ozone/om/request/key/acl/OMKeyAclRequest.java | 31 +-
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 9 +-
.../om/request/key/acl/OMKeyAddAclRequest.java | 5 +-
.../request/key/acl/OMKeyAddAclRequestWithFSO.java | 8 +-
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 4 +-
.../key/acl/OMKeyRemoveAclRequestWithFSO.java | 9 +-
.../om/request/key/acl/OMKeySetAclRequest.java | 4 +-
.../request/key/acl/OMKeySetAclRequestWithFSO.java | 8 +-
.../S3InitiateMultipartUploadRequest.java | 35 +-
.../S3InitiateMultipartUploadRequestWithFSO.java | 31 +-
.../multipart/S3MultipartUploadAbortRequest.java | 39 +-
.../S3MultipartUploadAbortRequestWithFSO.java | 14 +-
.../S3MultipartUploadCommitPartRequest.java | 36 +-
.../S3MultipartUploadCommitPartRequestWithFSO.java | 12 +-
.../S3MultipartUploadCompleteRequest.java | 62 +-
.../S3MultipartUploadCompleteRequestWithFSO.java | 30 +-
.../om/request/volume/OMVolumeSetQuotaRequest.java | 6 +-
.../om/request/volume/acl/OMVolumeAclRequest.java | 2 +-
.../request/volume/acl/OMVolumeAddAclRequest.java | 2 +-
.../volume/acl/OMVolumeRemoveAclRequest.java | 2 +-
.../request/volume/acl/OMVolumeSetAclRequest.java | 2 +-
.../OMBucketSetOwnerResponse.java} | 71 +-
.../response/file/OMDirectoryCreateResponse.java | 10 +-
.../file/OMDirectoryCreateResponseWithFSO.java | 15 +-
.../om/response/file/OMFileCreateResponse.java | 6 +-
.../response/file/OMFileCreateResponseWithFSO.java | 5 +-
.../response/key/AbstractOMKeyDeleteResponse.java | 16 +-
.../om/response/key/OMAllocateBlockResponse.java | 13 +-
.../key/OMAllocateBlockResponseWithFSO.java | 14 +-
.../ozone/om/response/key/OMKeyCommitResponse.java | 11 +-
.../response/key/OMKeyCommitResponseWithFSO.java | 5 +-
.../ozone/om/response/key/OMKeyCreateResponse.java | 11 +-
.../response/key/OMKeyCreateResponseWithFSO.java | 6 +-
.../ozone/om/response/key/OMKeyDeleteResponse.java | 8 +-
.../response/key/OMKeyDeleteResponseWithFSO.java | 5 +-
.../ozone/om/response/key/OMKeyPurgeResponse.java | 3 +-
.../ozone/om/response/key/OMKeyRenameResponse.java | 18 +-
.../response/key/OMKeyRenameResponseWithFSO.java | 12 +-
.../om/response/key/OMKeysDeleteResponse.java | 6 +-
.../om/response/key/OMKeysRenameResponse.java | 10 +
.../om/response/key/OMOpenKeysDeleteResponse.java | 6 +-
.../response/key/OMPathsPurgeResponseWithFSO.java | 13 +-
.../om/response/key/OMTrashRecoverResponse.java | 8 +-
.../ozone/om/response/key/OmKeyResponse.java} | 36 +-
.../om/response/key/acl/OMKeyAclResponse.java | 17 +-
.../response/key/acl/OMKeyAclResponseWithFSO.java | 16 +-
.../S3InitiateMultipartUploadResponse.java | 14 +-
.../S3InitiateMultipartUploadResponseWithFSO.java | 10 +-
.../multipart/S3MultipartUploadAbortResponse.java | 14 +-
.../S3MultipartUploadAbortResponseWithFSO.java | 13 +-
.../S3MultipartUploadCommitPartResponse.java | 10 +-
...S3MultipartUploadCommitPartResponseWithFSO.java | 9 +-
.../S3MultipartUploadCompleteResponse.java | 40 +-
.../S3MultipartUploadCompleteResponseWithFSO.java | 11 +-
.../om/snapshot/OzoneManagerSnapshotProvider.java | 7 +
.../protocolPB/OMAdminProtocolServerSideImpl.java | 116 +++
...OzoneManagerProtocolServerSideTranslatorPB.java | 47 +-
.../protocolPB/OzoneManagerRequestHandler.java | 3 +-
.../hadoop/ozone/protocolPB/RequestHandler.java | 4 +-
.../hadoop/ozone/security/AWSV4AuthValidator.java | 10 -
.../OzoneDelegationTokenSecretManager.java | 2 +-
.../hadoop/ozone/security/OzoneSecretStore.java | 4 +-
.../hadoop/ozone/security/S3SecurityUtil.java | 26 +-
.../org/apache/hadoop/ozone/om/OmTestManagers.java | 122 +++
.../hadoop/ozone/om/TestBucketManagerImpl.java | 66 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 106 +-
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 72 +-
.../apache/hadoop/ozone/om/TestOMDBDefinition.java | 2 +-
.../hadoop/ozone/om/TestOmMetadataManager.java | 54 +-
.../hadoop/ozone/om/TestOzoneManagerStarter.java | 8 +-
.../apache/hadoop/ozone/om/TestTrashService.java | 40 +-
.../hadoop/ozone/om/failover/TestOMFailovers.java | 2 +-
...tOzoneManagerDoubleBufferWithDummyResponse.java | 2 +-
...TestOzoneManagerDoubleBufferWithOMResponse.java | 12 +-
.../om/ratis/TestOzoneManagerRatisServer.java | 2 +-
.../om/ratis/TestOzoneManagerStateMachine.java | 2 +-
...OMRequestUtils.java => OMRequestTestUtils.java} | 56 +-
.../request/TestOMClientRequestWithUserInfo.java | 2 +-
.../ozone/om/request/bucket/TestBucketRequest.java | 1 -
.../request/bucket/TestOMBucketCreateRequest.java | 8 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 11 +-
.../request/bucket/TestOMBucketDeleteRequest.java | 6 +-
....java => TestOMBucketDeleteRequestWithFSO.java} | 76 +-
.../bucket/TestOMBucketSetPropertyRequest.java | 8 +-
.../bucket/acl/TestOMBucketAddAclRequest.java | 12 +-
.../bucket/acl/TestOMBucketRemoveAclRequest.java | 14 +-
.../bucket/acl/TestOMBucketSetAclRequest.java | 12 +-
.../request/file/TestOMDirectoryCreateRequest.java | 24 +-
.../file/TestOMDirectoryCreateRequestWithFSO.java | 112 ++-
.../om/request/file/TestOMFileCreateRequest.java | 36 +-
.../file/TestOMFileCreateRequestWithFSO.java | 39 +-
.../om/request/key/TestOMAllocateBlockRequest.java | 19 +-
.../key/TestOMAllocateBlockRequestWithFSO.java | 17 +-
.../ozone/om/request/key/TestOMKeyAclRequest.java | 20 +-
.../om/request/key/TestOMKeyAclRequestWithFSO.java | 35 +-
.../om/request/key/TestOMKeyCommitRequest.java | 75 +-
.../request/key/TestOMKeyCommitRequestWithFSO.java | 27 +-
.../om/request/key/TestOMKeyCreateRequest.java | 48 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 17 +-
.../om/request/key/TestOMKeyDeleteRequest.java | 27 +-
.../request/key/TestOMKeyDeleteRequestWithFSO.java | 112 ++-
.../key/TestOMKeyPurgeRequestAndResponse.java | 10 +-
.../om/request/key/TestOMKeyRenameRequest.java | 15 +-
.../om/request/key/TestOMKeysDeleteRequest.java | 6 +-
.../om/request/key/TestOMKeysRenameRequest.java | 6 +-
.../request/key/TestOMOpenKeysDeleteRequest.java | 13 +-
.../om/request/key/TestOMPrefixAclRequest.java | 6 +-
.../TestS3InitiateMultipartUploadRequest.java | 33 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 10 +-
.../s3/multipart/TestS3MultipartRequest.java | 45 +-
.../TestS3MultipartUploadAbortRequest.java | 11 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 11 +-
.../TestS3MultipartUploadCommitPartRequest.java | 30 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 23 +-
.../TestS3MultipartUploadCompleteRequest.java | 34 +-
...estS3MultipartUploadCompleteRequestWithFSO.java | 34 +-
.../upgrade/TestOMCancelPrepareRequest.java | 2 +-
.../request/volume/TestOMVolumeCreateRequest.java | 6 +-
.../request/volume/TestOMVolumeDeleteRequest.java | 12 +-
.../volume/TestOMVolumeSetOwnerRequest.java | 27 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 29 +-
.../volume/acl/TestOMVolumeAddAclRequest.java | 12 +-
.../volume/acl/TestOMVolumeRemoveAclRequest.java | 14 +-
.../volume/acl/TestOMVolumeSetAclRequest.java | 12 +-
.../ozone/om/response/TestCleanupTableInfo.java | 9 +-
.../file/TestOMDirectoryCreateResponse.java | 6 +-
.../file/TestOMDirectoryCreateResponseWithFSO.java | 8 +-
.../file/TestOMFileCreateResponseWithFSO.java | 17 +-
.../response/key/TestOMAllocateBlockResponse.java | 6 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 19 +-
.../om/response/key/TestOMKeyCommitResponse.java | 6 +-
.../key/TestOMKeyCommitResponseWithFSO.java | 21 +-
.../key/TestOMKeyCreateResponseWithFSO.java | 17 +-
.../om/response/key/TestOMKeyDeleteResponse.java | 4 +-
.../key/TestOMKeyDeleteResponseWithFSO.java | 26 +-
.../om/response/key/TestOMKeyRenameResponse.java | 20 +-
.../ozone/om/response/key/TestOMKeyResponse.java | 4 +-
.../om/response/key/TestOMKeysDeleteResponse.java | 4 +-
.../om/response/key/TestOMKeysRenameResponse.java | 8 +-
.../response/key/TestOMOpenKeysDeleteResponse.java | 8 +-
.../s3/multipart/TestS3MultipartResponse.java | 13 +-
.../TestS3MultipartUploadAbortResponse.java | 20 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 22 +-
...S3MultipartUploadCommitPartResponseWithFSO.java | 10 +-
...stS3MultipartUploadCompleteResponseWithFSO.java | 18 +-
.../ozone/om/upgrade/TestOMUpgradeFinalizer.java | 2 +-
.../om/upgrade/TestOzoneManagerPrepareState.java | 2 +-
.../ozone/security/TestAWSV4AuthValidator.java | 2 +-
.../TestOzoneDelegationTokenSecretManager.java | 6 +-
.../ozone/security/TestOzoneTokenIdentifier.java | 6 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 91 +-
.../hadoop/ozone/security/acl/TestParentAcl.java | 50 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 51 +-
hadoop-ozone/ozonefs-common/pom.xml | 4 +-
.../fs/ozone/BasicOzoneClientAdapterImpl.java | 34 +-
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 9 +-
.../ozone/BasicRootedOzoneClientAdapterImpl.java | 57 +-
.../fs/ozone/BasicRootedOzoneFileSystem.java | 7 +-
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 3 +
.../apache/hadoop/fs/ozone/OzoneClientUtils.java | 89 ++
.../apache/hadoop/fs/ozone/OzoneFSInputStream.java | 2 +-
.../hadoop/fs/ozone/TestOzoneClientUtils.java | 59 ++
hadoop-ozone/ozonefs-hadoop2/pom.xml | 4 +-
hadoop-ozone/ozonefs-hadoop3/pom.xml | 4 +-
hadoop-ozone/ozonefs-shaded/pom.xml | 6 +-
hadoop-ozone/ozonefs/pom.xml | 4 +-
hadoop-ozone/pom.xml | 6 +-
hadoop-ozone/recon-codegen/pom.xml | 2 +-
.../org/hadoop/ozone/recon/codegen/SqlDbUtils.java | 4 +-
.../recon/schema/ContainerSchemaDefinition.java | 3 +-
hadoop-ozone/recon/pom.xml | 2 +-
.../apache/hadoop/ozone/recon/ReconConstants.java | 2 +
.../hadoop/ozone/recon/ReconControllerModule.java | 2 +
.../org/apache/hadoop/ozone/recon/ReconServer.java | 117 +++
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 27 +-
.../org/apache/hadoop/ozone/recon/ReconUtils.java | 29 +
.../ozone/recon/api/ClusterStateEndpoint.java | 15 +-
.../ozone/recon/api/MetricsProxyEndpoint.java | 4 +-
.../hadoop/ozone/recon/api/NSSummaryEndpoint.java | 2 +-
.../hadoop/ozone/recon/api/PipelineEndpoint.java | 2 +-
.../codec/ContainerReplicaHistoryListCodec.java | 45 +-
.../hadoop/ozone/recon/codec/NSSummaryCodec.java | 2 +-
.../ozone/recon/fsck/ContainerHealthStatus.java | 17 +-
.../ozone/recon/fsck/ContainerHealthTask.java | 6 +-
.../persistence/ContainerHealthSchemaManager.java | 12 +-
.../ozone/recon/persistence/ContainerHistory.java | 9 +-
.../ozone/recon/scm/ContainerReplicaHistory.java | 27 +-
.../recon/scm/ContainerReplicaHistoryList.java | 32 +-
.../ozone/recon/scm/ReconContainerManager.java | 25 +-
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 11 +
.../ozone/recon/scm/ReconPipelineFactory.java | 2 +-
.../hadoop/ozone/recon/scm/ReconStorageConfig.java | 61 +-
.../scm/ReconStorageContainerManagerFacade.java | 141 ++-
.../recon/spi/StorageContainerServiceProvider.java | 12 +
.../recon/spi/impl/ContainerKeyPrefixCodec.java | 2 +-
.../spi/impl/OzoneManagerServiceProviderImpl.java | 74 +-
.../impl/ReconContainerMetadataManagerImpl.java | 4 +-
.../ozone/recon/spi/impl/ReconDBDefinition.java | 13 +-
.../impl/StorageContainerServiceProviderImpl.java | 143 ++-
.../ozone/recon/tasks/FileSizeCountTask.java | 4 +-
.../hadoop/ozone/recon/tasks/TableCountTask.java | 2 +-
.../ozone/recon/api/TestContainerEndpoint.java | 26 +-
.../hadoop/ozone/recon/api/TestEndpoints.java | 4 +
.../ozone/recon/api/TestNSSummaryEndpoint.java | 3 -
.../ozone/recon/api/TestOpenContainerCount.java | 8 +-
.../ozone/recon/api/TestTaskStatusService.java | 2 +-
.../ozone/recon/fsck/TestContainerHealthTask.java | 18 +-
.../TestUtilizationSchemaDefinition.java | 2 +-
.../scm/AbstractReconContainerManagerTest.java | 3 +-
.../ozone/recon/scm/TestReconContainerManager.java | 13 +-
.../ozone/recon/scm/TestReconNodeManager.java | 12 +-
.../ozone/recon/scm/TestReconPipelineManager.java | 8 +-
.../impl/TestOzoneManagerServiceProviderImpl.java | 88 +-
.../impl/TestReconNamespaceSummaryManagerImpl.java | 6 +-
.../TestStorageContainerServiceProviderImpl.java | 12 +
.../recon/tasks/TestContainerKeyMapperTask.java | 2 +-
.../ozone/recon/tasks/TestNSSummaryTask.java | 2 -
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 4 +-
.../ozone/recon/tasks/TestTableCountTask.java | 2 +-
hadoop-ozone/s3gateway/pom.xml | 4 +-
.../hadoop/ozone/s3/OzoneClientProducer.java | 98 +-
.../hadoop/ozone/s3/OzoneServiceProvider.java | 94 --
.../hadoop/ozone/s3/VirtualHostStyleFilter.java | 8 +-
.../hadoop/ozone/s3/commontypes/CommonPrefix.java | 2 +
.../hadoop/ozone/s3/commontypes/KeyMetadata.java | 1 +
...CommonPrefix.java => ObjectKeyNameAdapter.java} | 37 +-
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 63 +-
.../ozone/s3/endpoint/CopyObjectResponse.java | 2 +-
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 47 +-
.../ozone/s3/endpoint/ListBucketResponse.java | 6 +-
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 141 +--
.../hadoop/ozone/s3/endpoint/RootEndpoint.java | 5 +
.../org/apache/hadoop/ozone/s3/endpoint/S3Acl.java | 6 +-
.../hadoop/ozone/s3/endpoint/S3BucketAcl.java | 4 +-
.../hadoop/ozone/s3/exception/S3ErrorTable.java | 20 +-
.../hadoop/ozone/s3/io/S3WrapperInputStream.java | 84 --
.../s3/signature/AuthorizationV4QueryParser.java | 5 +-
.../hadoop/ozone/s3/signature/Credential.java | 2 +-
.../ozone/s3/signature/StringToSignProducer.java | 5 +-
.../apache/hadoop/ozone/s3/util/ContinueToken.java | 2 +-
.../apache/hadoop/ozone/s3/util/OzoneS3Util.java | 81 --
.../apache/hadoop/ozone/s3/util/S3StorageType.java | 22 +-
.../org/apache/hadoop/ozone/s3/util/S3Utils.java | 31 +-
.../hadoop/ozone/client/OzoneBucketStub.java | 2 +-
.../hadoop/ozone/s3/TestOzoneClientProducer.java | 74 +-
.../ozone/s3/TestVirtualHostStyleFilter.java | 2 +-
.../TestObjectKeyNameAdapter.java} | 43 +-
.../s3/endpoint/TestAbortMultipartUpload.java | 2 +
.../s3/endpoint/TestInitiateMultipartUpload.java | 2 +
.../hadoop/ozone/s3/endpoint/TestListParts.java | 2 +
.../s3/endpoint/TestMultipartUploadComplete.java | 2 +
.../s3/endpoint/TestMultipartUploadWithCopy.java | 2 +
.../hadoop/ozone/s3/endpoint/TestObjectDelete.java | 2 +
.../hadoop/ozone/s3/endpoint/TestObjectGet.java | 2 +
.../hadoop/ozone/s3/endpoint/TestObjectHead.java | 4 +-
.../hadoop/ozone/s3/endpoint/TestObjectPut.java | 17 +-
.../hadoop/ozone/s3/endpoint/TestPartUpload.java | 2 +
.../ozone/s3/endpoint/TestPermissionCheck.java | 4 +
.../hadoop/ozone/s3/endpoint/TestRootList.java | 2 +-
.../hadoop/ozone/s3/util/TestOzoneS3Util.java | 136 ---
.../tools/dev-support/findbugsExcludeFile.xml | 4 -
hadoop-ozone/tools/pom.xml | 19 +-
.../ozone/admin/nssummary/DiskUsageSubCommand.java | 2 +-
.../admin/nssummary/FileSizeDistSubCommand.java | 2 +-
.../ozone/admin/nssummary/NSSummaryAdmin.java | 36 +-
.../ozone/admin/nssummary/NSSummaryCLIUtils.java | 8 +-
.../admin/nssummary/QuotaUsageSubCommand.java | 2 +-
.../ozone/admin/nssummary/SummarySubCommand.java | 2 +-
.../ozone/admin/om/DecommissionOMSubcommand.java | 212 ++++
.../ozone/admin/om/FinalizeUpgradeSubCommand.java | 6 +-
.../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 7 +-
.../admin/scm/FinalizeScmUpgradeSubcommand.java | 6 +-
.../admin/scm/FinalizeUpgradeCommandUtil.java | 4 +-
.../hadoop/ozone/audit/parser/AuditParser.java | 2 +-
.../ozone/audit/parser/common/DatabaseHelper.java | 16 +-
.../parser/handler/TemplateCommandHandler.java | 2 +-
.../ozone/audit/parser/model/AuditEntry.java | 22 +-
.../apache/hadoop/ozone/debug/ChunkKeyHandler.java | 12 +-
.../hadoop/ozone/debug/DBDefinitionFactory.java | 17 +-
.../org/apache/hadoop/ozone/debug/DBScanner.java | 14 +-
.../apache/hadoop/ozone/debug/PrefixParser.java | 22 +-
.../apache/hadoop/ozone/debug/ReadReplicas.java | 250 +++++
.../hadoop/ozone/freon/BaseFreonGenerator.java | 2 +-
.../hadoop/ozone/freon/ChunkManagerDiskWrite.java | 8 +-
.../hadoop/ozone/freon/DatanodeChunkGenerator.java | 20 +-
.../hadoop/ozone/freon/DatanodeChunkValidator.java | 155 ++-
.../java/org/apache/hadoop/ozone/freon/Freon.java | 3 +-
.../hadoop/ozone/freon/HadoopDirTreeGenerator.java | 4 +-
.../hadoop/ozone/freon/HadoopFsGenerator.java | 2 +-
.../hadoop/ozone/freon/RandomKeyGenerator.java | 54 +-
.../hadoop/ozone/freon/SCMThroughputBenchmark.java | 908 +++++++++++++++++
.../hadoop/ozone/freon/StreamingGenerator.java | 2 +-
.../containergenerator/GeneratorDatanode.java | 6 +-
.../GenerateOzoneRequiredConfigurations.java | 2 +-
.../hadoop/ozone/genesis/BenchMarkCRCBatch.java | 141 ---
.../ozone/genesis/BenchMarkCRCStreaming.java | 173 ----
.../ozone/genesis/BenchMarkContainerStateMap.java | 199 ----
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 339 -------
.../ozone/genesis/BenchMarkOMKeyAllocation.java | 137 ---
.../ozone/genesis/BenchMarkOzoneManager.java | 193 ----
.../apache/hadoop/ozone/genesis/BenchMarkSCM.java | 126 ---
.../ozone/genesis/BenchmarkBlockDataToString.java | 166 ----
.../ozone/genesis/BenchmarkChunkManager.java | 180 ----
.../org/apache/hadoop/ozone/genesis/Genesis.java | 108 ---
.../ozone/genesis/GenesisMemoryProfiler.java | 61 --
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 162 ----
.../apache/hadoop/ozone/genesis/package-info.java | 25 -
.../apache/hadoop/ozone/shell/OzoneAddress.java | 4 +-
.../hadoop/ozone/shell/bucket/BucketCommands.java | 3 +-
.../ozone/shell/bucket/CreateBucketHandler.java | 20 +-
.../ozone/shell/bucket/UpdateBucketHandler.java | 62 ++
.../hadoop/ozone/shell/keys/CopyKeyHandler.java | 18 +-
.../hadoop/ozone/shell/keys/PutKeyHandler.java | 18 +-
.../hadoop/ozone/audit/parser/TestAuditParser.java | 2 +-
.../hadoop/ozone/conf/TestGetConfOptions.java | 4 +-
.../ozone/debug/TestDBDefinitionFactory.java | 7 +-
.../apache/hadoop/ozone/freon/TestProgressBar.java | 6 +-
.../TestGenerateOzoneRequiredConfigurations.java | 4 +-
.../org/apache/hadoop/test/OzoneTestDriver.java | 6 +-
pom.xml | 72 +-
1205 files changed, 26175 insertions(+), 13764 deletions(-)
rename hadoop-hdds/{framework =>
common}/src/main/java/org/apache/hadoop/hdds/NodeDetails.java (100%)
copy
hadoop-hdds/{framework/src/main/java/org/apache/hadoop/hdds/ExitManager.java =>
common/src/main/java/org/apache/hadoop/hdds/client/ReplicatedReplicationConfig.java}
(61%)
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaInfo.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManagerReport.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReplicaInfo.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/TestReplicationManagerReport.java
copy {hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io =>
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container}/package-info.java
(88%)
rename
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/{TestUpgradeUtils.java
=> UpgradeTestUtils.java} (95%)
rename
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/{ChunkLayOutVersion.java
=> ContainerLayoutVersion.java} (77%)
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerInspector.java
copy
hadoop-hdds/{common/src/main/java/org/apache/hadoop/ozone/audit/AuditEventStatus.java
=>
container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/IncrementalReportSender.java}
(71%)
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/ContainerInspectorUtil.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerMetadataInspector.java
rename
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/{TestChunkLayOutVersion.java
=> TestContainerLayoutVersion.java} (73%)
rename
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/{ChunkLayoutTestInfo.java
=> ContainerLayoutTestInfo.java} (84%)
copy
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/{TestKeyValueContainerCheck.java
=> TestKeyValueContainerIntegrityChecks.java} (52%)
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerMetadataInspector.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestContainerScrubberConfiguration.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationConfig.java
copy hadoop-hdds/docs/content/feature/{Observability.md =>
Observability.zh.md} (65%)
create mode 100644 hadoop-hdds/docs/dev-support/bin/make_images_responsive.py
create mode 100644
hadoop-hdds/docs/themes/ozonedoc/layouts/shortcodes/image.html
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/{OMCertificateClient.java
=> CommonCertificateClient.java} (51%)
copy
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/CommonPrefix.java
=>
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/ReconCertificateClient.java
(55%)
rename
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/{FixedThreadPoolExecutor.java
=> FixedThreadPoolWithAffinityExecutor.java} (59%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/AbstractFindTargetGreedy.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindSourceGreedy.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindSourceStrategy.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindTargetGreedy.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindTargetGreedyByNetworkTopology.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindTargetGreedyByUsageInfo.java
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestUtils.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestFindTargetStrategy.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationManagerMetrics.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ReportSubcommand.java
create mode 100644
hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/container/TestInfoSubCommand.java
create mode 100644
hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/container/TestReportSubCommand.java
copy
hadoop-ozone/{s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/CommonPrefix.java
=>
client/src/main/java/org/apache/hadoop/ozone/client/checksum/AbstractBlockChecksumComputer.java}
(57%)
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/BaseFileChecksumHelper.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/ReplicatedBlockChecksumComputer.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/checksum/ReplicatedFileChecksumHelper.java
copy hadoop-ozone/{s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io =>
client/src/main/java/org/apache/hadoop/ozone/client/checksum}/package-info.java
(87%)
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum/TestReplicatedBlockChecksumComputer.java
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/checksum/TestReplicatedFileChecksumHelper.java
rename hadoop-ozone/{s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io =>
client/src/test/java/org/apache/hadoop/ozone/client/checksum}/package-info.java
(86%)
create mode 100644
hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/rpc/RpcClientTest.java
copy
hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageConfig.java
=>
common/src/main/java/org/apache/hadoop/ozone/om/protocol/OMAdminProtocol.java}
(56%)
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/OMConfiguration.java
copy
hadoop-ozone/{s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/CommonPrefix.java
=> common/src/main/java/org/apache/hadoop/ozone/om/protocol/S3Auth.java} (55%)
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OMAdminProtocolClientSideImpl.java
copy
hadoop-ozone/{s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/CommonPrefix.java
=>
common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OMAdminProtocolPB.java}
(52%)
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/ExitManager.java =>
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerClientProtocol.java
(52%)
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser2.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser2.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/HTTP.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/dn.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/om.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/recon.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/s3g.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/scm.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser.keytab
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser2.keytab
copy hadoop-ozone/dist/src/main/{compose/ozone-ha/.env =>
k8s/examples/getting-started/kustomization.yaml} (71%)
copy hadoop-ozone/dist/src/main/{compose/ozone-ha/.env =>
k8s/examples/minikube/kustomization.yaml} (71%)
copy hadoop-ozone/dist/src/main/{compose/ozone-csi/.env =>
k8s/examples/ozone-dev/kustomization.yaml} (56%)
copy hadoop-ozone/dist/src/main/{compose/compatibility/.env =>
k8s/examples/ozone-ha/kustomization.yaml} (78%)
copy hadoop-ozone/dist/src/main/{compose/compatibility/.env =>
k8s/examples/ozone/kustomization.yaml} (78%)
copy hadoop-ozone/dist/src/main/smoketest/{basic/ozone-shell-single.robot =>
compatibility/setup.robot} (73%)
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithLinks.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractDistCpWithFSO.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneOMHAClusterImpl.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/StandardOutputTestBase.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithKeyLatestVersion.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestAddRemoveOzoneManager.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestBucketOwner.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMStartupWithBucketLayout.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerBootstrap.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmHASnapshot.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconScmSnapshot.java
create mode 100644
hadoop-ozone/interface-client/src/main/proto/OMAdminProtocol.proto
delete mode 100644
hadoop-ozone/interface-storage/src/test/java/org/apache/hadoop/ozone/om/helpers/TestInstanceHelper.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneAclUtils.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerUtils.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/OMKeyRequestFactory.java
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/{OMBucketSetPropertyRequest.java
=> OMBucketSetOwnerRequest.java} (50%)
rename
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/{response
=> request}/key/OMOpenKeysDeleteRequest.java (97%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/{key/OMAllocateBlockResponse.java
=> bucket/OMBucketSetOwnerResponse.java} (56%)
copy
hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ContainerReplicaHistoryList.java
=>
ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OmKeyResponse.java}
(53%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OMAdminProtocolServerSideImpl.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/OmTestManagers.java
rename
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/{TestOMRequestUtils.java
=> OMRequestTestUtils.java} (96%)
copy
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/{TestOMBucketDeleteRequest.java
=> TestOMBucketDeleteRequestWithFSO.java} (51%)
create mode 100644
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientUtils.java
create mode 100644
hadoop-ozone/ozonefs-common/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneClientUtils.java
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
copy
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/{CommonPrefix.java
=> ObjectKeyNameAdapter.java} (58%)
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io/S3WrapperInputStream.java
delete mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/OzoneS3Util.java
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/ExitManager.java =>
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Utils.java
(59%)
copy
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/{endpoint/TestObjectDelete.java
=> commontypes/TestObjectKeyNameAdapter.java} (50%)
delete mode 100644
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestOzoneS3Util.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/DecommissionOMSubcommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/ReadReplicas.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/SCMThroughputBenchmark.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkCRCBatch.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkCRCStreaming.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkContainerStateMap.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkDatanodeDispatcher.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOMKeyAllocation.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOzoneManager.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkSCM.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkBlockDataToString.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkChunkManager.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/Genesis.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisMemoryProfiler.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisUtil.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/package-info.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/UpdateBucketHandler.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
