This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a change to branch HDDS-4440-s3-performance
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from 0b4779c HDDS-5073. Use ReplicationConfig on client side (#2136)
add 3e21d75 HDDS-3829. Introduce Layout Feature interface in Ozone.
(#1322)
add 3fe83ce HDDS-4173. Implement HDDS Version management using the
LayoutVersionManager interface. (#1392)
add 9b55f69 HDDS-4141. Implement Finalize command in Ozone Manager
client. (#1400)
add 95b8712 HDDS-4174. Add current HDDS layout version to Datanode
heartbeat/registration (#1421)
add 6fa5fa1 HDDS-4143. Implement a factory for OM Requests that returns
an instance based on layout version. (#1405)
add b96834f HDDS-4252. Add the current layout versions to DN - SCM proto
payload. (#1432)
add 08d8d36 HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that
applies all committed Ratis transactions. (#1430)
add 8fe8a1f HDDS-4253. Add LayoutVersion request/response for DN
registration. (#1457)
add 515c619 HDDS-4319. Compile error with Java 11 (#1483)
add bfce8b9 HDDS-4172. Implement Finalize command in Ozone Manager
server. (#1456)
add 0d014c0 HDDS-4314. OM Layout Version Manager init throws silent CNF
error in integration tests. (#1492)
add b8dcf6b Merge remote-tracking branch 'upstream/master' into
HDDS-3698-upgrade
add d4212a7 Fix copilation issue after mergee
add 5186dec HDDS-4401. Fix compilation issue in HDDS-3698-upgrade branch.
(#1529)
add 8e29229 HDDS-4296. SCM changes to process Layout Info in heartbeat
request/response (#1486)
add 18bef92 HDDS-4142. Expose upgrade related state through JMX. (#1534)
add 28ecd8d Merge remote-tracking branch 'upstream/master' into
HDDS-3698-upgrade
add 7a26250 HDDS-4342. Add DataNode state and transitions for a node
going through upgrade. (#1508)
add 4384789 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-upgrade
add bf7624b Merge remote-tracking branch 'upstream/master' into
HDDS-3698-upgrade
add 6b9ed19 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-upgrade
add ad9d019 HDDS-4536. Add more unit tests for OM layout version manager.
(#1646)
add 9c4f2b4 HDDS-4463. Verify that OM/SCM start fails when Software
Layout Version < Metadata Layout Version. (#1645)
add c3e3b35 HDDS-4179. Implement post-finalize SCM logic. (#1611)
add a7797aa HDDS-4480. Implement OM Prepare Request/Response. (#1613)
add 2489968 HDDS-4540. Add a new OM admin operation to submit the
OMPrepareRequest. (#1664)
add 6519142 HDDS-4178. SCM Finalize client command implementation (#1651)
add bdd00a7 HDDS-4564. Prepare client should check every OM individually
for the prepared check based on Txn ID. (#1692)
add 17a126d HDDS-4569. Add pre append gate and marker file to OM prepare
state. (#1705)
add 72c215c HDDS-4612. Create OMCancelPrepareRequest and Response to
cancel the prepared state of an OM. (#1765)
add d36b87d HDDS-4175. Implement Datanode Finalization. (#1720)
add 81c1dee HDDS-4219. Revisit 'static' nature of OM Layout Version
Manager. (#1773)
add 1de63f4 HDDS-4645. Starting OM with the --upgrade flag should delete
the prepare marker file. (#1798)
add a81b36e HDDS-4614. Add Integration test for HDDS upgrade (happy path
cases). (#1726)
add ee8d276 HDDS-4611. Add an admin command to cancel "preparation" of an
OM quorum. (#1830)
add 1e187f7 Merge remote-tracking branch 'upstream/master' into
upgrade-branch-merge-candidate
add 7ae013e Fix spotbugs issues.
add ba4aeca trigger new CI check
add dd14670 trigger new CI check
add 5720ebf HDDS-4182. Onboard HDDS-3869 into Layout version management.
(#1913)
add 277cc0b HDDS-4610. Fix issues in 'prepare' operation with one OM
down. (#1836)
add 9a078c4 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-nonrolling-upgrade
add 8a9be0d trigger new CI check
add 447024b HDDS-4817. Fresh deploy of Ozone must use the highest layout
version by default. (#1933)
add 485ec25 HDDS-4902 Add Layout version information to Recon datanode
info API. (#1993)
add df57ebc HDDS-4946. Do not wait one heartbeat to move newly registered
datanodes that match SCM's MLV from HEALTHY_READONLY to HEALTHY (#2015)
add 7eefee7 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-nonrolling-upgrade-merge-candidate
add f6a302a HDDS-4908. Layout version should be available in DB for an
un-finalized OM to be finalized through a Ratis snapshot. (#2017)
add f6ecb1d Merge remote-tracking branch 'ssh-upstream/master' into
HDDS-3698-nonrolling-upgrade
add d55a714 HDDS-5023. Datanodes should always use MLV 0 when no VERSION
file is present. (#2080)
add ed00db7 HDDS-4990. Introduce First upgrade startup action and
Pre-finalized state validation in Layout Feature. (#2040)
add 2f96d94 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-nonrolling-upgrade
add 1730b0d Allow layout version to be specified through config on --init
(for MiniOzone cluster testing).
add de3d735 Fix SCM admin command issues after merge.
add 666f33b Ignore failing upgrade integration test.
add 8f3b5ae Fix rat failure.
add dfc4d99 trigger new CI check
add f16092e Intialize DN layout version before security init.
add 805d93d Checkstyle fix.
add 32d0834 Merge remote-tracking branch 'upstream/master' into
HDDS-5046-2
add 9e2e43f Merge remote-tracking branch 'upstream/master' into
HDDS-5046-2
add 9363434 Fix ozone-ha acceptance test.
add 3ea88e1 HDDS-5014. Add non-rolling upgrade design docs.
add a55f657 Merge remote-tracking branch 'upstream/master' into
ssh-upstream-upgrade-branch
add baeb46d Merge remote-tracking branch 'ssh-upstream/master' into
ssh-upstream-upgrade-branch
add c4b7e7b HDDS-4828. SCM should go into "safe mode" until there is at
least 1 pipeline to work with after finalization. (#2101)
add 5c6c858 HDDS-4181. Add acceptance tests for upgrade, finalization and
downgrade. (#2056)
add 6e027a1 HDDS-5014. Upgrade usage primer documentation. (#2133)
add e8e7ced HDDS-5014. Move upgrade user flow to 'feature' folder.
add 7266f32 HDDS-4914. Failure injection and validating HDDS upgrade.
(#1998)
add 58a87fb HDDS-5086. Add pre-finalize validation action for SCM HA.
(#2143)
add d2da5a4 HDDS-4992. SCM should not use pipelines with HEALTHY_READONLY
datanodes (#2142)
add 3221362 Update upgrade-dev-primer.md
add 2823320 HDDS-5108. Attempt to remove state from *UpgradeFinalizer
classes. (#2160)
add 40223ec HDDS-4818. Add command to get status of Finalize in OM & SCM
(#2083)
add 5927bd2 HDDS-5118. Recover from failure during upgrade action. (#2179)
add 9168e07 HDDS-4959. NoSuchMethodException when wrapping RpcException
on downgrade (#2178)
add bef4aed HDDS-5137. Update licenses for non-rolling upgrade new
dependencies. (#2201)
add cdb1471 HDDS-5170. Race condition in NodestateManager#addNode allows
datanodes with lower MLV to be used in pipelines. (#2204)
add 253cc55 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-nonrolling-upgrade-master-merge2
add 1686e56 Fix compilation errors afte merge Update javassist in recon
pom Fix changes introduced in merge that failed TestSCMNodeManager upgrade
tests Fix checkstyle Fix intermittent test failure
TestSCMNodeManager#testSetNodeOpStateAndCommandFired after merge Skip scm init
default layout version in TestOzoneConfigurationFields
add ea239ee HDDS-5226. Do not fail SCM HA pre-finalize validation if SCM
HA was already being used. (#2257)
add 5f046fa HDDS-5138. Upgrade related RPC calls should be allowed only
for admins. (#2217)
add 92b5106 HDDS-5244. Allow multiple OM request versions to be supported
at same layout version. (#2265)
add 2dbf04e HDDS-5191. Increase default pvc storage size (#2219)
add 000e7c5 HDDS-5252. PipelinePlacementPolicy filter out datanodes with
not enough space. (#2271)
add 2671b48 HDDS-5219. Limit number of bad volumes by
dfs.datanode.failed.volumes.tolerated. (#2243)
add d9ae1b3 HDDS-5241. SCM UI should have leader/follower and Primordial
SCM information (#2260)
add 53bbabf HDDS-5275. Datanode Report Publisher publishes one extra
report after DN shutdown (#2283)
add 4301315 HDDS-5158. Add documentation for SCM HA Security. (#2205)
add d567360 HDDS-5273. Handle unsecure cluster convert to secure cluster
for SCM. (#2281)
add 00b675d HDDS-4927. Determine over and under utilized datanodes in
Container Balancer. (#2230)
add 8586815 HDDS-4043. allow deletion from Trash directory without
-skipTrash option (#2110)
add fa4ddbc HDDS-4936. Change ozone groupId from org.apache.hadoop to
org.apache.ozone (#2018)
add 3ffbdd0 HDDS-4993. Add guardrail for reserved buffer size when DN
reads a chunk (#2058)
add e5a774f HDDS-5123. Use the pre-created apache/ozone-testkrb5 image
during secure acceptance tests (#2165)
add 5fd9780 HDDS-5272. Make ozonefs.robot execution repeatable (#2280)
add 12e2918 HDDS-5280. Make XceiverClientManager creation when necessary
in ContainerOperationClient (#2289)
add 1d18f47 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-nonrolling-upgrade-master-merge
add a1b50ad HDDS-5321. Merge master branch at 12e2918 into upgrade branch
(#2316)
add 2c0adac HDDS-5109. Track OM prepare intermittent integration test
failure. (#2288)
add d6f4d07 HDDS-2949: mkdir : store directory entries in a separate
table (#1404)
add 8e17992 HDDS-4266: CreateFile : store parent dir entries into
DirTable and file entry into separate FileTable (#1473)
add 60dd692 HDDS-4332: ListFileStatus - do lookup in directory and file
tables (#1503)
add 172ab51 HDDS-4357: Rename : make rename an atomic ops by updating key
path entry in dir/file table (#1557)
add 995b3bd HDDS-4358: Delete : make delete an atomic operation (#1607)
add d810193 HDDS-4321. Fix compilation errors : merge HDDS-4308 and
HDDS-4473 changes into the branch (#1668)
add b9771e4 HDDS-4596. Directory table, fileTable and openFile Table is
missing from the OM DB Definition. (#1724)
add c10abe6 HDDS-4514. AllocateBlock : lookup and update open file table
for the given path (#1679)
add 0bdbe41 HDDS-2942. Putkey : create key table entries for intermediate
directories in the key path (#1764)
add f132f1e HDDS-4658. LookupKey: do lookup in dir and file tables (#1775)
add 9003eca HDDS-4717. Fix TestOzoneFileSystemV1 and TestObjectStoreV1
cases (#1815)
add f5e4167 HDDS-4720. RenameKey : add unit test to verify
bucket#renameKey (#1847)
add b4d75d6 HDDS-4486. Feature Config: Make proper enableFSPaths and OFS
optimized flag combinations (#1848)
add 5b86a07 HDDS-4771. [FSO]S3MultiPart: Implement
InitiateMultiPartUpload (#1877)
add 797cf96 HDDS-4805. [FSO]Fix findbugs issues after HDDS-2195 (#1906)
add 85f07a2 HDDS-4781. [FSO]S3MultiPart: Implement create and commit
upload part file (#1897)
add b3327bc HDDS-4742. Make trash work with FS Optimised Buckets. (#1915)
add 30d9fa4 HDDS-4813. [FSO]S3Multipart: Implement UploadCompleteRequest
(#1923)
add f28699c HDDS-4513.[FSO]OzoneContract unit test case fixes (#1945)
add 0dfad20 HDDS-4683. [FSO]ListKeys: do lookup in dir and file tables
(#1954)
add 7be14c8 HDDS-4490.[FSO]RenameAndDelete : make ofs#rename and
ofs#delete an atomic operation. (#1965)
add 46d9192 HDDS-4835. [FSO]S3Multipart: Implement UploadAbortRequest
(#1997)
add cdc7855 HDDS-4924. [FSO]S3Multipart: Implement OzoneBucket#listParts
(#2016)
add 7ca2c5f HDDS-4973. [FSO] Missed to cleanup new FileTables in
OMRequests (#2035)
add 2f7606b HDDS-4790. Add a tool to parse entries in the prefix format
(#1891)
add 81361d6 HDDS-4917.[FSO]Implement ACL requests for new layout (#2024)
add 5ed34df HDDS-4932. [FSO] Provide list subpaths function to perform
recursive ACL check during delete and rename op (#2008)
add 459e990 HDDS-4495. [FSO]Delete : Implement async cleanup of garbage
and orphan sub-dirs/files (#2093)
add 9ecfafe HDDS-5018. [FSO] Add robot tests for new Filesystem layout
(#2071)
add bd22405 HDDS-4727. [FSO]OMConfig: Modify 'ozone.om.layout.version'
config (#2118)
add c326199 HDDS-4691. [FSO]Authorizer: OM can do recursive ACL check for
subpaths (#2011)
add f9910c6 HDDS-5072. [FSO] Addendum patch to fix compilation error
(#2130)
add 45c62e8 HDDS-5067. [FSO] Add documentation (#2117)
add aa0abab HDDS-5042. [FSO] Improve KeyDeletingService to cleanup FSO
files (#2128)
add 4449cfda HDDS-5079. [FSO] Rename om metadata layout configuration
value LEGACY (#2135)
add ec38f01 HDDS-5092. [FSO] Update statis links in the design doc
section of the Ozone docs (#2150)
add e63d80a HDDS-5106. [FSO] Avoid using V1 postfixes for prefix related
classes (#2174)
add f1c6757 Merge remote-tracking branch 'origin/master' into HDDS-2939
add a4a9971 HDDS-5093. [FSO] Reducing time of ozonefs acceptance
testmatrix (#2164)
add b6908b0 HDDS-5097. [FSO] Cleanup integration tests and reduce the
execution time (#2185)
add 3c82503 HDDS-5094. [FSO] Fail OM startup when turn on prefix layout
with old buckets (#2151)
add 72d3106 HDDS-5201. [FSO] S3MultiPart: Use existing ozone key format
for MPU Info in DB (#2228)
add 974136d HDDS-4923. [FSO]S3Multipart: Run S3 acceptance test with
prefix layout (#2208)
add 723cdaa Merge remote-tracking branch 'origin/master' into HDDS-2939
add 6b09219 HDDS-5264. SCM should send token for CloseContainer command
(#2277)
add e35dde9 HDDS-5151. Support ByteBuffer read in OzoneInputStream (#2203)
add 96d9ed7 HDDS-5240. Fix out of bound exception when loading
auditparser (#2258)
add 69b92a0 HDDS-5281. Add reinitialize() for SequenceIdGenerator. (#2292)
add 183f4b4 HDDS-5242. Skip `failing` acceptance suite by default (#2263)
add 18f0fe1 HDDS-4733. Remove leveldb from codebase (#2274)
add 656339e HDDS-5204. Missing type-level Javadoc comments (#2262)
add 90a8d7c HDDS-5276. Use built-in cancel support for duplicates (#2284)
add f3f258a HDDS-5277. Recon shows operational status as
"DECOMMISSIONING" for "DECOMMISSIONED" DNs (#2286)
add b6a035a HDDS-5288. Recon UI cannot sort DN by operational state
(#2298)
add 3030c9f HDDS-5282. Relocate classes copied from Hadoop (#2293)
add 095d09b HDDS-4483. Datanodes should send last processed CRL sequence
ID in heartbeats (#2285)
add 25d66d7 HDDS-5154. Add SCM GRPC server to publish CRL update. (#2216)
add c2ae90b HDDS-5295. testCRLStatusReportPublisher fails to create
CRLInfo (#2302)
add 8c1de61 HDDS-5274. Revert "HDDS-5153. Decommissioning a dead node
should complete immediately (#2190)" (#2282)
add bb8eb39 HDDS-5286. Document SCM HA classes (#2297)
add 405925f HDDS-5168. Use ReplicationConfig in OmKeyArgs (#2287).
Contributed by elek.
add 7bc1a25 HDDS-5292. Introduce the WritableContainerInterface to SCM
(#2300)
add 9ad2e98 HDDS-5298. Use netty-bom to ensure consistent Netty version
(#2304)
add 86bcebb Merge remote-tracking branch 'origin/master' into HDDS-2939
add 0323530 HDDS-5311. [FSO] Fix checkstyle warnings after branch rebase
(#2307)
add 39954ad HDDS-5111. DataNode should not always report full information
in heartbeat (#2182)
add 3d3a87f HDDS-5235. Add precheck to avoid ContainerHealthTask setting
Deleted containers to Missing state (#2250)
add 6e278ba HDDS-5243. Return latest key location for clients (#2261)
add 94fb6f3 Merge remote-tracking branch 'origin/master' into HDDS-2939
add 23352f8 HDDS-5320. [FSO] Fix minor checkstyle warnings (#2313)
add fe9ca6e HDDS-5284. [SCM-HA] SCM start failed with
PipelineNotFoundException. (#2295)
add 2bd7f12 HDDS-5313. ContainerInfo should use ReplicationConfig (#2309)
add d1dc91d HDDS-5318. Intermittent failure in
TestOzoneManagerDoubleBufferWithOMResponse (#2314)
add 2e385fc HDDS-5312. Intermittent failure in SCM Ratis integration test
(#2308)
add fb4436f HDDS-5285. Document block deleting service classes (#2296)
add baf8272 HDDS-5323. Avoid unncessary report processing log messages in
follower. (#2317)
add 6752a71 HDDS-5317. BootStrapped SCM fails to bootstrap if it connects
to another bootstrapped SCM first. (#2312)
add 159fb24 HDDS-5304. InterSCM protocol should be server-only (#2305)
add 1290b87 Merge remote-tracking branch 'ssh-upstream/master' into
ssh-upstream-upgrade-branch
add 4c8b62f HDDS-5333. Remove getRequestType method from OM request
classes.
add b335431 HDDS-5336. Fix datanode capacity related race condition.
(#2327)
add f7ac8c1 HDDS-5307. Remove checkAclRight method, duplicates
checkAclRights (#2323)
add 6e6aefe HDDS-5315 Skip storing unwanted block tokens on OM DB (#2311)
add ac7166b HDDS-5263. SCM may stay in safe mode forever after a unclean
shutdown of SCM. (#2294)
add 2262395 HDDS-4913. Refine the native authorizer parent context right
check. (#2012)
add 7e59cb8 HDDS-5334. Remove unncessary log added durig HDDS-5263.
(#2326)
add 838d5d8 HDDS-5324. Shouldn't dectNumKeys when delete keys fail (#2319)
add d45819f HDDS-5329. Remove lockmanager and synchronize on
ContainerInfo in Replication Manager (#2325)
add ad02d6c HDDS-5316. Fix datanode reserved space calculation. (#2310)
add 9cf7d31 HDDS-5268. Ensure disk checker also scans the ratis log disks
periodically (#2290)
add 623e1d9 HDDS-5344. Adding debug log for block token verification.
(#2333)
add 92eb6a2 Merge remote-tracking branch 'ssh-upstream/master' into
ssh-upstream-upgrade-branch
add 4b4e475 Fix checkstyle after merge.
add 7bfd76b HDDS-5348. Pipeline creator may miss one-shot run (#2339)
add 4506883 Merge remote-tracking branch 'ssh-upstream/master' into
ssh-upstream-upgrade-branch
add c0eb347 HDDS-5237. Add SSL support to the Ozone streaming API (#2315)
add 698456b HDDS-5363. Datanode shutdown due to too many bad volumes in
CI (#2352)
add b545e69 HDDS-4692. Handle CRLStatusReport got from DN heartbeats and
persist them (#2334)
add 51df153 HDDS-5356. [SCM-HA] SCM start failed with
PipelineNotFoundException (#2346)
add b9caec3 HDDS-5342. HTML report missing from acceptance results (#2332)
add bb4a5b2 HDDS-5361. Suppress logging of ServerNotLeaderException.
(#2351)
add 76d9a10 HDDS-5347. Wrong cache key for integration tests (#2336)
add eb6c1b6 HDDS-5314. Show number of Open containers per Node in Recon
UI (#2321)
add 5c3f9aa HDDS-5355. In ContainerStateMachine, share the executor
threads between the containers. (#2350)
add 5a9257e HDDS-5357. Move SCMUpdateProtocol to hdds interface-server
package. (#2348)
add fa9616d HDDS-5352. java.lang.ClassNotFoundException:
org/eclipse/jetty/alpn/ALPN (#2347)
add 5e1308a HDDS-5367. Update modification time when updating
quota/storageType/versioning (#2355)
add 2cd55d7 HDDS-5269. Datandoe with low ratis log volume space should
not be considered for new pipeline allocation. (#2344)
add dc02775 HDDS-4926. Support start/stop for container balancer via
command line (#2278)
add b563669 HDDS-5350 : Add allocate block support in MockOmTransport
(#2341). Contributed by Uma Maheswara Rao G.
add 93fb714 HDDS-5353. Avoid unnecessary executeBatch call in
insertAudits (#2342)
add 0766d2c HDDS-5381. SCM terminated with exit status 1: null. (#2362)
add bf42bea HDDS-5369. Cleanup unused configuration related to SCM HA
(#2359)
add e251bab HDDS-5289. Update container's deleteTransactionId on creation
of the transaction in SCM. (#2361)
add 3232fb0 HDDS-5387. ProfileServlet to move the default output location
to an ozone specific directory (#2368)
add 450c375 HDDS-5341. Container report processing is single threaded
(#2338)
add be7264d HDDS-5380. Get more accurate space info for
DedicatedDiskSpaceUsage. (#2365)
add b7ecf8b HDDS-5383. Eliminate expensive string creation in debug log
messages (#2372)
add 474457c HDDS-5389. Include ozoneserviceid in fs.defaultFS when
configuring o3fs (#2370)
add 1e5c708 HDDS-5395. Avoid unnecessary numKeyOps.incr() call in
OMMetrics (#2374)
add 88383d1 HDDS-5396. Fix negligence issue conditional expressions in
MockCRLStore.java (#2380)
add 1d8f972 HDDS-5398. Avoid object creation in ReplicationManger debug
log statements (#2379)
add 039afb3 Merge remote-tracking branch 'ssh-upstream/master' into
ssh-upstream-upgrade-branch
add 4e8aec5 HDDS-5406. add proto version to all the proto files. (#2385)
add 9948632 HDDS-5405. Refactor pom files for HadoopRpc and Grpc/Ratis
compilation properties. (#2386)
add b9a86c3 HDDS-5332. Add a new column family and a service provider in
Recon DB for Namespace Summaries (#2366)
add 01e8811 Merge remote-tracking branch 'upstream/master' into
HDDS-3698-nonrolling-upgrade
add aaeae47 HDDS-5415. Fix TestSCMNodeManager after merge of master at
1d8f972 into upgrade branch (#2388)
add 19ed794 HDDS-3292. Support Hadoop 3.3 (#2356)
add 240af4d Merge branch 'master' into HDDS-3698-nonrolling-upgrade
add 4859acc HDDS-5417. Fix json parse stackoverflow for cmd ozone debug
container list. (#2391)
add eb753e1 HDDS-5416. Intermittent test failure in
TestSCMPipelineManager#testPipelineReload (#2394)
add a2739e2 HDDS-5424. Pipeline list command's output should be in local
timezone. (#2400)
add 77c83c0 HDDS-5400. Remove inflight actions in Replication Manager if
node switches to not IN_SERVICE (#2381)
add dba2d54 HDDS-5421. SCM throws NPE during JMX call. (#2398)
add 5af5a33 Merge remote-tracking branch 'ssh-upstream/master' into
ssh-upstream-upgrade-branch
No new revisions were added by this update.
Summary of changes:
.github/workflows/cancel-ci.yaml | 8 -
.github/workflows/post-commit.yml | 41 +-
hadoop-hdds/client/pom.xml | 30 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 29 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 6 +-
.../hadoop/hdds/scm/storage/ByteArrayReader.java | 67 ++
.../hadoop/hdds/scm/storage/ByteBufferReader.java | 71 ++
.../hdds/scm/storage/ByteReaderStrategy.java | 41 +
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 38 +-
.../hdds/scm/storage/TestBlockInputStream.java | 37 +-
.../hdds/scm/storage/TestChunkInputStream.java | 2 +-
hadoop-hdds/common/pom.xml | 48 +-
.../apache/hadoop/hdds/DFSConfigKeysLegacy.java | 17 -
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 8 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 17 +
.../hadoop/hdds/client/RatisReplicationConfig.java | 7 +-
.../hdds/client/StandaloneReplicationConfig.java | 15 +-
.../hadoop/hdds/fs/AbstractSpaceUsageSource.java | 7 +-
.../hadoop/hdds/fs/DedicatedDiskSpaceUsage.java | 8 +-
.../java/org/apache/hadoop/hdds/scm/ScmConfig.java | 17 +
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 86 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 28 +
.../hadoop/hdds/scm/container/ContainerInfo.java | 55 +-
.../hadoop/hdds/scm/exceptions/SCMException.java | 4 +-
.../scm/ha/RetriableWithFailOverException.java | 31 +
.../scm/ha/RetriableWithNoFailoverException.java | 4 +
.../hadoop/hdds/scm/ha/SCMHAConfiguration.java | 4 +-
.../org/apache/hadoop/hdds/scm/ha/SCMHAUtils.java | 2 +
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 4 +-
.../protocol/StorageContainerLocationProtocol.java | 29 +
.../hadoop/hdds/scm/utils/ClientCommandsUtils.java | 3 +
.../hadoop/hdds/security/x509/crl/CRLInfo.java | 183 ----
.../hadoop/hdds/security/x509/crl/CRLStatus.java | 87 ++
.../hadoop/hdds/upgrade/HDDSLayoutFeature.java | 76 ++
.../hdds/upgrade/HDDSLayoutVersionManager.java | 97 ++
.../hadoop/hdds/upgrade/HDDSUpgradeAction.java | 27 +
.../apache/hadoop/hdds/upgrade/package-info.java | 23 +
.../apache/hadoop/hdds/utils/EntryConsumer.java | 38 -
.../apache/hadoop/hdds/utils/HddsVersionInfo.java | 3 +-
.../org/apache/hadoop/hdds/utils/VersionInfo.java | 8 +-
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 6 +
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 12 +-
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +
.../ozone/common/ChecksumByteBufferFactory.java | 3 +
.../ozone/common/ChecksumByteBufferImpl.java | 5 +-
.../org/apache/hadoop/ozone/common/Storage.java | 41 +-
.../apache/hadoop/ozone/common/StorageInfo.java | 33 +-
.../hadoop/ozone/common/utils/BufferUtils.java | 3 +
.../java/org/apache/hadoop/ozone/ha/ConfUtils.java | 3 +
.../upgrade/AbstractLayoutVersionManager.java | 162 +++
.../ozone/upgrade/BasicUpgradeFinalizer.java | 367 +++++++
.../DefaultUpgradeFinalizationExecutor.java | 66 ++
.../apache/hadoop/ozone/upgrade/LayoutFeature.java | 79 ++
.../upgrade/LayoutVersionInstanceFactory.java | 253 +++++
.../hadoop/ozone/upgrade/LayoutVersionManager.java | 76 ++
.../ozone/upgrade/LayoutVersionManagerMXBean.java | 28 +
.../hadoop/ozone/upgrade/UpgradeActionHdds.java | 51 +
.../hadoop/ozone/upgrade/UpgradeException.java | 114 +++
.../hadoop/ozone/upgrade/UpgradeFinalizer.java | 195 ++++
.../hadoop/ozone/upgrade/VersionFactoryKey.java | 70 ++
.../apache/hadoop/ozone/upgrade/package-info.java | 24 +
.../main/resources/hdds-version-info.properties | 3 +-
.../common/src/main/resources/ozone-default.xml | 189 ++--
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 2 +-
.../hdds/fs/TestCachingSpaceUsageSource.java | 2 +-
.../java/org/apache/hadoop/hdds/fs/TestDU.java | 2 +-
.../org/apache/hadoop/hdds/fs/TestDUFactory.java | 2 +-
.../hdds/fs/TestDedicatedDiskSpaceUsage.java | 2 +-
.../fs/TestDedicatedDiskSpaceUsageFactory.java | 2 +-
.../hadoop/hdds/fs/TestSaveSpaceUsageToFile.java | 4 +-
.../hadoop/hdds/fs/TestSpaceUsageFactory.java | 2 +-
.../apache/hadoop/hdds/scm/ha/TestSCMNodeInfo.java | 3 +
.../hadoop/hdds/tracing/TestStringCodec.java | 2 +-
.../hdds/upgrade/TestHDDSLayoutVersionManager.java | 102 ++
.../hadoop/hdds/upgrade/test/MockComponent.java | 67 ++
.../hadoop/hdds/utils/TestResourceLimitCache.java | 2 +-
.../common/TestChecksumImplsComputeSameValues.java | 4 +
.../apache/hadoop/ozone/lock/TestLockManager.java | 2 +-
.../InjectedUpgradeFinalizationExecutor.java | 128 +++
.../upgrade/TestAbstractLayoutVersionManager.java | 203 ++++
.../ozone/upgrade/TestBasicUpgradeFinalizer.java | 126 +++
.../TestDefaultUpgradeFinalizationExecutor.java | 99 ++
.../upgrade/TestLayoutVersionInstanceFactory.java | 198 ++++
.../ozone/upgrade/TestUpgradeFinalizerActions.java | 210 ++++
.../hadoop/ozone/upgrade/TestUpgradeUtils.java | 54 +
hadoop-hdds/config/pom.xml | 10 +-
.../hadoop/hdds/conf/ConfigFileGenerator.java | 17 +-
.../org/apache/hadoop/hdds/conf/ConfigTag.java | 3 +-
hadoop-hdds/container-service/pom.xml | 42 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 17 +-
.../container/common/DataNodeLayoutVersion.java | 80 --
.../container/common/DatanodeLayoutStorage.java | 98 ++
.../container/common/HDDSVolumeLayoutVersion.java | 80 ++
.../ozone/container/common/impl/ContainerData.java | 2 +
.../common/impl/StorageLocationReport.java | 55 +-
.../common/report/CRLStatusReportPublisher.java | 82 ++
.../container/common/report/ReportPublisher.java | 12 +-
.../common/report/ReportPublisherFactory.java | 2 +
.../common/statemachine/DatanodeConfiguration.java | 119 +++
.../common/statemachine/DatanodeStateMachine.java | 78 +-
.../common/statemachine/StateContext.java | 143 ++-
.../CloseContainerCommandHandler.java | 9 +-
.../FinalizeNewLayoutVersionCommandHandler.java | 121 +++
.../states/endpoint/HeartbeatEndpointTask.java | 74 +-
.../states/endpoint/RegisterEndpointTask.java | 60 +-
.../states/endpoint/VersionEndpointTask.java | 14 +-
.../server/ratis/ContainerStateMachine.java | 78 +-
.../transport/server/ratis/XceiverServerRatis.java | 54 +-
.../container/common/utils/HddsVolumeUtil.java | 17 +-
.../container/common/utils/StorageVolumeUtil.java | 51 +
.../ozone/container/common/volume/HddsVolume.java | 182 +---
.../container/common/volume/HddsVolumeChecker.java | 425 --------
.../container/common/volume/HddsVolumeFactory.java | 93 ++
.../common/volume/ImmutableVolumeSet.java | 8 +-
.../container/common/volume/MetadataVolume.java | 52 +
.../common/volume/MetadataVolumeFactory.java | 56 +
.../container/common/volume/MutableVolumeSet.java | 362 +++----
.../container/common/volume/StorageVolume.java | 214 ++++
.../common/volume/StorageVolumeChecker.java | 424 ++++++++
.../common/volume/StorageVolumeFactory.java | 60 ++
.../ozone/container/common/volume/VolumeInfo.java | 56 +-
.../ozone/container/common/volume/VolumeSet.java | 4 +-
.../ozone/container/common/volume/VolumeUsage.java | 30 +-
.../container/keyvalue/KeyValueContainer.java | 12 +-
.../container/keyvalue/KeyValueContainerCheck.java | 4 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 6 +-
.../container/keyvalue/helpers/BlockUtils.java | 4 +-
.../container/keyvalue/helpers/ChunkUtils.java | 14 +-
.../keyvalue/impl/ChunkManagerDummyImpl.java | 6 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 7 +-
.../keyvalue/impl/FilePerChunkStrategy.java | 2 +
.../background/BlockDeletingService.java | 3 +
.../ozone/container/ozoneimpl/OzoneContainer.java | 54 +-
.../container/replication/ReplicationServer.java | 3 +
.../replication/ReplicationSupervisor.java | 3 +
.../container/stream/DirectoryServerSource.java | 6 +-
.../container/stream/DirstreamClientHandler.java | 5 +
.../container/stream/DirstreamServerHandler.java | 43 +-
.../ozone/container/stream/StreamingClient.java | 28 +-
.../ozone/container/stream/StreamingException.java | 38 +
.../ozone/container/stream/StreamingServer.java | 68 +-
.../upgrade/DataNodeUpgradeFinalizer.java | 95 ++
.../upgrade/DatanodeMetadataFeatures.java | 53 +
.../upgrade/DatanodeSchemaV2FinalizeAction.java | 48 +
.../ScmHAFirstUpgradeLayoutChangeAction.java | 45 +
.../ozone/container/upgrade/UpgradeUtils.java | 44 +
.../ozone/container/upgrade/package-info.java | 21 +
.../protocol/StorageContainerDatanodeProtocol.java | 6 +-
.../protocol/StorageContainerNodeProtocol.java | 12 +-
.../commands/FinalizeNewLayoutVersionCommand.java | 80 ++
.../hadoop/ozone/protocol/commands/SCMCommand.java | 10 +
...inerDatanodeProtocolClientSideTranslatorPB.java | 11 +-
...inerDatanodeProtocolServerSideTranslatorPB.java | 16 +-
.../metadata/TestDatanodeCRLStoreImpl.java | 2 +-
.../hadoop/ozone/TestHddsDatanodeService.java | 2 +-
.../hadoop/ozone/TestHddsSecureDatanodeInit.java | 4 +-
.../ozone/container/common/SCMTestUtils.java | 2 +-
.../hadoop/ozone/container/common/ScmTestMock.java | 11 +-
.../container/common/TestBlockDeletingService.java | 13 +-
.../common/TestDatanodeLayOutVersion.java | 10 +-
.../container/common/TestDatanodeStateMachine.java | 15 +-
.../TestSchemaOneBackwardsCompatibility.java | 17 +-
.../common/helpers/TestDatanodeVersionFile.java | 6 +-
.../common/impl/TestContainerDataYaml.java | 2 +-
.../impl/TestContainerDeletionChoosingPolicy.java | 2 +-
.../common/impl/TestContainerPersistence.java | 8 +-
.../container/common/impl/TestContainerSet.java | 2 +-
.../container/common/impl/TestHddsDispatcher.java | 14 +-
.../common/report/TestReportPublisher.java | 54 +-
.../common/report/TestReportPublisherFactory.java | 13 +-
.../statemachine/TestDatanodeConfiguration.java | 55 +
.../common/statemachine/TestStateContext.java | 138 +--
.../states/endpoint/TestHeartbeatEndpointTask.java | 26 +-
.../container/common/volume/TestHddsVolume.java | 116 +++
.../common/volume/TestHddsVolumeChecker.java | 319 ------
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 2 +-
.../common/volume/TestStorageVolumeChecker.java | 323 ++++++
.../container/common/volume/TestVolumeSet.java | 53 +-
.../common/volume/TestVolumeSetDiskChecks.java | 75 +-
.../keyvalue/TestKeyValueBlockIterator.java | 8 +-
.../container/keyvalue/TestKeyValueContainer.java | 11 +-
.../keyvalue/TestKeyValueContainerCheck.java | 8 +-
.../TestKeyValueContainerMarkUnhealthy.java | 2 +-
.../container/keyvalue/TestKeyValueHandler.java | 8 +-
.../container/keyvalue/TestTarContainerPacker.java | 2 +-
.../container/keyvalue/helpers/TestChunkUtils.java | 4 +-
.../keyvalue/impl/CommonChunkManagerTestCases.java | 35 +-
.../keyvalue/impl/TestBlockManagerImpl.java | 2 +-
.../container/ozoneimpl/TestContainerReader.java | 11 +-
.../container/ozoneimpl/TestOzoneContainer.java | 19 +-
.../replication/TestReplicationSupervisor.java | 2 +-
.../replication/TestSimpleContainerDownloader.java | 4 +-
.../stream/TestDirstreamClientHandler.java | 139 +++
.../container/stream/TestStreamingServer.java | 50 +-
.../ozone/container/stream/package-info.java | 20 +
.../upgrade/TestDataNodeStartupSlvLessThanMlv.java | 96 ++
hadoop-hdds/dev-support/checkstyle/checkstyle.xml | 3 +
.../docs/content/design/namespace-support.md | 13 +-
.../docs/content/design/nonrolling-upgrade.md | 32 +
hadoop-hdds/docs/content/design/omprepare.md | 60 ++
.../docs/content/design/upgrade-dev-primer.md | 111 ++
.../docs/content/feature/PrefixFSO-Delete.png | Bin 0 -> 253792 bytes
.../docs/content/feature/PrefixFSO-Format.png | Bin 0 -> 270550 bytes
.../docs/content/feature/PrefixFSO-Rename.png | Bin 0 -> 227851 bytes
hadoop-hdds/docs/content/feature/PrefixFSO.md | 75 ++
hadoop-hdds/docs/content/feature/SCM-HA.md | 68 +-
.../feature/how-to-do-a-nonrolling-upgrade.md | 54 +
hadoop-hdds/docs/content/feature/scm-secure-ha.png | Bin 0 -> 46757 bytes
hadoop-hdds/docs/content/interface/O3fs.md | 12 +-
hadoop-hdds/docs/content/interface/O3fs.zh.md | 16 +-
hadoop-hdds/docs/content/interface/Ofs.md | 4 +-
hadoop-hdds/docs/content/recipe/SparkOzoneFSK8S.md | 6 +-
.../docs/content/recipe/SparkOzoneFSK8S.zh.md | 6 +-
hadoop-hdds/docs/pom.xml | 6 +-
hadoop-hdds/framework/pom.xml | 34 +-
.../java/org/apache/hadoop/hdds/NodeDetails.java | 5 +-
...inerLocationProtocolClientSideTranslatorPB.java | 127 ++-
.../SCMBlockLocationFailoverProxyProvider.java | 8 +-
.../SCMSecurityProtocolFailoverProxyProvider.java | 8 +-
.../scm/update/client/CRLClientUpdateHandler.java | 201 ++++
.../hadoop/hdds/scm/update/client/CRLStore.java | 34 +
.../hdds/scm/update/client/ClientCRLStore.java | 98 ++
.../scm/update/client/ClientUpdateHandler.java | 29 +
.../client/SCMUpdateClientConfiguration.java | 53 +
.../update/client/SCMUpdateServiceGrpcClient.java | 218 ++++
.../scm/update/client/UpdateServiceConfig.java | 44 +
.../hdds/scm/update/client/package-info.java | 22 +
.../hdds/scm/update/server/CRLClientInfo.java | 56 +
.../scm/update/server/SCMUpdateClientInfo.java | 67 ++
.../hdds/scm/update/server/package-info.java | 22 +
.../hdds/security/token/BlockTokenVerifier.java | 8 +
.../security/token/ContainerTokenGenerator.java | 58 ++
.../token/ContainerTokenSecretManager.java | 30 +-
.../certificate/authority/CertificateStore.java | 6 +
.../authority/PKIProfiles/DefaultProfile.java | 2 +-
.../x509/certificate/client/CertificateClient.java | 26 +-
.../client/DefaultCertificateClient.java | 65 ++
.../certificate/client/OMCertificateClient.java | 11 +-
.../hadoop/hdds/security/x509/crl/CRLCodec.java | 0
.../hadoop/hdds/security/x509/crl/CRLInfo.java | 222 ++++
.../hdds/security/x509/crl/CRLInfoCodec.java | 0
.../hadoop/hdds/server/events/EventExecutor.java | 5 +
.../hadoop/hdds/server/events/EventQueue.java | 44 +-
.../server/events/FixedThreadPoolExecutor.java | 134 +++
.../hdds/server/events/SingleThreadExecutor.java | 9 +
.../hadoop/hdds/server/http/ProfileServlet.java | 2 +-
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 25 +
.../org/apache/hadoop/hdds/utils/LevelDBStore.java | 400 --------
.../hadoop/hdds/utils/LevelDBStoreIterator.java | 64 --
.../apache/hadoop/hdds/utils/MetadataStore.java | 233 -----
.../hadoop/hdds/utils/MetadataStoreBuilder.java | 151 ---
.../org/apache/hadoop/hdds/utils/RocksDBStore.java | 397 --------
.../hadoop/hdds/utils/RocksDBStoreIterator.java | 66 --
.../apache/hadoop/hdds/utils/db/ShortCodec.java | 43 +
.../hadoop/hdds/utils/db/cache/TableCache.java | 3 +
.../token/TestOzoneBlockTokenIdentifier.java | 2 +-
.../x509/certificate/authority/MockCAStore.java | 12 +
.../certificate/authority/TestDefaultCAServer.java | 2 +-
.../client/TestCertificateClientInit.java | 2 +-
.../client/TestDefaultCertificateClient.java | 4 +-
.../security/x509/keys/TestHDDSKeyGenerator.java | 2 +-
.../hdds/security/x509/keys/TestKeyCodec.java | 2 +-
.../hadoop/hdds/server/events/TestEventQueue.java | 62 +-
.../hadoop/hdds/utils/TestMetadataStore.java | 592 -----------
.../hadoop/hdds/utils/TestRocksDBStoreMBean.java | 230 -----
.../hdds/utils/db/TestTypedRDBTableStore.java | 2 +-
.../hadoop/hdds/utils/db/cache/TestTableCache.java | 2 +-
hadoop-hdds/hadoop-dependency-client/pom.xml | 10 +-
hadoop-hdds/hadoop-dependency-server/pom.xml | 10 +-
hadoop-hdds/hadoop-dependency-test/pom.xml | 15 +-
hadoop-hdds/interface-admin/pom.xml | 12 +-
.../src/main/proto/ScmAdminProtocol.proto | 59 +-
hadoop-hdds/interface-client/pom.xml | 24 +-
.../interface-client/src/main/proto/hdds.proto | 24 +-
.../dev-support/findbugsExcludeFile.xml | 3 +
hadoop-hdds/interface-server/pom.xml | 68 +-
.../src/main/proto/InterSCMProtocol.proto | 0
.../src/main/proto/SCMRatisProtocol.proto | 2 +-
.../src/main/proto/SCMUpdateProtocol.proto | 98 ++
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 41 +
.../src/main/proto/ScmServerProtocol.proto | 4 +-
.../src/main/proto/ScmServerSecurityProtocol.proto | 1 +
hadoop-hdds/pom.xml | 78 +-
hadoop-hdds/server-scm/pom.xml | 80 +-
.../hadoop/hdds/scm/SCMCommonPlacementPolicy.java | 31 +-
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 89 +-
.../block/DatanodeDeletedBlockTransactions.java | 6 -
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 2 +-
.../hdds/scm/block/DeletedBlockLogImplV2.java | 3 +-
.../scm/block/DeletedBlockLogStateManager.java | 4 +
.../scm/block/DeletedBlockLogStateManagerImpl.java | 30 +-
.../hdds/scm/block/PendingDeleteHandler.java | 41 -
.../hdds/scm/block/PendingDeleteStatusList.java | 85 --
.../hdds/scm/block/SCMBlockDeletingService.java | 26 +-
.../scm/container/CloseContainerEventHandler.java | 9 +
.../hdds/scm/container/ContainerReportHandler.java | 38 -
.../hdds/scm/container/ContainerStateManager.java | 4 +-
.../scm/container/ContainerStateManagerImpl.java | 2 +-
.../hdds/scm/container/ReplicationManager.java | 264 ++---
.../scm/container/balancer/ContainerBalancer.java | 406 +++++++-
.../balancer/ContainerBalancerConfiguration.java | 80 +-
.../balancer/ContainerBalancerMetrics.java | 130 +++
.../scm/container/states/ContainerStateMap.java | 10 +-
.../hdds/scm/crl/CRLStatusReportHandler.java | 87 ++
.../apache/hadoop/hdds/scm/crl/package-info.java | 22 +
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 33 +-
.../apache/hadoop/hdds/scm/ha/HASecurityUtils.java | 7 +-
.../hdds/scm/ha/MockSCMHADBTransactionBuffer.java | 4 +
.../org/apache/hadoop/hdds/scm/ha/RatisUtil.java | 22 +
.../org/apache/hadoop/hdds/scm/ha/SCMContext.java | 79 +-
.../hdds/scm/ha/SCMDBCheckpointProvider.java | 4 +-
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 1 +
.../hadoop/hdds/scm/ha/SCMHANodeDetails.java | 8 +
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 8 +-
.../apache/hadoop/hdds/scm/ha/SCMStateMachine.java | 69 +-
.../hadoop/hdds/scm/ha/SequenceIdGenerator.java | 29 +-
.../apache/hadoop/hdds/scm/ha/io/BooleanCodec.java | 3 +
.../org/apache/hadoop/hdds/scm/ha/io/Codec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/CodecFactory.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/EnumCodec.java | 3 +
.../hdds/scm/ha/io/GeneratedMessageCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/ListCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/LongCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/StringCodec.java | 3 +
.../apache/hadoop/hdds/scm/node/DatanodeInfo.java | 57 +-
.../hadoop/hdds/scm/node/DatanodeUsageInfo.java | 3 +
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 3 +
.../hdds/scm/node/HealthyReadOnlyNodeHandler.java | 70 ++
.../hdds/scm/node/NodeDecommissionManager.java | 31 +-
.../apache/hadoop/hdds/scm/node/NodeManager.java | 39 +
.../hadoop/hdds/scm/node/NodeStateManager.java | 216 +++-
.../apache/hadoop/hdds/scm/node/NodeStatus.java | 11 +-
.../scm/node/NonHealthyToHealthyNodeHandler.java | 53 -
.../node/ReadOnlyHealthyToHealthyNodeHandler.java | 56 +
.../hadoop/hdds/scm/node/SCMNodeManager.java | 170 +++-
.../hadoop/hdds/scm/node/SCMNodeMetrics.java | 3 +-
.../hadoop/hdds/scm/node/states/NodeStateMap.java | 10 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 15 +
.../scm/pipeline/BackgroundPipelineCreatorV2.java | 26 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 12 +-
.../hdds/scm/pipeline/PipelineManagerV2Impl.java | 25 +
.../hdds/scm/pipeline/PipelinePlacementPolicy.java | 38 +-
.../hdds/scm/pipeline/PipelineReportHandler.java | 17 +-
.../scm/pipeline/PipelineStateManagerV2Impl.java | 10 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 35 +-
.../scm/pipeline/WritableContainerFactory.java | 58 ++
.../scm/pipeline/WritableContainerProvider.java | 57 ++
.../pipeline/WritableRatisContainerProvider.java | 158 +++
...inerLocationProtocolServerSideTranslatorPB.java | 133 +++
.../hdds/scm/safemode/ContainerSafeModeRule.java | 56 +-
.../hdds/scm/safemode/DataNodeSafeModeRule.java | 8 +
.../scm/safemode/HealthyPipelineSafeModeRule.java | 85 +-
.../safemode/OneReplicaPipelineSafeModeRule.java | 70 +-
.../hdds/scm/safemode/SCMSafeModeManager.java | 41 +-
.../hadoop/hdds/scm/safemode/SafeModeExitRule.java | 9 +
.../scm/server/OzoneStorageContainerManager.java | 3 +
.../hdds/scm/server/SCMBlockProtocolServer.java | 2 +
.../hadoop/hdds/scm/server/SCMCertStore.java | 23 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 125 ++-
.../hadoop/hdds/scm/server/SCMConfigurator.java | 20 +
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 46 +-
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 31 +-
.../apache/hadoop/hdds/scm/server/SCMMXBean.java | 10 +
.../hdds/scm/server/SCMSecurityProtocolServer.java | 25 +-
.../hadoop/hdds/scm/server/SCMStorageConfig.java | 9 +-
.../hdds/scm/server/StorageContainerManager.java | 265 +++--
.../scm/server/upgrade/SCMUpgradeFinalizer.java | 161 +++
.../ScmHAUnfinalizedStateValidationAction.java | 72 ++
.../ScmOnFinalizeActionForDatanodeSchemaV2.java | 46 +
.../hdds/scm/server/upgrade/package-info.java | 23 +
.../hadoop/hdds/scm/update/server/SCMCRLStore.java | 51 +
.../scm/update/server/SCMCRLUpdateHandler.java | 151 +++
.../scm/update/server/SCMUpdateClientManager.java | 149 +++
.../hdds/scm/update/server/SCMUpdateHandler.java | 53 +
.../update/server/SCMUpdateServiceGrpcServer.java | 99 ++
.../scm/update/server/SCMUpdateServiceImpl.java | 119 +++
.../hdds/scm/update/server/package-info.java | 22 +
.../main/resources/webapps/scm/scm-overview.html | 8 +
.../scm/TestStorageContainerManagerHttpServer.java | 2 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 136 ++-
.../hadoop/hdds/scm/block/TestBlockManager.java | 4 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 107 +-
.../command/TestCommandStatusReportHandler.java | 2 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 122 ++-
.../hdds/scm/container/SimpleMockNodeManager.java | 40 +-
.../container/TestCloseContainerEventHandler.java | 2 +-
.../scm/container/TestContainerManagerImpl.java | 2 +-
.../scm/container/TestContainerReportHandler.java | 4 +-
.../scm/container/TestContainerStateManager.java | 2 +-
.../TestIncrementalContainerReportHandler.java | 16 +-
.../hdds/scm/container/TestReplicationManager.java | 51 +-
.../scm/container/TestSCMContainerManager.java | 2 +-
.../container/balancer/TestContainerBalancer.java | 236 +++++
.../algorithms/TestContainerPlacementFactory.java | 15 +-
.../TestSCMContainerPlacementCapacity.java | 20 +-
.../TestSCMContainerPlacementRackAware.java | 29 +-
.../TestSCMContainerPlacementRandom.java | 23 +-
.../hdds/scm/crl/TestCRLStatusReportHandler.java | 137 +++
.../apache/hadoop/hdds/scm/ha/TestSCMContext.java | 9 +-
.../hadoop/hdds/scm/ha/TestSCMHAConfiguration.java | 3 +
.../hadoop/hdds/scm/ha/TestSCMServiceManager.java | 7 +-
.../hdds/scm/ha/TestSequenceIDGenerator.java | 3 +
.../hdds/scm/node/TestContainerPlacement.java | 23 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 57 +-
.../hdds/scm/node/TestNodeDecommissionManager.java | 37 +-
.../hdds/scm/node/TestNodeReportHandler.java | 39 +-
.../hadoop/hdds/scm/node/TestNodeStateManager.java | 57 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 550 +++++++++-
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 13 +-
.../hadoop/hdds/scm/node/TestStatisticsUpdate.java | 33 +-
.../hdds/scm/node/states/TestNodeStateMap.java | 14 +-
.../hdds/scm/pipeline/MockPipelineManager.java | 10 +
.../hdds/scm/pipeline/TestPipelineManagerImpl.java | 7 +-
.../scm/pipeline/TestPipelinePlacementPolicy.java | 56 +
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 167 ++-
.../safemode/TestHealthyPipelineSafeModeRule.java | 8 +-
.../TestOneReplicaPipelineSafeModeRule.java | 4 +-
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 37 +-
.../scm/server/TestSCMBlockProtocolServer.java | 2 +-
.../scm/server/TestSCMSecurityProtocolServer.java | 4 +-
.../hdds/scm/update/server/MockCRLStore.java | 138 +++
.../server/TestSCMUpdateServiceGrpcServer.java | 301 ++++++
.../TestSCMHAUnfinalizedStateValidationAction.java | 128 +++
.../scm/upgrade/TestScmStartupSlvLessThanMlv.java | 75 ++
.../hadoop/hdds/scm/upgrade/package-info.java | 21 +
.../ozone/container/common/TestEndPoint.java | 54 +-
.../testutils/ReplicationNodeManagerMock.java | 23 +-
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 63 +-
hadoop-hdds/test-utils/pom.xml | 6 +-
.../org/apache/hadoop/test/GenericTestUtils.java | 365 -------
.../org/apache/hadoop/test/LambdaTestUtils.java | 814 ---------------
.../apache/hadoop/test/TimedOutTestsListener.java | 183 ----
.../org/apache/ozone/test/GenericTestUtils.java | 365 +++++++
.../org/apache/ozone/test/LambdaTestUtils.java | 814 +++++++++++++++
.../apache/ozone/test/TimedOutTestsListener.java | 183 ++++
.../java/org/apache/ozone/test/package-info.java | 22 +
hadoop-hdds/tools/pom.xml | 38 +-
.../hdds/scm/cli/ContainerBalancerCommands.java | 108 ++
.../scm/cli/ContainerBalancerStartSubcommand.java | 66 ++
.../scm/cli/ContainerBalancerStatusSubcommand.java | 45 +
.../scm/cli/ContainerBalancerStopSubcommand.java | 40 +
.../hdds/scm/cli/ContainerOperationClient.java | 78 +-
.../org/apache/hadoop/hdds/scm/cli/ScmOption.java | 14 +-
.../datanode/TestContainerBalancerSubCommand.java | 141 +++
.../scm/cli/datanode/TestListInfoSubcommand.java | 17 +-
hadoop-ozone/client/pom.xml | 14 +-
.../apache/hadoop/ozone/client/OzoneBucket.java | 313 +++++-
.../org/apache/hadoop/ozone/client/OzoneKey.java | 41 +-
.../hadoop/ozone/client/OzoneKeyDetails.java | 20 +
.../hadoop/ozone/client/OzoneMultipartUpload.java | 34 +-
.../client/OzoneMultipartUploadPartListParts.java | 34 +-
.../client/io/BlockOutputStreamEntryPool.java | 7 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 51 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 38 +-
.../client/io/MultipartCryptoKeyInputStream.java | 3 +
.../hadoop/ozone/client/io/OzoneInputStream.java | 15 +-
.../ozone/client/protocol/ClientProtocol.java | 5 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 47 +-
.../hadoop/ozone/client/MockOmTransport.java | 12 +
.../hadoop/ozone/client/TestOzoneClient.java | 36 +-
hadoop-ozone/common/pom.xml | 26 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 32 +
.../hadoop/ozone/client/io/LengthInputStream.java | 17 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 25 +
.../apache/hadoop/ozone/om/OmUpgradeConfig.java | 82 ++
.../hadoop/ozone/om/exceptions/OMException.java | 10 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 12 +
.../hadoop/ozone/om/helpers/OmDirectoryInfo.java | 264 +++++
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 76 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 184 +++-
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 6 +-
.../ozone/om/helpers/OmMultipartKeyInfo.java | 110 +-
.../hadoop/ozone/om/helpers/OmMultipartUpload.java | 32 +-
.../om/helpers/OmMultipartUploadListParts.java | 25 +-
.../hadoop/ozone/om/helpers/OzoneAclUtil.java | 25 -
.../hadoop/ozone/om/helpers/OzoneFSUtils.java | 122 +++
.../ozone/om/helpers/WithParentObjectId.java | 55 +
.../ozone/om/protocol/OzoneManagerProtocol.java | 105 ++
...OzoneManagerProtocolClientSideTranslatorPB.java | 188 +++-
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 2 +
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 20 +-
.../hadoop/ozone/security/acl/OzonePrefixPath.java | 67 ++
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 3 +-
.../main/resources/ozone-version-info.properties | 3 +-
.../java/org/apache/hadoop/ozone/TestOmUtils.java | 31 +
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 2 +-
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 10 +-
.../ozone/om/helpers/TestOmMultipartKeyInfo.java | 5 +-
hadoop-ozone/csi/pom.xml | 22 +-
hadoop-ozone/datanode/pom.xml | 14 +-
hadoop-ozone/dev-support/checks/coverage.sh | 3 +-
hadoop-ozone/dev-support/checks/integration.sh | 2 +-
hadoop-ozone/dev-support/checks/unit.sh | 2 +-
hadoop-ozone/dev-support/docker/Dockerfile | 2 +-
.../intellij/runConfigurations/CsiServer.xml | 4 +-
.../intellij/runConfigurations/Datanode1.xml | 2 +-
.../intellij/runConfigurations/Datanode2.xml | 2 +-
.../intellij/runConfigurations/Datanode3.xml | 2 +-
.../intellij/runConfigurations/FreonStandalone.xml | 4 +-
.../intellij/runConfigurations/OzoneFsShell.xml | 2 +-
.../intellij/runConfigurations/OzoneManager.xml | 2 +-
.../runConfigurations/OzoneManagerInit.xml | 2 +-
.../intellij/runConfigurations/OzoneShell.xml | 2 +-
.../intellij/runConfigurations/Recon.xml | 4 +-
.../intellij/runConfigurations/S3Gateway.xml | 4 +-
.../runConfigurations/StorageContainerManager.xml | 2 +-
.../StorageContainerManagerInit.xml | 2 +-
.../dist/dev-support/bin/dist-layout-stitching | 3 +
.../dist/dev-support/bin/dist-tar-stitching | 6 +-
hadoop-ozone/dist/pom.xml | 75 +-
.../docker-image/docker-krb5/Dockerfile-krb5 | 35 -
.../common/docker-image/docker-krb5/README.md | 34 -
.../common/docker-image/docker-krb5/kadm5.acl | 20 -
.../common/docker-image/docker-krb5/krb5.conf | 41 -
.../common/docker-image/docker-krb5/launcher.sh | 25 -
.../src/main/compose/compatibility/docker-config | 1 +
.../dist/src/main/compose/ozone-csi/docker-config | 1 +
.../dist/src/main/compose/ozone-ha/docker-config | 2 +
.../dist/src/main/compose/ozone-mr/common-config | 1 +
.../main/compose/ozone-mr/hadoop27/docker-config | 4 +-
.../main/compose/ozone-mr/hadoop31/docker-config | 4 +-
.../main/compose/ozone-mr/hadoop32/docker-config | 4 +-
.../dist/src/main/compose/ozone-mr/test.sh | 6 +-
.../src/main/compose/ozone-om-ha/docker-config | 1 +
.../dist/src/main/compose/ozone-om-prepare/.env | 25 +
.../compose/ozone-om-prepare/docker-compose.yaml | 125 +++
.../main/compose/ozone-om-prepare/docker-config | 35 +
.../dist/src/main/compose/ozone-om-prepare/test.sh | 74 ++
.../src/main/compose/ozone-topology/docker-config | 1 +
.../src/main/compose/ozone/docker-compose.yaml | 10 +
.../dist/src/main/compose/ozone/docker-config | 1 +
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 28 +-
.../src/main/compose/ozones3-haproxy/docker-config | 1 +
.../dist/src/main/compose/ozonesecure-ha/.env | 1 +
.../compose/ozonesecure-ha/docker-compose.yaml | 44 +-
.../src/main/compose/ozonesecure-ha/docker-config | 25 +-
.../compose/ozonesecure-ha/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../main/compose/ozonesecure-ha/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure-ha/keytabs/om.keytab | Bin 0 -> 278 bytes
.../compose/ozonesecure-ha/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure-ha/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure-ha/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure-ha/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../ozonesecure-ha/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure-ha/krb5.conf | 41 +
.../dist/src/main/compose/ozonesecure-mr/.env | 1 +
.../dist/src/main/compose/ozonesecure-mr/README.md | 2 +-
.../compose/ozonesecure-mr/docker-compose.yaml | 30 +-
.../src/main/compose/ozonesecure-mr/docker-config | 27 +-
.../compose/ozonesecure-mr/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../main/compose/ozonesecure-mr/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure-mr/keytabs/om.keytab | Bin 0 -> 278 bytes
.../compose/ozonesecure-mr/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure-mr/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure-mr/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure-mr/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../ozonesecure-mr/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure-mr/krb5.conf | 39 +
.../dist/src/main/compose/ozonesecure/.env | 1 +
.../main/compose/ozonesecure/docker-compose.yaml | 23 +-
.../src/main/compose/ozonesecure/docker-config | 24 +-
.../main/compose/ozonesecure/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../src/main/compose/ozonesecure/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../src/main/compose/ozonesecure/keytabs/om.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../compose/ozonesecure/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure/krb5.conf | 41 +
.../dist/src/main/compose/restart/docker-config | 1 +
hadoop-ozone/dist/src/main/compose/test-all.sh | 8 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 42 +-
hadoop-ozone/dist/src/main/compose/upgrade/.env | 21 -
.../dist/src/main/compose/upgrade/0.5.0/test.sh | 24 -
.../dist/src/main/compose/upgrade/1.0.0/test.sh | 24 -
.../dist/src/main/compose/upgrade/README.md | 95 +-
.../dist/src/main/compose/upgrade/compose/ha/.env | 26 +
.../compose/upgrade/compose/ha/docker-compose.yaml | 160 +++
.../main/compose/upgrade/compose/ha/docker-config | 43 +
.../src/main/compose/upgrade/compose/ha/load.sh | 28 +
.../src/main/compose/upgrade/compose/non-ha/.env | 21 +
.../upgrade/compose/non-ha/docker-compose.yaml | 137 +++
.../compose/upgrade/compose/non-ha/docker-config | 42 +
.../main/compose/upgrade/compose/non-ha/load.sh | 27 +
.../src/main/compose/upgrade/docker-compose.yaml | 134 ---
.../dist/src/main/compose/upgrade/docker-config | 38 -
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 34 +-
.../dist/src/main/compose/upgrade/testlib.sh | 135 ++-
.../src/main/compose/upgrade/upgrade_to_current.sh | 51 -
.../src/main/compose/upgrade/upgrade_to_release.sh | 51 -
.../manual-upgrade/0.5.0-1.1.0/callback.sh | 40 +
.../upgrade/upgrades/manual-upgrade/driver.sh | 56 +
.../non-rolling-upgrade/1.0.0-1.1.0/callback.sh | 79 ++
.../upgrade/upgrades/non-rolling-upgrade/driver.sh | 101 ++
.../dist/src/main/compose/versions/0.5.0.sh | 4 -
.../dist/src/main/compose/versions/1.0.0.sh | 4 -
.../dist/src/main/compose/versions/1.1.0.sh | 4 -
.../dist/src/main/compose/versions/README.md | 3 -
.../dist/src/main/compose/xcompat/docker-config | 1 +
.../definitions/ozone/definitions/persistence.yaml | 4 +-
.../examples/ozone-ha/datanode-statefulset.yaml | 2 +-
.../k8s/examples/ozone/datanode-statefulset.yaml | 2 +-
hadoop-ozone/dist/src/main/keytabs/HTTP.keytab | Bin 0 -> 432 bytes
hadoop-ozone/dist/src/main/keytabs/dn.keytab | Bin 0 -> 278 bytes
hadoop-ozone/dist/src/main/keytabs/hadoop.keytab | Bin 0 -> 146 bytes
hadoop-ozone/dist/src/main/keytabs/jhs.keytab | Bin 0 -> 142 bytes
hadoop-ozone/dist/src/main/keytabs/nm.keytab | Bin 0 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/om.keytab | Bin 0 -> 278 bytes
hadoop-ozone/dist/src/main/keytabs/recon.keytab | Bin 0 -> 296 bytes
hadoop-ozone/dist/src/main/keytabs/rm.keytab | Bin 0 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/s3g.keytab | Bin 0 -> 434 bytes
hadoop-ozone/dist/src/main/keytabs/scm.keytab | Bin 0 -> 586 bytes
hadoop-ozone/dist/src/main/keytabs/testuser.keytab | Bin 0 -> 450 bytes
.../dist/src/main/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/keytabs/update-keytabs.sh | 77 ++
hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 11 +-
.../license/bin/licenses/LICENSE-org.aspectj.html | 91 ++
.../LICENSE-org.reflections-reflections.txt | 14 +
hadoop-ozone/dist/src/main/license/jar-report.txt | 92 +-
.../dist/src/main/license/update-jar-report.sh | 2 +-
.../src/main/smoketest/admincli/container.robot | 11 +-
.../dist/src/main/smoketest/basic/links.robot | 10 +-
.../src/main/smoketest/basic/ozone-shell-lib.robot | 6 +-
.../dist/src/main/smoketest/cli/classpath.robot | 14 +-
.../dist/src/main/smoketest/commonlib.robot | 8 +-
.../main/smoketest/omha/om-cancel-prepare.robot | 43 +
.../dist/src/main/smoketest/omha/om-prepare.robot | 49 +
.../dist/src/main/smoketest/omha/om-prepared.robot | 36 +
.../src/main/smoketest/ozonefs/hadoopo3fs.robot | 2 +
.../dist/src/main/smoketest/ozonefs/setup.robot | 10 +-
.../main/smoketest/security/ozone-secure-fs.robot | 18 +-
.../dist/src/main/smoketest/spnego/web.robot | 2 +-
.../src/main/smoketest/upgrade/check-mlv.robot | 41 +
.../dist/src/main/smoketest/upgrade/finalize.robot | 31 +
.../dist/src/main/smoketest/upgrade/generate.robot | 33 +
.../dist/src/main/smoketest/upgrade/prepare.robot | 26 +
.../dist/src/main/smoketest/upgrade/validate.robot | 30 +
hadoop-ozone/dist/src/shell/ozone/ozone | 42 +-
hadoop-ozone/dist/src/shell/upgrade/1.0.0.sh | 23 +
.../upgrade/{v1 => 1.0.0}/01-migrate-scm-db.sh | 0
hadoop-ozone/dist/src/shell/upgrade/v1.sh | 23 -
.../dist/src/test/shell/compose_testlib.bats | 13 +
hadoop-ozone/dist/src/test/shell/failing1/test.sh | 17 +
.../fault-injection-test/mini-chaos-tests/pom.xml | 24 +-
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 5 +-
.../fault-injection-test/network-tests/pom.xml | 6 +-
hadoop-ozone/fault-injection-test/pom.xml | 6 +-
hadoop-ozone/insight/pom.xml | 42 +-
.../hadoop/ozone/insight/BaseInsightPoint.java | 4 +-
hadoop-ozone/integration-test/pom.xml | 73 +-
.../ozone/TestDirectoryDeletingServiceWithFSO.java | 402 ++++++++
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 18 +-
.../fs/ozone/TestOzoneFileInterfacesWithFSO.java | 96 ++
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 511 +++++++++-
.../fs/ozone/TestOzoneFileSystemMissingParent.java | 6 +-
.../fs/ozone/TestOzoneFileSystemPrefixParser.java | 168 +++
.../fs/ozone/TestOzoneFileSystemWithFSO.java | 479 +++++++++
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 2 +-
.../hadoop/fs/ozone/TestRootedOzoneFileSystem.java | 234 ++++-
.../fs/ozone/TestRootedOzoneFileSystemWithFSO.java | 160 +++
.../ozone/contract/ITestOzoneContractCreate.java | 26 +-
.../ozone/contract/ITestOzoneContractDelete.java | 26 +-
.../contract/ITestOzoneContractGetFileStatus.java | 27 +-
.../fs/ozone/contract/ITestOzoneContractMkdir.java | 26 +-
.../fs/ozone/contract/ITestOzoneContractOpen.java | 27 +-
.../ozone/contract/ITestOzoneContractRename.java | 26 +-
.../ozone/contract/ITestOzoneContractRootDir.java | 26 +-
.../fs/ozone/contract/ITestOzoneContractSeek.java | 1 +
.../fs/ozone/contract/ITestOzoneContractUtils.java | 61 ++
.../hadoop/fs/ozone/contract/OzoneContract.java | 13 +
.../hadoop/hdds/scm/TestRatisPipelineLeader.java | 2 +-
.../hadoop/hdds/scm/TestSCMInstallSnapshot.java | 3 +-
.../apache/hadoop/hdds/scm/TestSCMSnapshot.java | 4 +
.../TestContainerStateManagerIntegration.java | 7 +-
.../metrics/TestSCMContainerManagerMetrics.java | 2 +-
.../hdds/scm/pipeline/TestLeaderChoosePolicy.java | 4 +-
.../hadoop/hdds/scm/pipeline/TestNodeFailure.java | 2 +-
.../hdds/scm/pipeline/TestPipelineClose.java | 2 +-
.../TestRatisPipelineCreateAndDestroy.java | 2 +-
.../safemode/TestSCMSafeModeWithPipelineRules.java | 2 +-
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 1067 ++++++++++++++++++++
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 22 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 61 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 47 +-
.../hadoop/ozone/MiniOzoneOMHAClusterImpl.java | 2 +-
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 2 +-
.../ozone/TestContainerBalancerOperations.java | 112 ++
.../apache/hadoop/ozone/TestDelegationToken.java | 6 +-
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 8 +-
.../hadoop/ozone/TestMiniOzoneOMHACluster.java | 2 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 2 +
.../hadoop/ozone/TestSecureOzoneCluster.java | 6 +-
.../hadoop/ozone/TestStorageContainerManager.java | 38 +-
.../ozone/client/CertificateClientTestImpl.java | 14 +
.../ozone/client/rpc/Test2WayCommitInRatis.java | 2 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 9 +-
.../rpc/TestCloseContainerHandlingByClient.java | 39 +-
.../rpc/TestContainerReplicationEndToEnd.java | 2 +-
.../client/rpc/TestContainerStateMachine.java | 2 +-
.../TestContainerStateMachineFailureOnRead.java | 2 +-
.../rpc/TestContainerStateMachineFailures.java | 2 +-
.../rpc/TestContainerStateMachineFlushDelay.java | 2 +-
.../client/rpc/TestDeleteWithSlowFollower.java | 10 +-
.../client/rpc/TestFailureHandlingByClient.java | 28 +-
.../rpc/TestFailureHandlingByClientFlushDelay.java | 7 +-
.../rpc/TestMultiBlockWritesWithDnFailures.java | 13 +-
.../client/rpc/TestOzoneAtRestEncryption.java | 6 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 1024 +++++++++++++++++++
.../client/rpc/TestOzoneRpcClientAbstract.java | 8 +-
.../rpc/TestOzoneRpcClientForAclAuditLog.java | 2 +-
.../hadoop/ozone/client/rpc/TestReadRetries.java | 60 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 8 +-
.../ozone/client/rpc/TestWatchForCommit.java | 2 +-
.../ozone/client/rpc/read/TestInputStreamBase.java | 5 +-
.../ozone/client/rpc/read/TestKeyInputStream.java | 71 +-
.../ozone/container/TestContainerReplication.java | 6 +-
.../apache/hadoop/ozone/container/TestHelper.java | 6 +-
.../commandhandler/TestBlockDeletion.java | 59 +-
.../TestCloseContainerByPipeline.java | 23 +-
.../commandhandler/TestCloseContainerHandler.java | 13 +-
.../commandhandler/TestDeleteContainerHandler.java | 12 +-
.../transport/server/ratis/TestCSMMetrics.java | 2 +-
.../container/metrics/TestContainerMetrics.java | 13 +-
.../ozoneimpl/TestOzoneContainerWithTLS.java | 2 +-
.../ozoneimpl/TestSecureOzoneContainer.java | 2 +-
.../hadoop/ozone/container/package-info.java | 22 +
.../container/server/TestContainerServer.java | 2 +-
.../server/TestSecureContainerServer.java | 8 +-
.../apache/hadoop/ozone/dn/DatanodeTestUtils.java | 40 +-
.../ozone/dn/TestDatanodeLayoutUpgradeTool.java | 4 +-
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +-
.../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 4 +-
.../TestDatanodeHddsVolumeFailureDetection.java | 28 +-
.../TestDatanodeHddsVolumeFailureToleration.java | 135 +++
.../ozone/freon/TestHadoopDirTreeGenerator.java | 24 +-
.../freon/TestHadoopDirTreeGeneratorWithFSO.java | 36 +
.../ozone/freon/TestHadoopNestedDirGenerator.java | 4 +-
.../hadoop/ozone/fsck/TestContainerMapper.java | 7 +-
.../ozone/om/TestContainerReportWithKeys.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 136 ++-
.../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +-
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 2 +-
.../hadoop/ozone/om/TestOMStartupWithLayout.java | 197 ++++
.../hadoop/ozone/om/TestOMUpgradeFinalization.java | 254 +++++
.../hadoop/ozone/om/TestObjectStoreWithFSO.java | 722 +++++++++++++
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 3 +-
.../hadoop/ozone/om/TestOmBlockVersioning.java | 4 +
.../ozone/om/TestOmStartupSlvLessThanMlv.java | 80 ++
.../ozone/om/TestOzoneManagerConfiguration.java | 2 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 2 +-
.../ozone/om/TestOzoneManagerHAKeyDeletion.java | 5 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerPrepare.java | 510 ++++++++++
.../hadoop/ozone/om/TestOzoneManagerRestart.java | 2 +-
.../ozone/om/TestOzoneManagerRocksDBLogging.java | 2 +-
.../hadoop/ozone/om/TestRecursiveAclWithFSO.java | 338 +++++++
.../apache/hadoop/ozone/om/TestScmSafeMode.java | 4 +-
.../hadoop/ozone/om/TestSecureOzoneManager.java | 4 +-
.../ozone/om/parser/TestOMRatisLogParser.java | 2 +-
.../hadoop/ozone/recon/TestReconAsPassiveScm.java | 5 +-
.../apache/hadoop/ozone/recon/TestReconTasks.java | 2 +-
.../ozone/recon/TestReconWithOzoneManager.java | 5 +-
.../ozone/recon/TestReconWithOzoneManagerHA.java | 10 +-
.../hadoop/ozone/scm/TestCloseContainer.java | 2 +-
.../hadoop/ozone/scm/TestFailoverWithSCMHA.java | 2 +-
.../ozone/scm/TestSCMInstallSnapshotWithHA.java | 3 +-
.../ozone/scm/TestStorageContainerManagerHA.java | 10 +-
.../hadoop/ozone/scm/TestXceiverClientManager.java | 2 +-
.../hadoop/ozone/scm/TestXceiverClientMetrics.java | 2 +-
.../scm/node/TestDecommissionAndMaintenance.java | 2 +-
.../hadoop/ozone/scm/node/TestQueryNode.java | 2 +-
.../TestSCMPipelineBytesWrittenMetrics.java | 2 +-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 104 +-
hadoop-ozone/interface-client/pom.xml | 12 +-
.../src/main/proto/OmClientProtocol.proto | 114 +++
.../interface-client/src/main/proto/Security.proto | 1 +
hadoop-ozone/interface-storage/pom.xml | 32 +-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 52 +
.../ozone/om/codec/OmDirectoryInfoCodec.java | 60 ++
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 8 +
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 5 +-
.../om/codec/TestOmMultipartKeyInfoCodec.java | 7 +-
.../ozone/om/codec/TestOmPrefixInfoCodec.java | 2 +-
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 5 +-
.../ozone/om/codec/TestS3SecretValueCodec.java | 2 +-
.../ozone/om/codec/TestTransactionInfoCodec.java | 2 +-
hadoop-ozone/native-client/README.md | 4 +-
.../dev-support/findbugsExcludeFile.xml | 4 +
hadoop-ozone/ozone-manager/pom.xml | 103 +-
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 7 +-
.../hadoop/ozone/om/DirectoryDeletingService.java | 286 ++++++
.../org/apache/hadoop/ozone/om/KeyManager.java | 56 +
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 900 +++++++++++++++--
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 18 +-
.../apache/hadoop/ozone/om/OMStarterInterface.java | 2 +
.../java/org/apache/hadoop/ozone/om/OMStorage.java | 7 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 141 ++-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 362 ++++++-
.../hadoop/ozone/om/OzoneManagerPrepareState.java | 311 ++++++
.../hadoop/ozone/om/OzoneManagerStarter.java | 32 +-
.../hadoop/ozone/om/OzonePrefixPathImpl.java | 164 +++
.../java/org/apache/hadoop/ozone/om/ScmClient.java | 11 +
.../hadoop/ozone/om/TrashOzoneFileSystem.java | 146 ++-
.../apache/hadoop/ozone/om/TrashPolicyOzone.java | 40 +-
.../hadoop/ozone/om/codec/OMDBDefinition.java | 45 +-
.../ozone/om/ratis/OzoneManagerDoubleBuffer.java | 15 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 28 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 57 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 94 ++
.../hadoop/ozone/om/request/OMClientRequest.java | 55 +-
.../om/request/bucket/OMBucketCreateRequest.java | 34 +
.../om/request/bucket/OMBucketDeleteRequest.java | 1 +
.../request/bucket/OMBucketSetPropertyRequest.java | 20 +-
.../request/bucket/acl/OMBucketAddAclRequest.java | 5 +-
.../bucket/acl/OMBucketRemoveAclRequest.java | 1 +
.../om/request/file/OMDirectoryCreateRequest.java | 10 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 314 ++++++
.../ozone/om/request/file/OMFileCreateRequest.java | 62 +-
.../request/file/OMFileCreateRequestWithFSO.java | 261 +++++
.../ozone/om/request/file/OMFileRequest.java | 745 ++++++++++++++
.../om/request/key/OMAllocateBlockRequest.java | 2 +-
.../request/key/OMAllocateBlockRequestWithFSO.java | 226 +++++
.../ozone/om/request/key/OMKeyCommitRequest.java | 37 +-
.../om/request/key/OMKeyCommitRequestWithFSO.java | 190 ++++
.../ozone/om/request/key/OMKeyCreateRequest.java | 24 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 232 +++++
.../ozone/om/request/key/OMKeyDeleteRequest.java | 1 +
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 205 ++++
.../ozone/om/request/key/OMKeyPurgeRequest.java | 1 +
.../ozone/om/request/key/OMKeyRenameRequest.java | 1 +
.../om/request/key/OMKeyRenameRequestWithFSO.java | 301 ++++++
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 285 +++---
.../ozone/om/request/key/OMKeysDeleteRequest.java | 1 -
.../om/request/key/OMPathsPurgeRequestWithFSO.java | 64 ++
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 167 +++
.../om/request/key/acl/OMKeyAddAclRequest.java | 1 +
.../request/key/acl/OMKeyAddAclRequestWithFSO.java | 151 +++
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 1 +
.../key/acl/OMKeyRemoveAclRequestWithFSO.java | 161 +++
.../om/request/key/acl/OMKeySetAclRequest.java | 1 +
.../request/key/acl/OMKeySetAclRequestWithFSO.java | 155 +++
.../key/acl/prefix/OMPrefixAddAclRequest.java | 1 +
.../key/acl/prefix/OMPrefixRemoveAclRequest.java | 1 +
.../key/acl/prefix/OMPrefixSetAclRequest.java | 1 +
.../S3InitiateMultipartUploadRequest.java | 25 +-
.../S3InitiateMultipartUploadRequestWithFSO.java | 251 +++++
.../multipart/S3MultipartUploadAbortRequest.java | 58 +-
.../S3MultipartUploadAbortRequestWithFSO.java | 89 ++
.../S3MultipartUploadCommitPartRequest.java | 76 +-
.../S3MultipartUploadCommitPartRequestWithFSO.java | 87 ++
.../S3MultipartUploadCompleteRequest.java | 405 +++++---
.../S3MultipartUploadCompleteRequestWithFSO.java | 169 ++++
.../om/request/s3/security/S3GetSecretRequest.java | 1 +
.../security/OMCancelDelegationTokenRequest.java | 2 +-
.../security/OMGetDelegationTokenRequest.java | 2 +-
.../security/OMRenewDelegationTokenRequest.java | 1 +
.../om/request/upgrade/OMCancelPrepareRequest.java | 97 ++
.../request/upgrade/OMFinalizeUpgradeRequest.java | 116 +++
.../ozone/om/request/upgrade/OMPrepareRequest.java | 257 +++++
.../ozone/om/request/upgrade/package-info.java | 23 +
.../om/request/volume/OMVolumeDeleteRequest.java | 2 +
.../om/request/volume/OMVolumeSetOwnerRequest.java | 1 +
.../om/request/volume/OMVolumeSetQuotaRequest.java | 1 +
.../hadoop/ozone/om/response/CleanupTableInfo.java | 10 +-
.../file/OMDirectoryCreateResponseWithFSO.java | 103 ++
.../response/file/OMFileCreateResponseWithFSO.java | 94 ++
.../response/key/AbstractOMKeyDeleteResponse.java | 44 +
.../om/response/key/OMAllocateBlockResponse.java | 12 +
.../key/OMAllocateBlockResponseWithFSO.java | 67 ++
.../ozone/om/response/key/OMKeyCommitResponse.java | 15 +
.../response/key/OMKeyCommitResponseWithFSO.java | 75 ++
.../ozone/om/response/key/OMKeyCreateResponse.java | 14 +-
.../response/key/OMKeyCreateResponseWithFSO.java | 56 +
.../ozone/om/response/key/OMKeyDeleteResponse.java | 8 +
.../response/key/OMKeyDeleteResponseWithFSO.java | 100 ++
.../ozone/om/response/key/OMKeyRenameResponse.java | 11 +
.../response/key/OMKeyRenameResponseWithFSO.java | 77 ++
.../om/response/key/OMOpenKeysDeleteRequest.java | 1 +
.../response/key/OMPathsPurgeResponseWithFSO.java | 125 +++
.../om/response/key/acl/OMKeyAclResponse.java | 3 +
.../response/key/acl/OMKeyAclResponseWithFSO.java | 75 ++
.../S3InitiateMultipartUploadResponseWithFSO.java | 90 ++
.../multipart/S3MultipartUploadAbortResponse.java | 9 +-
.../S3MultipartUploadAbortResponseWithFSO.java | 57 ++
.../S3MultipartUploadCommitPartResponse.java | 3 +
...S3MultipartUploadCommitPartResponseWithFSO.java | 69 ++
.../S3MultipartUploadCompleteResponse.java | 41 +-
.../S3MultipartUploadCompleteResponseWithFSO.java | 85 ++
.../om/response/s3/security/package-info.java | 2 +-
.../response/upgrade/OMCancelPrepareResponse.java | 46 +
.../upgrade/OMFinalizeUpgradeResponse.java | 58 ++
.../om/response/upgrade/OMPrepareResponse.java | 60 ++
.../ozone/om/response/upgrade/package-info.java | 23 +
.../ozone/om/upgrade/BelongsToLayoutVersion.java | 33 +
.../om/upgrade/DisallowedUntilLayoutVersion.java | 35 +
.../hadoop/ozone/om/upgrade/OMLayoutFeature.java | 80 ++
.../ozone/om/upgrade/OMLayoutFeatureAspect.java | 114 +++
.../ozone/om/upgrade/OMLayoutVersionManager.java | 210 ++++
.../ozone/om/upgrade/OMUpgradeFinalizer.java | 50 +
.../hadoop/ozone/om/upgrade/OmUpgradeAction.java | 28 +
.../hadoop/ozone/om/upgrade/UpgradeActionOm.java | 37 +
.../hadoop/ozone/om/upgrade/package-info.java | 23 +
...OzoneManagerProtocolServerSideTranslatorPB.java | 13 +-
.../protocolPB/OzoneManagerRequestHandler.java | 80 +-
.../ozone/security/acl/OzoneNativeAuthorizer.java | 35 +-
.../hadoop/ozone/om/TestBucketManagerImpl.java | 10 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 21 +-
.../org/apache/hadoop/ozone/om/TestOMStorage.java | 2 +-
.../hadoop/ozone/om/TestOmMetadataManager.java | 9 +
.../ozone/om/TestOzoneManagerHttpServer.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerStarter.java | 28 +
.../apache/hadoop/ozone/om/TestTrashService.java | 4 +
.../hadoop/ozone/om/failover/TestOMFailovers.java | 2 +-
...tOzoneManagerDoubleBufferWithDummyResponse.java | 2 +-
...TestOzoneManagerDoubleBufferWithOMResponse.java | 7 +-
.../om/ratis/TestOzoneManagerRatisServer.java | 2 +-
.../om/ratis/TestOzoneManagerStateMachine.java | 108 +-
.../ozone/om/request/TestOMRequestUtils.java | 200 +++-
.../ozone/om/request/bucket/TestBucketRequest.java | 6 +
.../request/bucket/TestOMBucketCreateRequest.java | 4 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 115 +++
.../bucket/TestOMBucketSetPropertyRequest.java | 12 +-
.../file/TestOMDirectoryCreateRequestWithFSO.java | 657 ++++++++++++
.../om/request/file/TestOMFileCreateRequest.java | 69 +-
.../file/TestOMFileCreateRequestWithFSO.java | 197 ++++
.../om/request/key/TestOMAllocateBlockRequest.java | 44 +-
.../key/TestOMAllocateBlockRequestWithFSO.java | 120 +++
.../ozone/om/request/key/TestOMKeyAclRequest.java | 81 +-
.../om/request/key/TestOMKeyAclRequestWithFSO.java | 81 ++
.../om/request/key/TestOMKeyCommitRequest.java | 86 +-
.../request/key/TestOMKeyCommitRequestWithFSO.java | 114 +++
.../om/request/key/TestOMKeyCreateRequest.java | 53 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 135 +++
.../om/request/key/TestOMKeyDeleteRequest.java | 40 +-
.../request/key/TestOMKeyDeleteRequestWithFSO.java | 139 +++
.../ozone/om/request/key/TestOMKeyRequest.java | 51 +-
.../TestS3InitiateMultipartUploadRequest.java | 24 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 145 +++
.../s3/multipart/TestS3MultipartRequest.java | 60 +-
.../TestS3MultipartUploadAbortRequest.java | 32 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 69 ++
.../TestS3MultipartUploadCommitPartRequest.java | 70 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 118 +++
.../TestS3MultipartUploadCompleteRequest.java | 118 ++-
...estS3MultipartUploadCompleteRequestWithFSO.java | 139 +++
.../upgrade/TestOMCancelPrepareRequest.java | 110 ++
.../request/volume/TestOMVolumeCreateRequest.java | 4 +-
.../request/volume/TestOMVolumeDeleteRequest.java | 2 +-
.../om/request/volume/TestOMVolumeRequest.java | 9 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 2 +-
.../ozone/om/response/TestCleanupTableInfo.java | 26 +-
.../file/TestOMDirectoryCreateResponseWithFSO.java | 87 ++
.../file/TestOMFileCreateResponseWithFSO.java | 78 ++
.../response/key/TestOMAllocateBlockResponse.java | 37 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 85 ++
.../om/response/key/TestOMKeyCommitResponse.java | 64 +-
.../key/TestOMKeyCommitResponseWithFSO.java | 102 ++
.../om/response/key/TestOMKeyCreateResponse.java | 38 +-
.../key/TestOMKeyCreateResponseWithFSO.java | 76 ++
.../om/response/key/TestOMKeyDeleteResponse.java | 88 +-
.../key/TestOMKeyDeleteResponseWithFSO.java | 84 ++
.../ozone/om/response/key/TestOMKeyResponse.java | 25 +-
.../TestS3InitiateMultipartUploadResponse.java | 2 +-
...stS3InitiateMultipartUploadResponseWithFSO.java | 85 ++
.../s3/multipart/TestS3MultipartResponse.java | 191 +++-
.../TestS3MultipartUploadAbortResponse.java | 23 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 85 ++
...S3MultipartUploadCommitPartResponseWithFSO.java | 226 +++++
...stS3MultipartUploadCompleteResponseWithFSO.java | 279 +++++
.../hadoop/ozone/om/upgrade/MockOmRequest.java | 32 +
.../ozone/om/upgrade/OMLayoutFeatureUtil.java | 66 ++
.../om/upgrade/TestOMLayoutFeatureAspect.java | 99 ++
.../ozone/om/upgrade/TestOMUpgradeFinalizer.java | 290 ++++++
.../ozone/om/upgrade/TestOMVersionManager.java | 195 ++++
.../TestOmVersionManagerRequestFactory.java | 111 ++
.../om/upgrade/TestOzoneManagerPrepareState.java | 283 ++++++
.../security/TestOzoneBlockTokenSecretManager.java | 4 +-
.../TestOzoneDelegationTokenSecretManager.java | 2 +-
.../ozone/security/TestOzoneManagerBlockToken.java | 2 +-
.../ozone/security/TestOzoneTokenIdentifier.java | 2 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 7 +-
.../hadoop/ozone/security/acl/TestOzoneObj.java | 79 ++
.../hadoop/ozone/security/acl/TestParentAcl.java | 411 ++++++++
.../ozone/security/acl/TestRequestContext.java | 7 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 10 +-
hadoop-ozone/ozonefs-common/pom.xml | 42 +-
.../fs/ozone/BasicOzoneClientAdapterImpl.java | 34 +-
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 37 +
.../ozone/BasicRootedOzoneClientAdapterImpl.java | 28 +-
.../fs/ozone/BasicRootedOzoneFileSystem.java | 37 +
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 5 +-
.../apache/hadoop/fs/ozone/OzoneFSInputStream.java | 30 +-
hadoop-ozone/ozonefs-hadoop2/pom.xml | 16 +-
hadoop-ozone/ozonefs-hadoop3/pom.xml | 14 +-
hadoop-ozone/ozonefs-shaded/pom.xml | 14 +-
hadoop-ozone/ozonefs/pom.xml | 14 +-
hadoop-ozone/pom.xml | 142 +--
hadoop-ozone/recon-codegen/pom.xml | 10 +-
hadoop-ozone/recon/pom.xml | 43 +-
.../apache/hadoop/ozone/recon/ReconConstants.java | 7 +
.../hadoop/ozone/recon/ReconControllerModule.java | 18 +-
.../org/apache/hadoop/ozone/recon/ReconServer.java | 23 +-
.../ozone/recon/api/ClusterStateEndpoint.java | 3 +-
.../hadoop/ozone/recon/api/ContainerEndpoint.java | 17 +-
.../hadoop/ozone/recon/api/NodeEndpoint.java | 22 +-
.../ozone/recon/api/types/DatanodeMetadata.java | 28 +
.../hadoop/ozone/recon/api/types/NSSummary.java | 70 ++
.../hadoop/ozone/recon/codec/NSSummaryCodec.java | 81 ++
.../ozone/recon/fsck/ContainerHealthStatus.java | 10 +-
.../ozone/recon/fsck/ContainerHealthTask.java | 51 +-
.../ozone/recon/persistence/ContainerHistory.java | 3 +
.../hadoop/ozone/recon/scm/PipelineSyncTask.java | 46 +-
.../ozone/recon/scm/ReconContainerManager.java | 26 +-
.../ozone/recon/scm/ReconDeadNodeHandler.java | 79 ++
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 55 +-
.../scm/ReconStorageContainerManagerFacade.java | 32 +-
.../recon/spi/ContainerDBServiceProvider.java | 198 ----
.../recon/spi/ReconContainerMetadataManager.java | 193 ++++
.../recon/spi/ReconNamespaceSummaryManager.java | 37 +
.../recon/spi/StorageContainerServiceProvider.java | 6 +
.../spi/impl/ContainerDBServiceProviderImpl.java | 488 ---------
.../recon/spi/impl/ReconContainerDBProvider.java | 95 --
.../impl/ReconContainerMetadataManagerImpl.java | 458 +++++++++
.../ozone/recon/spi/impl/ReconDBDefinition.java | 13 +-
.../ozone/recon/spi/impl/ReconDBProvider.java | 117 +++
.../spi/impl/ReconNamespaceSummaryManagerImpl.java | 67 ++
.../impl/StorageContainerServiceProviderImpl.java | 8 +
.../ozone/recon/tasks/ContainerKeyMapperTask.java | 35 +-
.../ozone/recon/tasks/FileSizeCountTask.java | 5 +-
.../hadoop/ozone/recon/tasks/ReconTaskConfig.java | 4 +-
.../webapps/recon/ozone-recon-web/api/db.json | 20 +
.../src/views/datanodes/datanodes.tsx | 13 +-
.../ozone/recon/OMMetadataManagerTestUtils.java | 7 +-
.../hadoop/ozone/recon/ReconTestInjector.java | 20 +-
.../ozone/recon/api/TestContainerEndpoint.java | 15 +-
.../hadoop/ozone/recon/api/TestEndpoints.java | 23 +-
.../ozone/recon/api/TestOpenContainerCount.java | 427 ++++++++
.../recon/fsck/TestContainerHealthStatus.java | 6 +-
.../ozone/recon/fsck/TestContainerHealthTask.java | 97 +-
.../TestContainerHealthTaskRecordGenerator.java | 6 +-
.../recovery/TestReconOmMetadataManagerImpl.java | 9 +-
.../scm/AbstractReconContainerManagerTest.java | 31 +-
.../ozone/recon/scm/TestReconContainerManager.java | 5 +-
...TestReconIncrementalContainerReportHandler.java | 16 +-
.../ozone/recon/scm/TestReconNodeManager.java | 58 +-
.../ozone/recon/scm/TestReconPipelineManager.java | 23 +-
.../impl/TestContainerDBServiceProviderImpl.java | 386 -------
.../spi/impl/TestReconContainerDBProvider.java | 74 --
.../TestReconContainerMetadataManagerImpl.java | 388 +++++++
.../ozone/recon/spi/impl/TestReconDBProvider.java | 71 ++
.../impl/TestReconNamespaceSummaryManagerImpl.java | 96 ++
.../recon/tasks/TestContainerKeyMapperTask.java | 63 +-
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 5 +-
hadoop-ozone/s3gateway/pom.xml | 47 +-
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 5 +
.../org/apache/hadoop/ozone/s3/endpoint/S3Acl.java | 6 +
.../hadoop/ozone/s3/endpoint/S3BucketAcl.java | 6 +
.../apache/hadoop/ozone/s3/endpoint/S3Owner.java | 3 +
.../hadoop/ozone/s3/signature/SignatureInfo.java | 3 +
.../ozone/s3/TestEmptyContentTypeFilter.java | 5 +-
.../ozone/s3/TestVirtualHostStyleFilter.java | 2 +-
.../signature/TestAuthorizationV4HeaderParser.java | 2 +-
.../signature/TestAuthorizationV4QueryParser.java | 5 +-
.../hadoop/ozone/s3/util/TestOzoneS3Util.java | 2 +-
hadoop-ozone/tools/pom.xml | 50 +-
.../ozone/admin/om/CancelPrepareSubCommand.java | 53 +
.../admin/om/FinalizationStatusSubCommand.java | 64 ++
.../ozone/admin/om/FinalizeUpgradeSubCommand.java | 171 ++++
.../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 44 +-
.../hadoop/ozone/admin/om/PrepareSubCommand.java | 173 ++++
.../admin/scm/FinalizationScmStatusSubcommand.java | 51 +
.../admin/scm/FinalizeScmUpgradeSubcommand.java | 156 +++
.../admin/scm/FinalizeUpgradeCommandUtil.java | 92 ++
.../apache/hadoop/ozone/admin/scm/ScmAdmin.java | 4 +-
.../ozone/audit/parser/common/DatabaseHelper.java | 7 +-
.../apache/hadoop/ozone/debug/DatanodeLayout.java | 8 +-
.../apache/hadoop/ozone/debug/PrefixParser.java | 236 +++++
.../ozone/debug/container/ContainerCommands.java | 14 +-
.../ozone/debug/container/ExportSubcommand.java | 3 +
.../ozone/debug/container/InfoSubcommand.java | 3 +
.../ozone/debug/container/ListSubcommand.java | 3 +
.../hadoop/ozone/freon/ChunkManagerDiskWrite.java | 4 +-
.../ozone/freon/ClosedContainerReplicator.java | 7 +-
.../apache/hadoop/ozone/freon/OmKeyGenerator.java | 5 +-
.../hadoop/ozone/freon/StreamingGenerator.java | 26 +-
.../freon/containergenerator/BaseGenerator.java | 3 +
.../containergenerator/GeneratorDatanode.java | 7 +-
.../freon/containergenerator/GeneratorOm.java | 11 +-
.../freon/containergenerator/GeneratorScm.java | 6 +-
.../hadoop/ozone/genesis/BenchMarkCRCBatch.java | 3 +
.../ozone/genesis/BenchMarkCRCStreaming.java | 3 +
.../ozone/genesis/BenchMarkContainerStateMap.java | 18 +-
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 4 +-
.../ozone/genesis/BenchMarkMetadataStoreReads.java | 70 --
.../genesis/BenchMarkMetadataStoreWrites.java | 62 --
.../ozone/genesis/BenchMarkOMKeyAllocation.java | 6 +-
.../ozone/genesis/BenchMarkOzoneManager.java | 13 +-
.../ozone/genesis/BenchMarkRocksDbStore.java | 119 ---
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 32 -
.../org/apache/hadoop/ozone/scm/cli/SQLCLI.java | 562 -----------
.../hadoop/ozone/audit/parser/TestAuditParser.java | 31 +-
.../TestGenerateOzoneRequiredConfigurations.java | 2 +-
.../tools/src/test/resources/testaudit.log | 3 +
.../tools/src/test/resources/testloadaudit.log | 21 +
pom.xml | 42 +-
1108 files changed, 50928 insertions(+), 12644 deletions(-)
create mode 100644
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteArrayReader.java
create mode 100644
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteBufferReader.java
create mode 100644
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteReaderStrategy.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ha/RetriableWithFailOverException.java
delete mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfo.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLStatus.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutFeature.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutVersionManager.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSUpgradeAction.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/package-info.java
delete mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/EntryConsumer.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/AbstractLayoutVersionManager.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/BasicUpgradeFinalizer.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/DefaultUpgradeFinalizationExecutor.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutFeature.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutVersionInstanceFactory.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutVersionManager.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutVersionManagerMXBean.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeActionHdds.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeException.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeFinalizer.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/VersionFactoryKey.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/package-info.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/upgrade/TestHDDSLayoutVersionManager.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/upgrade/test/MockComponent.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/InjectedUpgradeFinalizationExecutor.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestAbstractLayoutVersionManager.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestBasicUpgradeFinalizer.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestDefaultUpgradeFinalizationExecutor.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestLayoutVersionInstanceFactory.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestUpgradeFinalizerActions.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestUpgradeUtils.java
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DataNodeLayoutVersion.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DatanodeLayoutStorage.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/HDDSVolumeLayoutVersion.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/CRLStatusReportPublisher.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/FinalizeNewLayoutVersionCommandHandler.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/StorageVolumeUtil.java
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeChecker.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeFactory.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolume.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolumeFactory.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolume.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolumeChecker.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolumeFactory.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingException.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DataNodeUpgradeFinalizer.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeMetadataFeatures.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeSchemaV2FinalizeAction.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/ScmHAFirstUpgradeLayoutChangeAction.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/UpgradeUtils.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/package-info.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/FinalizeNewLayoutVersionCommand.java
delete mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestHddsVolumeChecker.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestStorageVolumeChecker.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/TestDirstreamClientHandler.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/package-info.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDataNodeStartupSlvLessThanMlv.java
create mode 100644 hadoop-hdds/docs/content/design/nonrolling-upgrade.md
create mode 100644 hadoop-hdds/docs/content/design/omprepare.md
create mode 100644 hadoop-hdds/docs/content/design/upgrade-dev-primer.md
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Delete.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Format.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Rename.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO.md
create mode 100644
hadoop-hdds/docs/content/feature/how-to-do-a-nonrolling-upgrade.md
create mode 100644 hadoop-hdds/docs/content/feature/scm-secure-ha.png
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/CRLClientUpdateHandler.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/CRLStore.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/ClientCRLStore.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/ClientUpdateHandler.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/SCMUpdateClientConfiguration.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/SCMUpdateServiceGrpcClient.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/UpdateServiceConfig.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/package-info.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/CRLClientInfo.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateClientInfo.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/package-info.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ContainerTokenGenerator.java
rename hadoop-hdds/{common =>
framework}/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLCodec.java
(100%)
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfo.java
rename hadoop-hdds/{common =>
framework}/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfoCodec.java
(100%)
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/FixedThreadPoolExecutor.java
delete mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/LevelDBStore.java
delete mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/LevelDBStoreIterator.java
delete mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/MetadataStore.java
delete mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/MetadataStoreBuilder.java
delete mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStore.java
delete mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreIterator.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/ShortCodec.java
delete mode 100644
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestMetadataStore.java
delete mode 100644
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestRocksDBStoreMBean.java
rename hadoop-hdds/{interface-client =>
interface-server}/src/main/proto/InterSCMProtocol.proto (100%)
create mode 100644
hadoop-hdds/interface-server/src/main/proto/SCMUpdateProtocol.proto
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteHandler.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteStatusList.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/crl/CRLStatusReportHandler.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/crl/package-info.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/HealthyReadOnlyNodeHandler.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NonHealthyToHealthyNodeHandler.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/ReadOnlyHealthyToHealthyNodeHandler.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableContainerFactory.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableContainerProvider.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/SCMUpgradeFinalizer.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/ScmHAUnfinalizedStateValidationAction.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/ScmOnFinalizeActionForDatanodeSchemaV2.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/package-info.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMCRLStore.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMCRLUpdateHandler.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateClientManager.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateHandler.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateServiceGrpcServer.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateServiceImpl.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/package-info.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestContainerBalancer.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/crl/TestCRLStatusReportHandler.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/update/server/MockCRLStore.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/update/server/TestSCMUpdateServiceGrpcServer.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/TestSCMHAUnfinalizedStateValidationAction.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/TestScmStartupSlvLessThanMlv.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/package-info.java
delete mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/GenericTestUtils.java
delete mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/LambdaTestUtils.java
delete mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/TimedOutTestsListener.java
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/GenericTestUtils.java
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/LambdaTestUtils.java
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/TimedOutTestsListener.java
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/package-info.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerCommands.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStartSubcommand.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java
create mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStopSubcommand.java
create mode 100644
hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestContainerBalancerSubCommand.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OmUpgradeConfig.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/WithParentObjectId.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzonePrefixPath.java
delete mode 100644
hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/Dockerfile-krb5
delete mode 100644
hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/README.md
delete mode 100644
hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/kadm5.acl
delete mode 100644
hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/krb5.conf
delete mode 100644
hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/launcher.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-om-prepare/.env
create mode 100644
hadoop-ozone/dist/src/main/compose/ozone-om-prepare/docker-compose.yaml
create mode 100644
hadoop-ozone/dist/src/main/compose/ozone-om-prepare/docker-config
create mode 100755 hadoop-ozone/dist/src/main/compose/ozone-om-prepare/test.sh
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/HTTP.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/dn.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/om.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/recon.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/s3g.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/scm.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/krb5.conf
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/HTTP.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/dn.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/om.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/recon.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/s3g.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/scm.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/krb5.conf
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/HTTP.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/dn.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/om.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/recon.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/s3g.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/scm.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser.keytab
create mode 100755
hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/krb5.conf
delete mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/.env
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/0.5.0/test.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/1.0.0/test.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/.env
create mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/docker-compose.yaml
create mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/docker-config
create mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/load.sh
create mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
create mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
create mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
create mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
delete mode 100644
hadoop-ozone/dist/src/main/compose/upgrade/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/docker-config
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrade_to_current.sh
delete mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrade_to_release.sh
create mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/0.5.0-1.1.0/callback.sh
create mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh
create mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/non-rolling-upgrade/1.0.0-1.1.0/callback.sh
create mode 100755
hadoop-ozone/dist/src/main/compose/upgrade/upgrades/non-rolling-upgrade/driver.sh
create mode 100755 hadoop-ozone/dist/src/main/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/hadoop.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/jhs.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/nm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/rm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/testuser2.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/update-keytabs.sh
create mode 100644
hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-org.aspectj.html
create mode 100644
hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-org.reflections-reflections.txt
create mode 100644
hadoop-ozone/dist/src/main/smoketest/omha/om-cancel-prepare.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/omha/om-prepare.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/omha/om-prepared.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/check-mlv.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/finalize.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/generate.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/prepare.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/validate.robot
create mode 100755 hadoop-ozone/dist/src/shell/upgrade/1.0.0.sh
rename hadoop-ozone/dist/src/shell/upgrade/{v1 => 1.0.0}/01-migrate-scm-db.sh
(100%)
delete mode 100755 hadoop-ozone/dist/src/shell/upgrade/v1.sh
create mode 100644 hadoop-ozone/dist/src/test/shell/failing1/test.sh
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestDirectoryDeletingServiceWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfacesWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemPrefixParser.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystemWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractUtils.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestHDDSUpgrade.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestContainerBalancerOperations.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneClientMultipartUploadWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/package-info.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/volume/TestDatanodeHddsVolumeFailureToleration.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestHadoopDirTreeGeneratorWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMStartupWithLayout.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStoreWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupSlvLessThanMlv.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerPrepare.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestRecursiveAclWithFSO.java
create mode 100644
hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/codec/OmDirectoryInfoCodec.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerPrepareState.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzonePrefixPathImpl.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMAllocateBlockRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyDeleteRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRenameRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMPathsPurgeRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAclRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAddAclRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyRemoveAclRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeySetAclRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3InitiateMultipartUploadRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadAbortRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCommitPartRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCompleteRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMCancelPrepareRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMFinalizeUpgradeRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/package-info.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/file/OMDirectoryCreateResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/file/OMFileCreateResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMAllocateBlockResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyCommitResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyCreateResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyDeleteResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyRenameResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMPathsPurgeResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/acl/OMKeyAclResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3InitiateMultipartUploadResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadAbortResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCommitPartResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCompleteResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMCancelPrepareResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMFinalizeUpgradeResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMPrepareResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/package-info.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/BelongsToLayoutVersion.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/DisallowedUntilLayoutVersion.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeature.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeatureAspect.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutVersionManager.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OmUpgradeAction.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/UpgradeActionOm.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/package-info.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/TestOMBucketCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMAllocateBlockRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyAclRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCommitRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCreateRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyDeleteRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3InitiateMultipartUploadRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadAbortRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCommitPartRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCompleteRequestWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/upgrade/TestOMCancelPrepareRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/file/TestOMDirectoryCreateResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/file/TestOMFileCreateResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyCommitResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyCreateResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyDeleteResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3InitiateMultipartUploadResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadAbortResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadCommitPartResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadCompleteResponseWithFSO.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/MockOmRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeatureUtil.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeatureAspect.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMUpgradeFinalizer.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMVersionManager.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOmVersionManagerRequestFactory.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOzoneManagerPrepareState.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestOzoneObj.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestParentAcl.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/NSSummary.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/codec/NSSummaryCodec.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconDeadNodeHandler.java
delete mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ContainerDBServiceProvider.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ReconContainerMetadataManager.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ReconNamespaceSummaryManager.java
delete mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ContainerDBServiceProviderImpl.java
delete mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerDBProvider.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerMetadataManagerImpl.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconDBProvider.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconNamespaceSummaryManagerImpl.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestOpenContainerCount.java
delete mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestContainerDBServiceProviderImpl.java
delete mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerDBProvider.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerMetadataManagerImpl.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconDBProvider.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconNamespaceSummaryManagerImpl.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/CancelPrepareSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/FinalizationStatusSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/FinalizeUpgradeSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/PrepareSubCommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/FinalizationScmStatusSubcommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/FinalizeScmUpgradeSubcommand.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/FinalizeUpgradeCommandUtil.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/PrefixParser.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreReads.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreWrites.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkRocksDbStore.java
delete mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/scm/cli/SQLCLI.java
create mode 100644 hadoop-ozone/tools/src/test/resources/testloadaudit.log
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
