This is an automated email from the ASF dual-hosted git repository.
licheng pushed a change to branch HDDS-2823
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
discard 63a4b0e HDDS-3187 Construct SCM StateMachine. (#819)
omit d156eae HDDS-3185 Construct a standalone ratis server for SCM. (#720)
add 6b5298b HDDS-3160. Disable index and filter block cache for RocksDB
add 94413cd HDDS-3023. Create Freon test to test isolated Ratis LEADER
add 18577a9 HDDS-3236. Fix Dropwizard metrics mapping for latest Ratis
metrics
add 2fefd46 HDDS-3142. Create isolated environment for OM to test it
without SCM
add 3d3b374 HDDS-3249: remove unrelated assert (#737)
add 97f59f8 HDDS-3248. shutdown defaultMetricsSystem before tests (#704)
add 924b50c HDDS-2621. Enable OM HA acceptance tests. (#265)
add bd1995a HDDS-2184. Rename ozone scmcli to ozone admin (#687)
add f2343f1 HDDS-3156 update allocateContainer to remove additional
createPipeline step. (#663)
add 410e4d3 HDDS-2339. Add OzoneManager to MiniOzoneChaosCluster (#643)
add 7faa07f HDDS-3289. Add a freon generator to create nested
directories. (#730)
add 32f13f7 HDDS-3289. (Checkstyle addendum) Add a freon generator to
create nested directories. (#730)
add f43b33e HDDS-3281. (Addendum: topology) Add timeouts to all robot
tests
add ce6ad30 HDDS-2621. Revert/Disable OM HA tests.
add 61b6be0 HDDS-3165. Integrate Recon missing containers UI with
endpoint. (#735)
add 1ccf78f HDDS-3047. ObjectStore#listVolumesByUser and
CreateVolumeHandler#call should get principal name by default (#582)
add 6384b8d HDDS-3130. Add jaeger trace span in s3gateway
add 8d4aa8d HDDS-3240. Improve write efficiency by creating container in
parallel (#734)
add e6c8e4c HDDS-3311. OM logs not available for OM HA acceptance test
(#747)
add 6661469 HDDS-3308. Update Ratis snapshot (#741)
add 55d78d6 HDDS-3330. Disable TestDeleteWithSlowFollower test as it has
known problem.
add f1da73b HDDS-3324. OM Client fails with
StringIndexOutOfBoundsException. (#759)
add 0189f8d HDDS-3334. OM Client failover to next OM on
NotLeaderException (#760)
add 63fde40 HDDS-3253. Initialize Recon metrics for prometheus at /prom
endpoint
add 1ebadbe HDDS-3173. Provide better default JVM options
add 12fb2f6 HDDS-3316. Checkstyle check fails silently in case of mvn
related errors
add 1ceb514 HDDS-3239. Provide message-level metrics from the generic
protocol dispatch
add 692cd7a HDDS-3320. Add read only based load generator to
MiniOzoneChaosCluster. (#754)
add 9f8ff0e HDDS-3295. Ozone admins getting Permission Denied error while
creating volume
add e9742db HDDS-3318. Support /close command in the Github comments
add dc9b24d HDDS-2798. beyond/Containers.md translation
add 598e24c HDDS-3332. Make Robot test invocation independent from Python
version (#757)
add b4eafde HDDS-3332. Upgrade Robot tests to Python 3 (#766)
add f04a409 HDDS-3339. Rename CONTRIBUTION.md to CONTRIBUTING.md (#765)
add f2c32e4 HDDS-3337. Export the IPC port of OM in docker-compose
add 065d443 HDDS-3224. Enforce volume and bucket name rule at create
time. (#718)
add 7d45380 HDDS-2691. Ozone Shell code cleanup
add 11b0984 HDDS-3341. Checkstyle fails for new modules/versions (#768)
add 876bec0 HDDS-2817. Fix listing buckets for setting --prefix equal to
bucket name (#430)
add e54d0b5 HDDS-3237. Recon should provide the list of datanodes that a
missing … (#753)
add 0fbacbc HDDS-2793. concept/Datanodes.md translation (#520)
add b80aaad HDDS-3349. Fix two link addresses in README.md (#774)
add d3b81a4 HDDS-3241. Invalid container reported to SCM should be deleted
add 54580ec HDDS-3351. Remove unnecessary dependency Curator. (#777)
add 7164c19 HDDS-3344. Remove Jenkins file (#778)
add 9872253 HDDS-3328. List design docs as part of the documentation
page. (#755)
add ad92019 HDDS-3312. Remove sever-side dependencies from
hdds/ozone-common
add 15db251 HDDS-3319. Handle HA for
BasicOzoneClientAdapterImpl#renew/cancel(). (#752)
add b0973ff HDDS-3280. Ozone BaseHTTPServer should honor
ozone.security.enabled c… (#758)
add a1ae540 HDDS-3347. PipelineActionHandler should handle unknown
pipeline. (#773)
add de30b37 HDDS-3348. scmcli container info command shows the wrong
container state. (#776)
add 89c34b4 HDDS-3353. Remove unnecessary transitive hadoop-common
dependencies on server side.
add ccf8bd8 HDDS-3331. Simplify s3bucket -> ozone volume/bucket mapping
(#756)
add a24a79b HDDS-3270. Allow safemode listeners to be notified when some
precheck rules pass (#719)
add aae0fd3 HDDS-3290. REVERT: Disable all the freon integration tests.
(#786)
add e2ebbf8 HDDS-3302. Add ability in MiniOzoneCluster to create multiple
storage directories
add b50e932 HDDS-3244. Improve write efficiency by opening RocksDB only
once (#709)
add eb177f4 HDDS-3366. Ozone Client should not retry on invalid token
(#802)
add ebc6160 HDDS-3340. Move Ozone Shell from ozone-manager to tools
add 1412b6c HDDS-3359. Remove Yetus helper code from ozone
add e31a4ce Revert "HDDS-3291. Write operation when both OM followers are
shutdown. (#733)" (#803)
add e71c383 HDDS-3365. Ensure OzoneConfiguration is initialized in
OzoneClientFactory#getOzoneClient. (#798)
add 216b41f HDDS-3335. Recon unit tests cleanup. (#772)
add 5d8be37 HDDS-2294. Create a new HISTORY.md in the new repository.
(#764)
add 486d749 HDDS-3135. Enable topology acceptance test added in HDDS-3084
to read data when racks stopped (#790)
add c75d4c7 HDDS-3372. Delete HISTORY.txt (#805)
add 90c29d7 HDDS-3093. Allow forced overwrite of local file (#800)
add 129496f HDDS-3368. Ozone filesystem jar should not include webapps
folder (#804)
add 09ff517 HDDS-3377. Remove guava 26.0-android jar. (#808)
add b037c93 Handle InterruptedException in OzoneContainer (#801)
add c9a1925 HDDS-2976. Recon throws error while trying to get snapshot
over https (#783)
add ac5fb0f HDDS-3374. OMVolumeSetOwnerRequest doesn't check if user is
already the owner (#806)
add 25913f7 HDDS-3355. Add wait time between client retries to OM (#784)
add 8287c72 HDDS-3374. Addendum: OMVolumeSetOwnerRequest doesn't check if
user is already the owner (#821)
add 506778f HDDS-3134. Debug Tool that gives chunk location information
given a key. (#684)
add 5936594 HDDS-2802. tools/AuditParser.md translation (#793)
add 9d0d341 HDDS-3394. Skip generation of encryptionkey for directory
create operation. (#823)
add 901a7bc HDDS-3384. Update SpringFramework to 5.1.14. (#829)
add bc8d43e HDDS-3357. Add check for import from shaded package (#788)
add e9a466d HDDS-2682: OM File create request does not check for existing
directory with the same name (#824)
add 8712d50 HDDS-3387. Fix ContainerOperationClient#createContainer (#831)
add 1505778 HDDS-3386. Remove unnecessary transitive hadoop-common
dependencies on server side (addendum). (#818)
add 17456bc HDDS-3386. Remove unnecessary transitive hadoop-common
dependencies on server side (addendum) -- fix checkstyle
add 3699e79 HDDS-3291. Write operation when both OM followers are
shutdown. (#815)
add f020e6a HDDS-3392.OM create key/file should not generate different
data encryption key during validateAndUpdateCache. (#830)
add 67ca42c HDDS-3401. Ozone audit entries could be consistent among
volume creation with quota and update quota (#832)
add 2f37a25 HDDS-3322. StandAlone Pipelines are created in an infinite
loop (#749)
add 57c9cc9 HDDS-3360. Duplicate Ozone Client in renew/cancel token
handlers (#799)
add 151fe08 HDDS-2575. Handle InterruptedException in LogSubcommand
add f58cae9 HDDS-3336. Metrics for Recon OzoneManager DB sync.
add 8b4d3f5 HDDS-3406. Remove RetryInvocation INFO logging from ozone CLI
output (#838)
add 77bb315 HDDS-3400. Extract test utilities to separate module
add 09fd46b HDDS-3403. Generate ozone specific version from type in
FSProto.proto
add 94beb2a HDDS-2911. lastUsed and stateEnterTime value in container
info is not human friendly
add 3bb5838 Update grpc/netty dependencies (#840)
add 8c80e3c Revert "Update grpc/netty dependencies (#840)" (#841)
add e800d2b HDDS-3397. Make jmh jar dependencies optional
add 491d69a HDDS-3456. Fix Acceptance test failures due to disk out of
space. (#844)
add ce94889 HDDS-3380. MiniOzoneHAClusterImpl#initOMRatisConf will reset
the conf… (#817)
add ffb340e HDDS-3101. Depend on lightweight ConfigurationSource
interface instead of Hadoop Configuration
add fc4d951 HDDS-3468. Organize log4j dependency in pom.xml. (#849)
add 2acd2b1 HDDS-3399. Update JaegerTracing
add 6afe2a8 HDDS-3471. Update to latest Ratis Snapshot
0.6.0-490b689-SNAPSHOT. (#851)
add fac26e1 HDDS-3139. Pipeline placement should max out pipeline usage
(#668)
add a59ff25 HDDS-3451. Add timeout to XceiverServerRatis#submitRequest
call. (#845)
add 091993b HDDS-3056. Allow users to list volumes they have access to,
and optionally allow all users to list all volumes (#696)
add 1b7f617 HDDS-3388. Add bucket encryption key info to bucket create
audit log. (#820)
add ca6da46 HDDS-2809. recipe/_index.md translation (#795)
add 00799c2 HDDS-3393. Recon throws NPE in clusterState endpoint (#854)
add 36cdf5a HDDS-3467. OM Client RPC failover retries happening more than
configured. (#853)
add 706fe1b HDDS-3168. Improve read efficiency by merging a lot of RPC
call getContainerWithPipeline into one. (#692)
add 2098516 HDDS-3155. Improved ozone client flush implementation to make
it faster. (#716)
add 0466ade HDDS-3172. Use DBStore instead of MetadataStore in SCM
add fa3aaf1 HDDS-2810. recipe/Prometheus.md translation (#796)
add 811e18c HDDS-3373. Intermittent failure in TestDnRatisLogParser and
TestOMRatisLogParser (#858)
add 52f68c1 HDDS-2801. tools/Genconf.md translation (#792)
add cadf414 HDDS-2804. tools/TestTools.md (#794)
add b1b12a1 HDDS-3472. Remove redundant hdds.version from
hadoop-hdds/common/pom.xml (#852)
add d980e5a HDDS-3483. Handle unhealthy replica state of an open
container.
add 266a9e8 HDDS-3464. Use dedicated build partition for acceptance tests
in github actions environment
add 0e3f807 HDDS-3480. TestSCMNodeMetrics is flaky (#860)
add a53209e HDDS-3352. Support for native ozone filesystem client using
libhdfs. (#782)
add c973bac HDDS-3461. Add acceptance test to smoketest CSI service
startup
add ab0e66e HDDS-3373. Intermittent failure in TestOMRatisLogParser (#867)
add 76e3460 HDDS-3222. Add integration test for Recon FSCK. (#850)
add cb35ed8 Revert "HDDS-3461. Add acceptance test to smoketest CSI
service startup"
add 4b1fa10 HDDS-3465. OM Failover retry happens too quickly when new
leader suggested and retrying on same OM. (#859)
add 81a2886 HDDS-3318. Addendum: Support /close command in the Github
comments. (HDDS-3489)
add f7fb073 HDDS-2569. Handle InterruptedException in LogStreamServlet
add e4febef HDDS-3455. Change MiniLoadGenerator to a pluggable model.
(#842)
add 68fc181 HDDS-3488. Remove unnecessary jackson dependencies from Ozone
add 2791d71 HDDS-3487. Ozone start fails with NullPointerException in TLS
enabled cluster (#871)
add ea94276 HDDS-3490. Remove avro as a dependency in Ozone. (#873)
add eeb7ea2 HDDS-3375. S3A failing complete multipart upload with Ozone
S3. (#807)
add ade09f1 HDDS-3315. Use EventQueue for delayed/immediate safe mode
rule notification
add 9042944 HDDS-3504. Disable flaky topology related acceptance test
until the fix.
add 51e9cfa HDDS-3506. Disable
TestOzoneFileInterfaces.testOzoneManagerLocatedFileStatusBlockOffsetsWithMultiBlockFile
during the fix
add 1293258 HDDS-3321. Prometheus endpoint should have an option to be
configured with Token based authentication.
add b4f594d HDDS-3398. Update Ratis/gRPC/Netty dependencies. (#879)
add 5fbb045 HDDS-3223. Improve s3g read 1GB object efficiency by 100
times (#843)
add 693a88c HDDS-3507. Disable Jaeger tracing by default. (#883)
add 099e808 HDDS-3443. Enable TestOmMetrics#testBucketOps (#882)
add 1a51d58 HDDS-3314. scmcli container info command failing
intermittently (#833)
add 8ce8efa HDDS-2406. Ozone shell key get throws
IllegalArgumentException if pipeline is empty. (#884)
add a7a754b HDDS-3469. Add third party jar versions as properties in
pom.xml (#878)
add d849904 HDDS-3482. Impl getUriDefaultPort to BasicOzFsOzFs, return -1
forever to pass the checkPath
add f2b5b10 HDDS-3053. Decrease the number of the chunk writer threads
add b5d6743 HDDS-3385. Simplify S3 -> Ozone volume mapping
add 3a1681b HDDS-3411. Switch Recon SQL DB to Derby. (#839)
add 72bb9b7 HDDS-3515. Ensure consistent OM token service field in HA
environment. (#889)
add 3271ad0 HDDS-3495. Clarify the document of OzoneFs (#875)
add 7f66b06 HDDS-3502. Remove dependence on commons-lang (#888)
add b0051d7 HDDS-3418. Enable TestNodeFailure test cases. (#848)
add f20cb4e HDDS-2424. Add the recover-trash command server side
handling. (#399)
add 69ee3a3 HDDS-3178. Add unit tests for OMGetDelegationToken Request
and Response (#770)
add e4ee2f3 HDDS-3449. Enable TestSCMPipelineMetrics test cases (#892)
add b2f2d2b HDDS-2796. beyond/_index.md translation (#523)
add 22f1712 HDDS-3504. Topology Test is Intermitent. Reduce container
size to 256MB and blocksize to 64MB (#891)
new 91cb835 HDDS-3185 Construct a standalone ratis server for SCM. (#720)
new 61a9bb4 HDDS-3187 Construct SCM StateMachine. (#819)
new 390a3c1 Resolve conflicts with merge from master.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (63a4b0e)
\
N -- N -- N refs/heads/HDDS-2823 (390a3c1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/comment-commands/close.sh | 41 ++
.github/comment-commands/pending.sh | 7 +
.github/comment-commands/ready.sh | 6 +
.github/workflows/post-commit.yml | 68 +-
.github/workflows/pr.yml | 77 +-
CONTRIBUTION.md => CONTRIBUTING.md | 0
HISTORY.md | 61 ++
README.md | 6 +-
dev-support/bin/qbt | 18 -
dev-support/bin/smart-apply-patch | 18 -
dev-support/bin/test-patch | 18 -
dev-support/bin/yetus-wrapper | 188 -----
dev-support/byteman/mock-scm.btm | 34 +
dev-support/byteman/ratis-mock-followers.btm | 42 ++
hadoop-hdds/client/pom.xml | 9 +-
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 75 +-
.../hadoop/hdds/scm/XceiverClientManager.java | 23 +-
.../hadoop/hdds/scm/XceiverClientMetrics.java | 9 +
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 57 +-
.../hadoop/hdds/scm/client/HddsClientUtils.java | 49 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 15 +-
hadoop-hdds/common/pom.xml | 73 +-
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 6 +
.../java/org/apache/hadoop/hdds/HddsUtils.java | 25 +-
.../hadoop/hdds/annotation/InterfaceStability.java | 3 -
.../org/apache/hadoop/hdds/cli/GenericCli.java | 12 +-
.../hadoop/hdds/conf/HddsPrometheusConfig.java | 32 +-
.../hadoop/hdds/conf/OzoneConfiguration.java | 192 +----
.../hadoop/hdds/freon/FakeClusterTopology.java | 92 +++
.../freon/FakeScmBlockLocationProtocolClient.java | 100 +++
.../FakeScmContainerLocationProtocolClient.java | 76 ++
.../apache/hadoop/hdds/freon}/package-info.java | 14 +-
.../java/org/apache/hadoop/hdds/fs/DUFactory.java | 11 +-
.../hdds/fs/DedicatedDiskSpaceUsageFactory.java | 11 +-
.../hadoop/hdds/fs/SpaceUsageCheckFactory.java | 26 +-
.../hdds/function/SupplierWithIOException.java | 19 +-
.../hdds/ratis/ContainerCommandRequestMessage.java | 7 +-
.../org/apache/hadoop/hdds/ratis/RatisHelper.java | 32 +-
.../hadoop/hdds/scm/ByteStringConversion.java | 4 +-
.../java/org/apache/hadoop/hdds/scm/ScmConfig.java | 24 +
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 4 +-
.../hadoop/hdds/scm/container/ContainerInfo.java | 56 +-
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 4 +-
.../hadoop/hdds/scm/net/NodeSchemaManager.java | 4 +-
.../protocol/StorageContainerLocationProtocol.java | 12 +
...inerLocationProtocolClientSideTranslatorPB.java | 42 +-
.../hadoop/hdds/security/x509/SecurityConfig.java | 22 +-
.../hadoop/hdds/tracing/GrpcServerInterceptor.java | 11 +-
.../apache/hadoop/hdds/tracing/TraceAllMethod.java | 8 +-
.../apache/hadoop/hdds/tracing/TracingUtil.java | 71 +-
.../utils/LegacyHadoopConfigurationSource.java | 80 ++
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 27 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 8 +-
.../org/apache/hadoop/ozone/OzoneSecurityUtil.java | 30 +-
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +-
.../org/apache/hadoop/ozone/lock/LockManager.java | 6 +-
.../java/org/apache/hadoop/ozone/package-info.java | 21 +-
.../proto/StorageContainerLocationProtocol.proto | 12 +
.../common/src/main/resources/ozone-default.xml | 186 ++---
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 7 +-
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 18 +-
.../org/apache/hadoop/hdds/fs/TestDUFactory.java | 8 +-
.../fs/TestDedicatedDiskSpaceUsageFactory.java | 8 +-
.../hadoop/hdds/fs/TestSpaceUsageFactory.java | 40 +-
.../hdds/scm/net/TestNetworkTopologyImpl.java | 38 +-
.../hadoop/hdds/scm/net/TestNodeSchemaManager.java | 16 +-
hadoop-hdds/config/pom.xml | 9 +
.../hdds/conf/ConfigurationReflectionUtil.java | 159 ++++
.../hadoop/hdds/conf/ConfigurationSource.java | 289 ++++++++
.../org/apache/hadoop/hdds/conf/StorageSize.java | 102 +++
.../org/apache/hadoop/hdds/conf/StorageUnit.java | 529 +++++++++++++
.../apache/hadoop/hdds/conf/TimeDurationUtil.java | 154 ++++
hadoop-hdds/container-service/pom.xml | 35 +-
.../hadoop/hdds/freon/FakeRatisFollower.java | 125 ++++
.../apache/hadoop/hdds/freon}/package-info.java | 8 +-
.../hadoop/ozone/HddsDatanodeHttpServer.java | 4 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 4 +-
.../container/common/helpers/ContainerMetrics.java | 4 +-
.../container/common/impl/ChunkLayOutVersion.java | 19 +-
.../container/common/impl/HddsDispatcher.java | 87 ++-
.../ozone/container/common/interfaces/Handler.java | 8 +-
.../container/common/report/ReportManager.java | 21 +-
.../container/common/report/ReportPublisher.java | 27 +-
.../common/report/ReportPublisherFactory.java | 32 +-
.../common/statemachine/DatanodeStateMachine.java | 51 +-
.../common/statemachine/EndpointStateMachine.java | 6 +-
.../common/statemachine/SCMConnectionManager.java | 63 +-
.../common/statemachine/StateContext.java | 62 +-
.../CreatePipelineCommandHandler.java | 29 +-
.../commandhandler/DeleteBlocksCommandHandler.java | 6 +-
.../ReplicateContainerCommandHandler.java | 6 +-
.../common/states/datanode/InitDatanodeState.java | 32 +-
.../states/datanode/RunningDatanodeState.java | 6 +-
.../states/endpoint/HeartbeatEndpointTask.java | 10 +-
.../states/endpoint/RegisterEndpointTask.java | 10 +-
.../states/endpoint/VersionEndpointTask.java | 23 +-
.../common/transport/server/XceiverServerGrpc.java | 39 +-
.../server/ratis/ContainerStateMachine.java | 110 ++-
.../transport/server/ratis/XceiverServerRatis.java | 128 ++--
.../container/common/utils/ContainerCache.java | 39 +-
.../container/common/utils/ReferenceCountedDB.java | 2 +-
.../ozone/container/common/volume/HddsVolume.java | 25 +-
.../container/common/volume/HddsVolumeChecker.java | 4 +-
.../container/common/volume/MutableVolumeSet.java | 37 +-
.../ozone/container/common/volume/VolumeInfo.java | 17 +-
.../container/keyvalue/KeyValueContainer.java | 49 +-
.../container/keyvalue/KeyValueContainerCheck.java | 6 +-
.../container/keyvalue/KeyValueContainerData.java | 5 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 115 +--
.../container/keyvalue/helpers/BlockUtils.java | 36 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 23 +-
.../container/keyvalue/impl/BlockManagerImpl.java | 6 +-
.../keyvalue/impl/ChunkManagerFactory.java | 4 +-
.../background/BlockDeletingService.java | 81 +-
.../ozone/container/ozoneimpl/ContainerReader.java | 30 +-
.../ozone/container/ozoneimpl/OzoneContainer.java | 9 +-
.../replication/ContainerStreamingOutput.java | 44 --
.../replication/SimpleContainerDownloader.java | 4 +-
...inerDatanodeProtocolServerSideTranslatorPB.java | 3 +-
.../ozone/container/common/ContainerTestUtils.java | 21 +-
.../ozone/container/common/SCMTestUtils.java | 23 +-
.../container/common/TestBlockDeletingService.java | 63 +-
.../ozone/container/common/TestContainerCache.java | 2 -
.../container/common/TestDatanodeStateMachine.java | 60 +-
.../container/common/impl/TestHddsDispatcher.java | 4 +
.../container/common/interfaces/TestHandler.java | 19 +-
.../container/common/report/TestReportManager.java | 11 +-
.../common/report/TestReportPublisher.java | 4 +-
.../common/report/TestReportPublisherFactory.java | 14 +-
.../states/endpoint/TestHeartbeatEndpointTask.java | 52 +-
.../container/common/volume/TestHddsVolume.java | 28 +-
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 22 +-
.../common/volume/TestVolumeSetDiskChecks.java | 16 +-
.../keyvalue/TestKeyValueBlockIterator.java | 43 +-
.../container/keyvalue/TestKeyValueHandler.java | 121 +--
.../testutils/BlockDeletingServiceTestImpl.java | 16 +-
hadoop-hdds/dev-support/checkstyle/checkstyle.xml | 10 +-
hadoop-hdds/docs/archetypes/design.md | 60 ++
hadoop-hdds/docs/content/beyond/Containers.zh.md | 203 +++++
.../Datanodes.zh.md => beyond/_index.zh.md} | 13 +-
hadoop-hdds/docs/content/concept/Datanodes.md | 4 +-
hadoop-hdds/docs/content/concept/Datanodes.zh.md | 30 +-
hadoop-hdds/docs/content/design/configless.md | 31 +
hadoop-hdds/docs/content/design/decommissioning.md | 17 +-
hadoop-hdds/docs/content/design/gdpr.md | 31 +
hadoop-hdds/docs/content/design/locks.md | 29 +
hadoop-hdds/docs/content/design/multiraft.md | 31 +
.../docs/content/design/namespace-support.md | 29 +
hadoop-hdds/docs/content/design/nfs.md | 31 +
hadoop-hdds/docs/content/design/ofs.md | 33 +
hadoop-hdds/docs/content/design/omha.md | 33 +
.../content/design/ozone-enhancement-proposals.md | 2 +-
.../docs/content/design/ozone-volume-management.md | 181 +++++
hadoop-hdds/docs/content/design/recon1.md | 32 +
hadoop-hdds/docs/content/design/recon2.md | 29 +
hadoop-hdds/docs/content/design/s3gateway.md | 31 +
hadoop-hdds/docs/content/design/scmha.md | 29 +
hadoop-hdds/docs/content/design/tde.md | 29 +
hadoop-hdds/docs/content/design/token.md | 25 +
hadoop-hdds/docs/content/design/trash.md | 25 +
hadoop-hdds/docs/content/design/typesafeconfig.md | 33 +
hadoop-hdds/docs/content/interface/OzoneFS.md | 15 +-
hadoop-hdds/docs/content/interface/OzoneFS.zh.md | 12 +-
hadoop-hdds/docs/content/interface/S3.md | 30 +-
hadoop-hdds/docs/content/interface/S3.zh.md | 21 +-
hadoop-hdds/docs/content/recipe/Prometheus.zh.md | 93 +++
hadoop-hdds/docs/content/recipe/SparkOzoneFSK8S.md | 17 +-
.../{tools/SCMCLI.md => recipe/_index.zh.md} | 15 +-
hadoop-hdds/docs/content/shell/VolumeCommands.md | 6 +-
.../docs/content/shell/VolumeCommands.zh.md | 5 +-
.../docs/content/start/StartFromDockerHub.md | 11 +-
hadoop-hdds/docs/content/tools/Admin.md | 35 +
hadoop-hdds/docs/content/tools/AuditParser.zh.md | 68 ++
.../Datanodes.zh.md => tools/Genconf.zh.md} | 11 +-
hadoop-hdds/docs/content/tools/TestTools.md | 8 +-
.../tools/{TestTools.md => TestTools.zh.md} | 53 +-
hadoop-hdds/docs/content/tools/_index.md | 4 +-
hadoop-hdds/docs/pom.xml | 1 -
.../ozonedoc/layouts/_default/baseof.html} | 31 +-
.../themes/ozonedoc/layouts/design/section.html | 68 ++
.../themes/ozonedoc/layouts/design/single.html | 47 ++
.../themes/ozonedoc/layouts/partials/sidebar.html | 1 +
hadoop-hdds/framework/pom.xml | 47 +-
.../hadoop/hdds/conf/DatanodeRatisGrpcConfig.java | 0
.../hdds/conf/DatanodeRatisServerConfig.java | 0
.../apache/hadoop/hdds/conf/HddsConfServlet.java | 30 +-
.../org/apache/hadoop/hdds/conf/package-info.java | 0
.../certificates/utils/CertificateSignRequest.java | 23 +-
.../certificates/utils/SelfSignedCertificate.java | 27 +-
.../hdds/security/x509/keys/HDDSKeyGenerator.java | 13 +-
.../org/apache/hadoop/hdds/server}/JsonUtils.java | 8 +-
.../server/OzoneProtocolMessageDispatcher.java | 18 +-
.../org/apache/hadoop/hdds/server/ServerUtils.java | 12 +-
.../hadoop/hdds/server/events/EventQueue.java | 21 +-
.../hadoop/hdds/server/http/BaseHttpServer.java | 56 +-
.../hadoop/hdds/server/http/FilterInitializer.java | 4 +-
.../apache/hadoop/hdds/server/http/HttpConfig.java | 4 +-
.../hadoop/hdds/server/http/HttpServer2.java | 98 +--
.../hadoop/hdds/server/http/LogStreamServlet.java | 5 +-
.../hadoop/hdds/server/http/PrometheusServlet.java | 15 +
.../server/http/RatisNameRewriteSampleBuilder.java | 17 +-
.../hdds/server/http/StaticUserWebFilter.java | 6 +-
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 39 +-
.../hadoop/hdds/utils/MetadataStoreBuilder.java | 13 +-
.../hadoop/hdds/utils/ProtocolMessageMetrics.java | 42 +-
.../hdds/utils/db/BatchOperationHandler.java | 24 +-
.../hdds/utils/db/DBColumnFamilyDefinition.java | 81 ++
.../apache/hadoop/hdds/utils/db/DBDefinition.java | 27 +-
.../org/apache/hadoop/hdds/utils/db/DBProfile.java | 1 -
.../org/apache/hadoop/hdds/utils/db/DBStore.java | 18 +-
.../hadoop/hdds/utils/db/DBStoreBuilder.java | 56 +-
.../apache/hadoop/hdds/utils/db/RDBMetrics.java | 13 +-
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 2 +-
.../apache/hadoop/hdds/server/TestJsonUtils.java | 37 +-
.../apache/hadoop/hdds/server/TestServerUtils.java | 21 +-
.../hdds/server/http/TestBaseHttpServer.java | 4 +-
.../hdds/server/http/TestRatisNameRewrite.java | 10 +
.../hadoop/hdds/utils/TestMetadataStore.java | 13 +-
.../hadoop/hdds/utils/TestRocksDBStoreMBean.java | 23 +-
hadoop-hdds/hadoop-dependency-client/README.md | 60 ++
hadoop-hdds/hadoop-dependency-client/pom.xml | 282 +++++++
.../pom.xml | 66 +-
.../{client => hadoop-dependency-test}/pom.xml | 22 +-
hadoop-hdds/pom.xml | 57 +-
hadoop-hdds/server-scm/pom.xml | 28 +-
.../hadoop/hdds/scm/SCMCommonPlacementPolicy.java | 35 +-
.../apache/hadoop/hdds/scm/block/BlockManager.java | 16 +-
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 43 +-
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 34 +-
.../hdds/scm/block/SCMBlockDeletingService.java | 41 +-
.../hdds/scm/container/ContainerManager.java | 11 +-
.../hdds/scm/container/ContainerReportHandler.java | 57 +-
.../hdds/scm/container/ContainerStateManager.java | 61 +-
.../hdds/scm/container/ReplicationManager.java | 23 +-
.../hdds/scm/container/SCMContainerManager.java | 138 ++--
.../ContainerPlacementPolicyFactory.java | 11 +-
.../algorithms/SCMContainerPlacementCapacity.java | 4 +-
.../algorithms/SCMContainerPlacementRackAware.java | 4 +-
.../algorithms/SCMContainerPlacementRandom.java | 4 +-
.../container/placement/metrics/ContainerStat.java | 2 +-
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 33 +-
.../hadoop/hdds/scm/metadata/ContainerIDCodec.java | 48 ++
.../hdds/scm/metadata/ContainerInfoCodec.java | 47 ++
.../hadoop/hdds/scm/metadata/PipelineCodec.java | 56 ++
.../hadoop/hdds/scm/metadata/PipelineIDCodec.java | 25 +-
.../hadoop/hdds/scm/metadata/SCMDBDefinition.java | 98 +++
.../hadoop/hdds/scm/metadata/SCMMetadataStore.java | 27 +-
.../hdds/scm/metadata/SCMMetadataStoreRDBImpl.java | 113 +--
.../hadoop/hdds/scm/node/NewNodeHandler.java | 15 +-
.../hadoop/hdds/scm/node/NodeStateManager.java | 55 +-
.../scm/node/NonHealthyToHealthyNodeHandler.java | 4 +-
.../hadoop/hdds/scm/node/SCMNodeMetrics.java | 17 +-
.../hadoop/hdds/scm/node/StaleNodeHandler.java | 4 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 15 +-
.../hdds/scm/pipeline/PipelineActionHandler.java | 63 +-
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 6 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 15 +-
.../hdds/scm/pipeline/PipelinePlacementPolicy.java | 140 ++--
.../hadoop/hdds/scm/pipeline/PipelineProvider.java | 75 +-
.../hdds/scm/pipeline/PipelineReportHandler.java | 6 +-
.../hdds/scm/pipeline/RatisPipelineProvider.java | 76 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 11 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 181 +++--
.../hdds/scm/pipeline/SCMPipelineMetrics.java | 12 +-
.../hdds/scm/pipeline/SimplePipelineProvider.java | 15 +-
...lockLocationProtocolServerSideTranslatorPB.java | 4 +-
...inerLocationProtocolServerSideTranslatorPB.java | 27 +-
.../hdds/scm/safemode/ContainerSafeModeRule.java | 4 +-
.../hdds/scm/safemode/DataNodeSafeModeRule.java | 4 +-
.../scm/safemode/HealthyPipelineSafeModeRule.java | 11 +-
.../safemode/OneReplicaPipelineSafeModeRule.java | 4 +-
.../hdds/scm/safemode/SCMSafeModeManager.java | 90 ++-
.../hadoop/hdds/scm/safemode/SafeModeHandler.java | 126 ----
.../hdds/scm/safemode/SafeModeNotification.java | 25 -
.../hadoop/hdds/scm/safemode/SafeModePrecheck.java | 4 +-
.../hdds/scm/server/SCMBlockProtocolServer.java | 6 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 193 +++--
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 21 +-
.../hdds/scm/server/StorageContainerManager.java | 96 +--
.../server/StorageContainerManagerHttpServer.java | 8 +-
.../hdds/scm/server/ratis/SCMRatisServer.java | 44 +-
.../hdds/scm/server/ratis/SCMStateMachine.java | 8 +-
.../apache/hadoop/hdds/scm/TestHddsServerUtil.java | 18 +-
.../hadoop/hdds/scm/TestHddsServerUtils.java | 42 +-
.../scm/TestStorageContainerManagerHttpServer.java | 6 +-
.../hadoop/hdds/scm/block/TestBlockManager.java | 42 +-
.../container/TestCloseContainerEventHandler.java | 38 +-
.../scm/container/TestContainerReportHandler.java | 4 +-
.../TestIncrementalContainerReportHandler.java | 4 +-
.../hdds/scm/container/TestReplicationManager.java | 34 +-
.../scm/container/TestSCMContainerManager.java | 65 +-
.../scm/container/TestUnknownContainerReport.java | 145 ++++
.../algorithms/TestContainerPlacementFactory.java | 19 +-
.../TestSCMContainerPlacementCapacity.java | 4 +-
.../TestSCMContainerPlacementRackAware.java | 6 +-
.../TestSCMContainerPlacementRandom.java | 4 +-
.../hdds/scm/node/TestContainerPlacement.java | 65 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 14 +-
.../scm/pipeline/MockRatisPipelineProvider.java | 20 +-
.../scm/pipeline/TestPipelineActionHandler.java | 69 ++
.../scm/pipeline/TestPipelinePlacementPolicy.java | 93 ++-
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 178 ++++-
.../scm/pipeline/TestSimplePipelineProvider.java | 2 +-
.../safemode/TestHealthyPipelineSafeModeRule.java | 38 +-
.../TestOneReplicaPipelineSafeModeRule.java | 19 +-
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 234 +++++-
.../hdds/scm/safemode/TestSafeModeHandler.java | 118 ---
.../scm/server/TestSCMClientProtocolServer.java | 79 --
.../ozone/container/common/TestEndPoint.java | 7 +-
.../placement/TestContainerPlacement.java | 32 +-
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 204 ++++++
.../upgrade => hadoop-hdds/test-utils}/pom.xml | 45 +-
.../org/apache/hadoop/test/GenericTestUtils.java | 365 +++++++++
.../org/apache/hadoop/test/LambdaTestUtils.java | 814 +++++++++++++++++++++
.../apache/hadoop/test/TimedOutTestsListener.java | 183 +++++
.../java/org/apache/hadoop/test}/package-info.java | 7 +-
hadoop-hdds/tools/pom.xml | 10 +-
.../hdds/scm/cli/ContainerOperationClient.java | 26 +-
.../hdds/scm/cli/ReplicationManagerCommands.java | 21 +-
.../org/apache/hadoop/hdds/scm/cli/SCMCLI.java | 114 ---
.../hadoop/hdds/scm/cli/SafeModeCommands.java | 16 +-
.../hadoop/hdds/scm/cli/TopologySubcommand.java | 26 +-
.../hdds/scm/cli/container/CloseSubcommand.java | 4 +-
.../hdds/scm/cli/container/ContainerCommands.java | 32 +-
.../hdds/scm/cli/container/DeleteSubcommand.java | 3 +-
.../hdds/scm/cli/container/InfoSubcommand.java | 20 +-
.../hdds/scm/cli/container/ListSubcommand.java | 21 +-
.../hdds/scm/cli/container/WithScmClient.java | 13 +-
.../hdds/scm/cli/datanode/DatanodeCommands.java | 22 +-
.../hdds/scm/cli/pipeline/PipelineCommands.java | 22 +-
hadoop-ozone/Jenkinsfile | 116 ---
hadoop-ozone/client/pom.xml | 5 +
.../apache/hadoop/ozone/client/ObjectStore.java | 174 +----
.../apache/hadoop/ozone/client/OzoneBucket.java | 8 +-
.../apache/hadoop/ozone/client/OzoneClient.java | 4 +-
.../hadoop/ozone/client/OzoneClientFactory.java | 69 +-
.../apache/hadoop/ozone/client/OzoneVolume.java | 18 +-
.../ozone/client/io/BlockOutputStreamEntry.java | 36 +-
.../client/io/BlockOutputStreamEntryPool.java | 8 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 61 ++
.../hadoop/ozone/client/io/KeyOutputStream.java | 16 +-
.../ozone/client/protocol/ClientProtocol.java | 61 +-
.../hadoop/ozone/client/rpc/OzoneKMSUtil.java | 32 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 220 +++---
.../hadoop/ozone/client/TestHddsClientUtils.java | 41 +-
hadoop-ozone/common/pom.xml | 26 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 158 ++--
.../apache/hadoop/ozone/conf/OMClientConfig.java | 67 ++
.../apache/hadoop/ozone/conf}/package-info.java | 5 +-
.../apache/hadoop/ozone/freon/OzoneGetConf.java | 27 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 3 +
.../hadoop/ozone/om/exceptions/OMException.java | 8 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 113 ++-
.../apache/hadoop/ozone/om/helpers/DBUpdates.java | 58 ++
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 2 +
.../hadoop/ozone/om/helpers/OzoneFileStatus.java | 116 ++-
.../hadoop/ozone/om/helpers/ServiceInfo.java | 24 +-
.../hadoop/ozone/om/lock/OzoneManagerLock.java | 4 +-
.../ozone/om/protocol/OzoneManagerProtocol.java | 77 +-
...OzoneManagerProtocolClientSideTranslatorPB.java | 223 ++----
.../ozone/security/OzoneTokenIdentifier.java | 22 +-
.../apache/hadoop/ozone/web/utils/OzoneUtils.java | 7 +-
.../common/src/main/proto/FSProtos.proto | 2 +-
.../src/main/proto/OzoneManagerProtocol.proto | 69 +-
.../java/org/apache/hadoop/ozone/TestOmUtils.java | 102 +--
.../ozone/om/ha/TestOMFailoverProxyProvider.java | 195 +++++
hadoop-ozone/csi/pom.xml | 12 +-
.../apache/hadoop/ozone/csi/ControllerService.java | 11 +-
.../org/apache/hadoop/ozone/csi/CsiServer.java | 7 +-
hadoop-ozone/datanode/pom.xml | 2 +-
hadoop-ozone/dev-support/checks/author.sh | 6 +-
hadoop-ozone/dev-support/checks/checkstyle.sh | 14 +-
.../intellij/runConfigurations/OzoneShell.xml | 6 +-
hadoop-ozone/dist/pom.xml | 17 +-
.../main/compose/ozone-hdfs/docker-compose.yaml | 1 +
.../dist/src/main/compose/ozone-mr/common-config | 2 +-
.../compose/ozone-mr/hadoop27/docker-compose.yaml | 1 +
.../compose/ozone-mr/hadoop31/docker-compose.yaml | 1 +
.../src/main/compose/ozone-mr/hadoop31/test.sh | 6 +-
.../compose/ozone-mr/hadoop32/docker-compose.yaml | 1 +
.../dist/src/main/compose/ozone-om-ha-s3/test.sh | 2 +-
.../dist/src/main/compose/ozone-om-ha/.ssh/id_rsa | 15 -
.../src/main/compose/ozone-om-ha/.ssh/id_rsa.pub | 15 -
.../src/main/compose/ozone-om-ha/docker-config | 3 +-
.../ozone-om-ha/{run.sh => test_disabled.sh} | 1 +
.../compose/ozone-topology/docker-compose.yaml | 3 +-
.../src/main/compose/ozone-topology/docker-config | 5 +-
.../src/main/compose/ozone-topology/hdds-3084.sh | 56 --
.../dist/src/main/compose/ozone-topology/test.sh | 20 +-
.../src/main/compose/ozone/docker-compose.yaml | 1 +
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 2 +
.../main/compose/ozoneblockade/docker-compose.yaml | 1 +
.../compose/ozones3-haproxy/docker-compose.yaml | 1 +
.../main/compose/ozonescripts/docker-compose.yaml | 1 +
.../dist/src/main/compose/ozonesecure-mr/README.md | 12 +-
.../compose/ozonesecure-mr/docker-compose.yaml | 1 +
.../src/main/compose/ozonesecure-mr/docker-config | 2 +-
.../main/compose/ozonesecure/docker-compose.yaml | 1 +
.../src/main/compose/ozonesecure/docker-config | 3 +-
.../dist/src/main/compose/ozonesecure/test.sh | 2 +-
hadoop-ozone/dist/src/main/compose/test-all.sh | 2 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 29 +-
hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 8 -
.../licenses/LICENSE-org.openjdk.jmh-jmh-core.txt | 347 ---------
...SE-org.openjdk.jmh-jmh-generator-annprocess.txt | 347 ---------
.../smoketest/{scmcli => admincli}/datanode.robot | 2 +-
.../smoketest/{scmcli => admincli}/pipeline.robot | 4 +-
.../src/main/smoketest/basic/ozone-shell.robot | 10 +-
.../dist/src/main/smoketest/commonlib.robot | 1 +
.../dist/src/main/smoketest/createbucketenv.robot | 2 +-
.../dist/src/main/smoketest/createmrenv.robot | 4 +-
.../dist/src/main/smoketest/mapreduce.robot | 4 +-
.../dist/src/main/smoketest/omha/testOMHA.robot | 46 +-
.../src/main/smoketest/ozonefs/hadoopo3fs.robot | 4 +-
.../dist/src/main/smoketest/ozonefs/ozonefs.robot | 4 +
.../dist/src/main/smoketest/recon/recon-api.robot | 3 +
.../dist/src/main/smoketest/s3/bucketcreate.robot | 8 +-
.../s3/{bucketcreate.robot => bucketdelete.robot} | 12 +-
.../dist/src/main/smoketest/s3/commonawslib.robot | 10 +-
.../smoketest/security/ozone-secure-token.robot | 76 ++
.../smoketest/topology/{scmcli.robot => cli.robot} | 5 +-
.../src/main/smoketest/topology/loaddata.robot | 1 +
.../src/main/smoketest/topology/readdata.robot | 1 +
.../src/shell/conf/ozone-shell-log4j.properties | 1 +
.../dist/src/shell/hdds/hadoop-functions.sh | 63 ++
hadoop-ozone/dist/src/shell/ozone/ozone | 68 +-
hadoop-ozone/dist/src/test/shell/gc_opts.bats | 44 ++
.../fault-injection-test/mini-chaos-tests/pom.xml | 10 +-
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 252 +++++--
.../ozone/MiniOzoneDatanodeChaosCluster.java | 57 ++
.../hadoop/ozone/MiniOzoneLoadGenerator.java | 106 ++-
.../hadoop/ozone/MiniOzoneOMChaosCluster.java | 132 ++++
.../hadoop/ozone/TestMiniChaosOzoneCluster.java | 66 +-
.../ozone/loadgenerators/AgedLoadGenerator.java | 20 +-
.../hadoop/ozone/loadgenerators/DataBuffer.java | 1 +
.../loadgenerators/FilesystemLoadGenerator.java | 15 +-
.../hadoop/ozone/loadgenerators/LoadExecutors.java | 39 +-
.../hadoop/ozone/loadgenerators/LoadGenerator.java | 23 +-
.../ozone/loadgenerators/RandomLoadGenerator.java | 25 +-
...adGenerator.java => ReadOnlyLoadGenerator.java} | 47 +-
.../org/apache/hadoop/ozone/utils/LoadBucket.java | 17 +-
.../src/test/resources/log4j.properties | 2 +-
.../fault-injection-test/network-tests/pom.xml | 1 -
hadoop-ozone/insight/pom.xml | 27 +-
.../apache/hadoop/ozone/insight/LogSubcommand.java | 2 +
hadoop-ozone/integration-test/pom.xml | 17 +-
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 2 +
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 112 ++-
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 7 -
.../ozone/contract/ITestOzoneContractCreate.java | 5 +-
.../ozone/contract/ITestOzoneContractDelete.java | 5 +-
.../ozone/contract/ITestOzoneContractDistCp.java | 5 +-
.../contract/ITestOzoneContractGetFileStatus.java | 5 +-
.../fs/ozone/contract/ITestOzoneContractMkdir.java | 5 +-
.../fs/ozone/contract/ITestOzoneContractOpen.java | 5 +-
.../ozone/contract/ITestOzoneContractRename.java | 5 +-
.../ozone/contract/ITestOzoneContractRootDir.java | 6 +-
.../fs/ozone/contract/ITestOzoneContractSeek.java | 5 +-
.../hadoop/hdds/scm/pipeline/TestNodeFailure.java | 110 ++-
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 33 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 24 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 158 +++-
.../org/apache/hadoop/ozone/RatisTestHelper.java | 3 +-
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 45 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 2 +-
.../ozone/TestStorageContainerManagerHelper.java | 10 +-
.../ozone/client/CertificateClientTestImpl.java | 24 +-
.../ozone/client/rpc/Test2BlockOutputStream.java | 173 +++++
.../client/rpc/TestDeleteWithSlowFollower.java | 24 +-
.../ozone/client/rpc/TestKeyInputStream.java | 62 ++
.../client/rpc/TestOzoneAtRestEncryption.java | 23 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 117 +--
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 2 +
.../ozone/dn/ratis/TestDnRatisLogParser.java | 21 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 39 +-
.../hadoop/ozone/om/TestOMDbCheckpointServlet.java | 116 ++-
.../org/apache/hadoop/ozone/om/TestOmMetrics.java | 46 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 58 +-
.../ozone/om/TestOzoneManagerListVolumes.java | 238 ++++++
.../apache/hadoop/ozone/om/TestScmSafeMode.java | 2 +-
.../ozone/om/parser/TestOMRatisLogParser.java | 13 +-
.../apache/hadoop/ozone/ozShell/TestS3Shell.java | 292 --------
.../hadoop/ozone/recon/TestReconAsPassiveScm.java | 8 +-
.../apache/hadoop/ozone/recon/TestReconTasks.java | 125 ++++
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 177 -----
.../TestSCMPipelineBytesWrittenMetrics.java | 2 +-
.../ozone/scm/pipeline/TestSCMPipelineMetrics.java | 21 +-
.../{ozShell => shell}/TestOzoneDatanodeShell.java | 2 +-
.../ozone/{ozShell => shell}/TestOzoneShellHA.java | 33 +-
hadoop-ozone/native-client/README.md | 96 +++
.../native-client/libo3fs-examples/libo3fs_read.c | 65 ++
.../native-client/libo3fs-examples/libo3fs_write.c | 91 +++
hadoop-ozone/native-client/libo3fs/o3fs.c | 66 ++
.../native-client/libo3fs/o3fs.h | 33 +-
hadoop-ozone/ozone-manager/pom.xml | 14 +-
.../apache/hadoop/ozone/om/KeyDeletingService.java | 4 +-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 70 +-
.../hadoop/ozone/om/OMDBCheckpointServlet.java | 91 ++-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 12 +
.../java/org/apache/hadoop/ozone/om/OMStorage.java | 4 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 14 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 305 ++++----
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 10 +-
.../apache/hadoop/ozone/om/S3BucketManager.java | 87 ---
.../hadoop/ozone/om/S3BucketManagerImpl.java | 254 -------
.../hadoop/ozone/om/S3SecretManagerImpl.java | 0
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 22 +-
.../hadoop/ozone/om/codec/OmBucketInfoCodec.java | 0
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 0
.../ozone/om/codec/OmMultipartKeyInfoCodec.java | 0
.../hadoop/ozone/om/codec/OmPrefixInfoCodec.java | 0
.../hadoop/ozone/om/codec/OmVolumeArgsCodec.java | 0
.../ozone/om/codec/RepeatedOmKeyInfoCodec.java | 0
.../hadoop/ozone/om/codec/S3SecretValueCodec.java | 0
.../ozone/om/codec/TokenIdentifierCodec.java | 0
.../hadoop/ozone/om/codec/UserVolumeInfoCodec.java | 0
.../apache/hadoop/ozone/om/codec/package-info.java | 0
.../ozone/om/ratis/OzoneManagerRatisServer.java | 37 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 8 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 9 +-
.../om/request/bucket/OMBucketCreateRequest.java | 4 +
.../om/request/file/OMDirectoryCreateRequest.java | 42 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 7 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 7 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 80 +-
.../om/request/key/OMTrashRecoverRequest.java | 136 ++++
.../request/s3/bucket/S3BucketCreateRequest.java | 412 -----------
.../request/s3/bucket/S3BucketDeleteRequest.java | 217 ------
.../om/request/volume/OMVolumeCreateRequest.java | 20 +-
.../ozone/om/request/volume/OMVolumeRequest.java | 21 +-
.../om/request/volume/OMVolumeSetOwnerRequest.java | 26 +-
.../om/request/volume/OMVolumeSetQuotaRequest.java | 2 +-
.../om/response/key/OMTrashRecoverResponse.java | 64 ++
.../response/s3/bucket/S3BucketCreateResponse.java | 86 ---
.../response/s3/bucket/S3BucketDeleteResponse.java | 61 --
.../response/volume/OMVolumeSetOwnerResponse.java | 21 +-
.../om/snapshot/OzoneManagerSnapshotProvider.java | 6 +-
...OzoneManagerProtocolServerSideTranslatorPB.java | 13 +-
.../protocolPB/OzoneManagerRequestHandler.java | 65 +-
.../hadoop/ozone/security/AWSV4AuthValidator.java | 0
.../security/OzoneBlockTokenSecretManager.java | 0
.../OzoneDelegationTokenSecretManager.java | 99 ++-
.../hadoop/ozone/security/OzoneSecretKey.java | 0
.../hadoop/ozone/security/OzoneSecretManager.java | 0
.../hadoop/ozone/security/OzoneSecretStore.java | 0
.../hadoop/ozone/security}/package-info.java | 7 +-
.../web/ozShell/bucket/AddAclBucketHandler.java | 105 ---
.../web/ozShell/bucket/CreateBucketHandler.java | 107 ---
.../web/ozShell/bucket/DeleteBucketHandler.java | 64 --
.../web/ozShell/bucket/GetAclBucketHandler.java | 85 ---
.../web/ozShell/bucket/InfoBucketHandler.java | 68 --
.../web/ozShell/bucket/ListBucketHandler.java | 105 ---
.../web/ozShell/bucket/RemoveAclBucketHandler.java | 106 ---
.../web/ozShell/bucket/SetAclBucketHandler.java | 100 ---
.../ozone/web/ozShell/keys/AddAclKeyHandler.java | 109 ---
.../ozone/web/ozShell/keys/DeleteKeyHandler.java | 70 --
.../ozone/web/ozShell/keys/GetAclKeyHandler.java | 89 ---
.../ozone/web/ozShell/keys/GetKeyHandler.java | 113 ---
.../ozone/web/ozShell/keys/InfoKeyHandler.java | 78 --
.../ozone/web/ozShell/keys/ListKeyHandler.java | 113 ---
.../ozone/web/ozShell/keys/PutKeyHandler.java | 129 ----
.../web/ozShell/keys/RemoveAclKeyHandler.java | 109 ---
.../ozone/web/ozShell/keys/SetAclKeyHandler.java | 102 ---
.../ozone/web/ozShell/s3/GetS3SecretHandler.java | 63 --
.../ozone/web/ozShell/s3/S3BucketMapping.java | 68 --
.../web/ozShell/token/CancelTokenHandler.java | 76 --
.../ozone/web/ozShell/token/GetTokenHandler.java | 81 --
.../ozone/web/ozShell/token/PrintTokenHandler.java | 71 --
.../ozone/web/ozShell/token/RenewTokenHandler.java | 79 --
.../web/ozShell/volume/AddAclVolumeHandler.java | 103 ---
.../web/ozShell/volume/CreateVolumeHandler.java | 103 ---
.../web/ozShell/volume/DeleteVolumeHandler.java | 62 --
.../web/ozShell/volume/GetAclVolumeHandler.java | 80 --
.../web/ozShell/volume/ListVolumeHandler.java | 111 ---
.../web/ozShell/volume/RemoveAclVolumeHandler.java | 103 ---
.../web/ozShell/volume/SetAclVolumeHandler.java | 106 ---
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 13 +-
.../org/apache/hadoop/ozone/om/TestOMStorage.java | 9 +-
.../hadoop/ozone/om/TestOmMetadataManager.java | 26 +-
.../ozone/om/TestOzoneManagerHttpServer.java | 21 +-
.../hadoop/ozone/om/TestS3BucketManager.java | 115 ---
.../apache/hadoop/ozone/om/TestTrashService.java | 13 +-
.../om/codec/TestOmMultipartKeyInfoCodec.java | 0
.../ozone/om/codec/TestOmPrefixInfoCodec.java | 0
.../ozone/om/codec/TestS3SecretValueCodec.java | 0
.../apache/hadoop/ozone/om/codec/package-info.java | 0
.../ozone/om/request/TestOMRequestUtils.java | 34 -
.../request/bucket/TestOMBucketCreateRequest.java | 5 +
.../request/file/TestOMDirectoryCreateRequest.java | 2 +-
.../s3/bucket/TestS3BucketCreateRequest.java | 216 ------
.../s3/bucket/TestS3BucketDeleteRequest.java | 161 ----
.../TestOMDelegationTokenRequest.java} | 57 +-
.../security/TestOMGetDelegationTokenRequest.java | 221 ++++++
.../bucket => request/security}/package-info.java | 4 +-
.../request/volume/TestOMVolumeCreateRequest.java | 5 +
.../volume/TestOMVolumeSetOwnerRequest.java | 49 ++
.../ozone/om/response/TestOMResponseUtils.java | 43 --
.../s3/bucket/TestS3BucketCreateResponse.java | 91 ---
.../s3/bucket/TestS3BucketDeleteResponse.java | 92 ---
.../security/TestOMDelegationTokenResponse.java | 51 ++
.../security/TestOMGetDelegationTokenResponse.java | 95 +++
.../response/security}/package-info.java | 4 +-
.../ozone/security/TestAWSV4AuthValidator.java | 0
.../TestOzoneDelegationTokenSecretManager.java | 14 +-
.../ozone/security/TestOzoneTokenIdentifier.java | 41 +-
hadoop-ozone/ozonefs-lib-current/pom.xml | 13 +-
hadoop-ozone/ozonefs/pom.xml | 37 +-
.../java/org/apache/hadoop/fs/ozone/BasicOzFs.java | 16 +-
.../fs/ozone/BasicOzoneClientAdapterImpl.java | 13 +-
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 20 +-
.../org/apache/hadoop/fs/ozone/O3fsDtFetcher.java | 8 +-
.../main/java/org/apache/hadoop/fs/ozone/OzFs.java | 12 +-
.../hadoop/fs/ozone/OzoneClientAdapterImpl.java | 5 +-
.../apache/hadoop/fs/ozone/OzoneFileSystem.java | 10 +-
.../org/apache/hadoop/fs/ozone/OzoneFsShell.java | 5 +-
.../fs/ozone/TestOzoneFileSystemWithMocks.java | 31 +-
.../hadoop/fs/ozone/TestReadWriteStatistics.java | 41 +-
hadoop-ozone/pom.xml | 50 +-
hadoop-ozone/recon-codegen/pom.xml | 13 +-
.../ozone/recon/codegen/JooqCodeGenerator.java | 53 +-
.../recon/codegen/ReconSchemaGenerationModule.java | 2 +
.../ozone/recon/codegen/ReconSqlDbConfig.java | 239 ++++++
.../org/hadoop/ozone/recon/codegen/SqlDbUtils.java | 97 +++
.../recon/schema/ContainerSchemaDefinition.java | 93 +++
.../recon/schema/ReconTaskSchemaDefinition.java | 12 +-
.../ozone/recon/schema/StatsSchemaDefinition.java | 10 +-
.../recon/schema/UtilizationSchemaDefinition.java | 40 +-
hadoop-ozone/recon/pom.xml | 79 +-
.../hadoop/ozone/recon/ConfigurationProvider.java | 5 +-
.../hadoop/ozone/recon/ReconControllerModule.java | 104 +--
.../hadoop/ozone/recon/ReconSchemaManager.java | 4 +-
.../org/apache/hadoop/ozone/recon/ReconServer.java | 10 +-
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 24 -
.../org/apache/hadoop/ozone/recon/ReconUtils.java | 54 +-
.../ozone/recon/api/ClusterStateEndpoint.java | 36 +-
.../hadoop/ozone/recon/api/ContainerEndpoint.java | 29 +-
.../hadoop/ozone/recon/api/types/KeysResponse.java | 61 +-
.../recon/api/types/MissingContainerMetadata.java | 14 +-
.../ozone/recon/fsck/MissingContainerTask.java | 44 +-
.../recon/metrics/OzoneManagerSyncMetrics.java | 133 ++++
.../hadoop/ozone/recon/metrics}/package-info.java | 16 +-
.../recon/persistence/ContainerSchemaManager.java | 104 +++
.../recon/persistence/DataSourceConfiguration.java | 6 +-
.../persistence/DefaultDataSourceProvider.java | 11 +-
.../recon/persistence/DerbyDataSourceProvider.java | 61 ++
.../recon/persistence/JooqPersistenceModule.java | 2 +-
...Provider.java => SqliteDataSourceProvider.java} | 44 +-
.../recon/recovery/ReconOMMetadataManager.java | 6 +
.../recon/recovery/ReconOmMetadataManagerImpl.java | 10 +
.../hadoop/ozone/recon/scm/PipelineSyncTask.java | 21 +-
.../ozone/recon/scm/ReconContainerManager.java | 59 +-
.../hadoop/ozone/recon/scm/ReconDBDefinition.java} | 18 +-
.../recon/scm/ReconDatanodeProtocolServer.java | 4 +-
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 17 +-
.../ozone/recon/scm/ReconPipelineFactory.java | 4 +-
.../ozone/recon/scm/ReconPipelineManager.java | 34 +-
.../recon/scm/ReconPipelineReportHandler.java | 4 +-
.../hadoop/ozone/recon/scm/ReconScmTask.java | 8 +-
.../scm/ReconStorageContainerManagerFacade.java | 56 +-
.../spi/impl/ContainerDBServiceProviderImpl.java | 10 +-
.../spi/impl/OzoneManagerServiceProviderImpl.java | 76 +-
.../hadoop/ozone/recon/tasks/ReconTaskConfig.java | 69 ++
.../webapps/recon/ozone-recon-web/api/db.json | 74 +-
.../webapps/recon/ozone-recon-web/api/routes.json | 3 +-
.../recon/ozone-recon-web/src/utils/common.tsx | 9 +-
.../views/MissingContainers/MissingContainers.less | 4 +
.../views/MissingContainers/MissingContainers.tsx | 99 ++-
.../src/views/Overview/Overview.tsx | 12 +-
...erTest.java => OMMetadataManagerTestUtils.java} | 60 +-
.../hadoop/ozone/recon/ReconTestInjector.java | 337 +++++++++
.../apache/hadoop/ozone/recon/TestReconUtils.java | 45 +-
.../ozone/recon/api/TestContainerEndpoint.java | 170 ++---
.../hadoop/ozone/recon/api/TestEndpoints.java | 141 ++--
.../ozone/recon/api/TestTaskStatusService.java | 23 +-
.../ozone/recon/fsck/TestMissingContainerTask.java | 46 +-
.../recon/persistence/AbstractReconSqlDBTest.java | 234 ++++++
.../recon/persistence/AbstractSqlDatabaseTest.java | 161 ----
.../TestReconInternalSchemaDefinition.java | 30 +-
.../persistence/TestReconWithDifferentSqlDBs.java | 159 ++++
.../recon/persistence/TestSqlSchemaSetup.java | 53 ++
.../persistence/TestStatsSchemaDefinition.java | 29 +-
.../TestUtilizationSchemaDefinition.java | 47 +-
.../scm/AbstractReconContainerManagerTest.java | 38 +-
.../ozone/recon/scm/TestReconContainerManager.java | 2 +-
.../ozone/recon/scm/TestReconPipelineManager.java | 37 +-
.../recon/scm/TestReconPipelineReportHandler.java | 2 +-
.../impl/TestContainerDBServiceProviderImpl.java | 56 +-
.../impl/TestOzoneManagerServiceProviderImpl.java | 64 +-
.../recon/tasks/TestContainerKeyMapperTask.java | 71 +-
.../ozone/recon/tasks/TestFileSizeCountTask.java | 16 +-
.../recon/tasks/TestReconTaskControllerImpl.java | 28 +-
.../recon/types/GuiceInjectorUtilsForTests.java | 136 ----
hadoop-ozone/s3gateway/pom.xml | 53 +-
.../java/org/apache/hadoop/ozone/s3/Gateway.java | 2 +
.../hadoop/ozone/s3/S3GatewayHttpServer.java | 4 +-
.../org/apache/hadoop/ozone/s3/TracingFilter.java | 82 +++
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 5 +-
.../endpoint/CompleteMultipartUploadRequest.java | 3 +-
...CompleteMultipartUploadRequestUnmarshaller.java | 84 +++
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 111 +--
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 15 +-
.../hadoop/ozone/s3/endpoint/RootEndpoint.java | 8 +-
.../s3/exception/BadRequestExceptionMapper.java | 38 +-
.../hadoop/ozone/s3/io/S3WrapperInputStream.java | 31 +
.../org/apache/hadoop/ozone/s3/util/S3Consts.java | 3 +
.../hadoop/ozone/client/ObjectStoreStub.java | 113 +--
.../hadoop/ozone/client/OzoneClientStub.java | 5 +-
.../hadoop/ozone/client/OzoneVolumeStub.java | 10 +-
.../s3/endpoint/TestAbortMultipartUpload.java | 7 +-
.../hadoop/ozone/s3/endpoint/TestBucketDelete.java | 5 +-
.../hadoop/ozone/s3/endpoint/TestBucketGet.java | 33 +-
.../hadoop/ozone/s3/endpoint/TestBucketHead.java | 12 +-
.../hadoop/ozone/s3/endpoint/TestBucketPut.java | 3 +-
...CompleteMultipartUploadRequestUnmarshaller.java | 93 +++
.../s3/endpoint/TestInitiateMultipartUpload.java | 15 +-
.../hadoop/ozone/s3/endpoint/TestListParts.java | 15 +-
.../TestMultiDeleteRequestUnmarshaller.java | 1 -
.../s3/endpoint/TestMultipartUploadComplete.java | 24 +-
.../s3/endpoint/TestMultipartUploadWithCopy.java | 52 +-
.../hadoop/ozone/s3/endpoint/TestObjectDelete.java | 8 +-
.../hadoop/ozone/s3/endpoint/TestObjectGet.java | 18 +-
.../hadoop/ozone/s3/endpoint/TestObjectHead.java | 12 +-
.../ozone/s3/endpoint/TestObjectMultiDelete.java | 11 +-
.../hadoop/ozone/s3/endpoint/TestObjectPut.java | 52 +-
.../hadoop/ozone/s3/endpoint/TestPartUpload.java | 10 +-
.../hadoop/ozone/s3/endpoint/TestRootList.java | 13 +-
hadoop-ozone/tools/pom.xml | 29 +-
.../org/apache/hadoop/ozone/admin/OzoneAdmin.java | 58 +-
.../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 12 +-
.../hadoop/ozone/debug/ChunkDataNodeDetails.java} | 31 +-
.../apache/hadoop/ozone/debug/ChunkDetails.java} | 50 +-
.../apache/hadoop/ozone/debug/ChunkKeyHandler.java | 171 +++++
.../hadoop/ozone/debug/ContainerChunkInfo.java | 93 +++
.../org/apache/hadoop/ozone/debug/OzoneDebug.java | 49 +-
.../apache/hadoop/ozone/debug}/package-info.java | 7 +-
.../hadoop/ozone/freon/BaseAppendLogGenerator.java | 10 +
.../hadoop/ozone/freon/BaseFreonGenerator.java | 17 +-
.../freon/FollowerAppendLogEntryGenerator.java | 8 -
.../java/org/apache/hadoop/ozone/freon/Freon.java | 4 +-
.../apache/hadoop/ozone/freon/FreonHttpServer.java | 4 +-
.../ozone/freon/HadoopNestedDirGenerator.java | 113 +++
.../ozone/freon/LeaderAppendLogEntryGenerator.java | 273 +++++++
.../hadoop/ozone/freon/RandomKeyGenerator.java | 27 +-
.../ozone/genesis/BenchMarkContainerStateMap.java | 8 +-
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 5 +-
.../ozone/genesis/BenchMarkOzoneManager.java | 11 +-
.../apache/hadoop/ozone/genesis/BenchMarkSCM.java | 11 +-
.../ozone/genesis/BenchmarkChunkManager.java | 33 +-
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 58 +-
.../org/apache/hadoop/ozone/shell/Handler.java | 133 ++++
.../org/apache/hadoop/ozone/shell/ListOptions.java | 58 ++
.../apache/hadoop/ozone/shell}/OzoneAddress.java | 45 +-
.../org/apache/hadoop/ozone/shell}/OzoneShell.java | 22 +-
.../java/org/apache/hadoop/ozone/shell}/Shell.java | 27 +-
.../hadoop/ozone/shell/StoreTypeOption.java} | 42 +-
.../apache/hadoop/ozone/shell/acl/AclHandler.java | 62 ++
.../apache/hadoop/ozone/shell/acl/AclOption.java | 91 +++
.../hadoop/ozone/shell/acl/GetAclHandler.java} | 24 +-
.../hadoop/ozone/shell/acl}/package-info.java | 4 +-
.../ozone/shell/bucket/AddAclBucketHandler.java} | 36 +-
.../hadoop/ozone/shell}/bucket/BucketCommands.java | 4 +-
.../hadoop/ozone/shell/bucket/BucketHandler.java} | 23 +-
.../hadoop/ozone/shell/bucket/BucketUri.java | 44 +-
.../ozone/shell/bucket/CreateBucketHandler.java | 88 +++
.../ozone/shell/bucket/DeleteBucketHandler.java} | 33 +-
.../ozone/shell/bucket/GetAclBucketHandler.java} | 28 +-
.../ozone/shell/bucket/InfoBucketHandler.java} | 32 +-
.../ozone/shell/bucket/ListBucketHandler.java | 68 ++
.../shell/bucket/RemoveAclBucketHandler.java} | 36 +-
.../ozone/shell/bucket/SetAclBucketHandler.java} | 37 +-
.../hadoop/ozone/shell/bucket}/package-info.java | 4 +-
.../hadoop/ozone/shell/keys/AddAclKeyHandler.java} | 36 +-
.../hadoop/ozone/shell/keys/DeleteKeyHandler.java | 50 ++
.../hadoop/ozone/shell/keys/GetAclKeyHandler.java} | 28 +-
.../hadoop/ozone/shell/keys/GetKeyHandler.java | 98 +++
.../hadoop/ozone/shell/keys/InfoKeyHandler.java | 58 ++
.../hadoop/ozone/shell}/keys/KeyCommands.java | 4 +-
.../hadoop/ozone/shell/keys/KeyHandler.java} | 23 +-
.../org/apache/hadoop/ozone/shell/keys/KeyUri.java | 44 +-
.../hadoop/ozone/shell/keys/ListKeyHandler.java | 78 ++
.../hadoop/ozone/shell/keys/PutKeyHandler.java | 108 +++
.../ozone/shell/keys/RemoveAclKeyHandler.java} | 36 +-
.../hadoop/ozone/shell}/keys/RenameKeyHandler.java | 45 +-
.../hadoop/ozone/shell/keys/SetAclKeyHandler.java} | 37 +-
.../hadoop/ozone/shell/keys}/package-info.java | 4 +-
.../apache/hadoop/ozone/shell}/package-info.java | 7 +-
.../hadoop/ozone/shell/s3/GetS3SecretHandler.java} | 30 +-
.../apache/hadoop/ozone/shell}/s3/S3Handler.java | 30 +-
.../org/apache/hadoop/ozone/shell}/s3/S3Shell.java | 21 +-
.../hadoop/ozone/shell}/s3/package-info.java | 2 +-
.../ozone/shell/token/CancelTokenHandler.java} | 29 +-
.../hadoop/ozone/shell/token/GetTokenHandler.java | 72 ++
.../ozone/shell/token/PrintTokenHandler.java} | 46 +-
.../ozone/shell/token/RenewTokenHandler.java} | 29 +-
.../hadoop/ozone/shell/token/RenewerOption.java} | 34 +-
.../hadoop/ozone/shell}/token/TokenCommands.java | 5 +-
.../hadoop/ozone/shell/token/TokenHandler.java | 54 ++
.../hadoop/ozone/shell/token/TokenOption.java | 56 ++
.../hadoop/ozone/shell}/token/package-info.java | 6 +-
.../ozone/shell/volume/AddAclVolumeHandler.java} | 36 +-
.../ozone/shell/volume/CreateVolumeHandler.java | 74 ++
.../ozone/shell/volume/DeleteVolumeHandler.java} | 30 +-
.../ozone/shell/volume/GetAclVolumeHandler.java} | 28 +-
.../ozone/shell}/volume/InfoVolumeHandler.java | 36 +-
.../ozone/shell/volume/ListVolumeHandler.java | 99 +++
.../shell/volume/RemoveAclVolumeHandler.java} | 36 +-
.../ozone/shell/volume/SetAclVolumeHandler.java} | 37 +-
.../ozone/shell}/volume/UpdateVolumeHandler.java | 46 +-
.../hadoop/ozone/shell}/volume/VolumeCommands.java | 4 +-
.../hadoop/ozone/shell/volume/VolumeHandler.java} | 23 +-
.../hadoop/ozone/shell/volume/VolumeUri.java | 44 +-
.../hadoop/ozone/shell/volume}/package-info.java | 4 +-
.../hadoop/ozone/shell}/TestOzoneAddress.java | 2 +-
.../apache/hadoop/ozone/shell}/package-info.java | 2 +-
hadoop-ozone/upgrade/pom.xml | 5 +
pom.xml | 359 ++++++---
816 files changed, 21498 insertions(+), 15184 deletions(-)
create mode 100755 .github/comment-commands/close.sh
rename CONTRIBUTION.md => CONTRIBUTING.md (100%)
create mode 100644 HISTORY.md
delete mode 100755 dev-support/bin/qbt
delete mode 100755 dev-support/bin/smart-apply-patch
delete mode 100755 dev-support/bin/test-patch
delete mode 100755 dev-support/bin/yetus-wrapper
create mode 100644 dev-support/byteman/mock-scm.btm
create mode 100644 dev-support/byteman/ratis-mock-followers.btm
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/HddsPrometheusConfig.java
(51%)
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/freon/FakeClusterTopology.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/freon/FakeScmBlockLocationProtocolClient.java
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/freon/FakeScmContainerLocationProtocolClient.java
rename {hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web
=>
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/freon}/package-info.java
(78%)
copy
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/GuiceInjectorUtilsForTestsImpl.java
=>
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/function/SupplierWithIOException.java
(71%)
create mode 100644
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/LegacyHadoopConfigurationSource.java
create mode 100644
hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java
create mode 100644
hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationSource.java
create mode 100644
hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/StorageSize.java
create mode 100644
hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/StorageUnit.java
create mode 100644
hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/TimeDurationUtil.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/freon/FakeRatisFollower.java
copy {hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/codec =>
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/freon}/package-info.java
(78%)
delete mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerStreamingOutput.java
create mode 100644 hadoop-hdds/docs/archetypes/design.md
create mode 100644 hadoop-hdds/docs/content/beyond/Containers.zh.md
copy hadoop-hdds/docs/content/{concept/Datanodes.zh.md => beyond/_index.zh.md}
(73%)
create mode 100644 hadoop-hdds/docs/content/design/configless.md
create mode 100644 hadoop-hdds/docs/content/design/gdpr.md
create mode 100644 hadoop-hdds/docs/content/design/locks.md
create mode 100644 hadoop-hdds/docs/content/design/multiraft.md
create mode 100644 hadoop-hdds/docs/content/design/namespace-support.md
create mode 100644 hadoop-hdds/docs/content/design/nfs.md
create mode 100644 hadoop-hdds/docs/content/design/ofs.md
create mode 100644 hadoop-hdds/docs/content/design/omha.md
create mode 100644 hadoop-hdds/docs/content/design/ozone-volume-management.md
create mode 100644 hadoop-hdds/docs/content/design/recon1.md
create mode 100644 hadoop-hdds/docs/content/design/recon2.md
create mode 100644 hadoop-hdds/docs/content/design/s3gateway.md
create mode 100644 hadoop-hdds/docs/content/design/scmha.md
create mode 100644 hadoop-hdds/docs/content/design/tde.md
create mode 100644 hadoop-hdds/docs/content/design/token.md
create mode 100644 hadoop-hdds/docs/content/design/trash.md
create mode 100644 hadoop-hdds/docs/content/design/typesafeconfig.md
create mode 100644 hadoop-hdds/docs/content/recipe/Prometheus.zh.md
rename hadoop-hdds/docs/content/{tools/SCMCLI.md => recipe/_index.zh.md} (55%)
create mode 100644 hadoop-hdds/docs/content/tools/Admin.md
create mode 100644 hadoop-hdds/docs/content/tools/AuditParser.zh.md
copy hadoop-hdds/docs/content/{concept/Datanodes.zh.md => tools/Genconf.zh.md}
(77%)
copy hadoop-hdds/docs/content/tools/{TestTools.md => TestTools.zh.md} (69%)
copy hadoop-hdds/docs/{content/concept/Datanodes.zh.md =>
themes/ozonedoc/layouts/_default/baseof.html} (64%)
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html
rename hadoop-hdds/{common =>
framework}/src/main/java/org/apache/hadoop/hdds/conf/DatanodeRatisGrpcConfig.java
(100%)
rename hadoop-hdds/{common =>
framework}/src/main/java/org/apache/hadoop/hdds/conf/DatanodeRatisServerConfig.java
(100%)
rename hadoop-hdds/{common =>
framework}/src/main/java/org/apache/hadoop/hdds/conf/HddsConfServlet.java (93%)
copy hadoop-hdds/{common =>
framework}/src/main/java/org/apache/hadoop/hdds/conf/package-info.java (100%)
rename hadoop-hdds/{common/src/main/java/org/apache/hadoop/ozone/web/utils =>
framework/src/main/java/org/apache/hadoop/hdds/server}/JsonUtils.java (98%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/bucket/package-info.java
=>
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/BatchOperationHandler.java
(55%)
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/DBColumnFamilyDefinition.java
rename
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/bucket/package-info.java
=>
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/DBDefinition.java
(59%)
rename
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell/TestObjectPrinter.java
=>
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/TestJsonUtils.java
(53%)
create mode 100644 hadoop-hdds/hadoop-dependency-client/README.md
create mode 100644 hadoop-hdds/hadoop-dependency-client/pom.xml
copy hadoop-hdds/{framework => hadoop-dependency-server}/pom.xml (55%)
copy hadoop-hdds/{client => hadoop-dependency-test}/pom.xml (79%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/ContainerIDCodec.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/ContainerInfoCodec.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/PipelineCodec.java
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/bucket/package-info.java
=>
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/PipelineIDCodec.java
(55%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/SCMDBDefinition.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/SafeModeHandler.java
delete mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/SafeModeNotification.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestUnknownContainerReport.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineActionHandler.java
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/safemode/TestSafeModeHandler.java
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMClientProtocolServer.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/scm/node/TestSCMNodeMetrics.java
copy {hadoop-ozone/upgrade => hadoop-hdds/test-utils}/pom.xml (60%)
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/GenericTestUtils.java
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/LambdaTestUtils.java
create mode 100644
hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/TimedOutTestsListener.java
rename hadoop-hdds/{common/src/main/java/org/apache/hadoop/ozone/web/utils =>
test-utils/src/main/java/org/apache/hadoop/test}/package-info.java (92%)
delete mode 100644
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/SCMCLI.java
copy
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/GuiceInjectorUtilsForTestsImpl.java
=>
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/WithScmClient.java
(76%)
delete mode 100644 hadoop-ozone/Jenkinsfile
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/conf/OMClientConfig.java
rename
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/bucket
=> common/src/main/java/org/apache/hadoop/ozone/conf}/package-info.java (87%)
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/DBUpdates.java
rename {hadoop-hdds => hadoop-ozone}/common/src/main/proto/FSProtos.proto (97%)
create mode 100644
hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/ha/TestOMFailoverProxyProvider.java
rename hadoop-ozone/dist/src/main/compose/ozone-om-ha/{run.sh =>
test_disabled.sh} (97%)
delete mode 100755
hadoop-ozone/dist/src/main/compose/ozone-topology/hdds-3084.sh
delete mode 100644
hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-org.openjdk.jmh-jmh-core.txt
delete mode 100644
hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-org.openjdk.jmh-jmh-generator-annprocess.txt
rename hadoop-ozone/dist/src/main/smoketest/{scmcli =>
admincli}/datanode.robot (93%)
rename hadoop-ozone/dist/src/main/smoketest/{scmcli =>
admincli}/pipeline.robot (88%)
copy hadoop-ozone/dist/src/main/smoketest/s3/{bucketcreate.robot =>
bucketdelete.robot} (70%)
create mode 100644
hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-token.robot
rename hadoop-ozone/dist/src/main/smoketest/topology/{scmcli.robot =>
cli.robot} (88%)
create mode 100644 hadoop-ozone/dist/src/test/shell/gc_opts.bats
create mode 100644
hadoop-ozone/fault-injection-test/mini-chaos-tests/src/test/java/org/apache/hadoop/ozone/MiniOzoneDatanodeChaosCluster.java
create mode 100644
hadoop-ozone/fault-injection-test/mini-chaos-tests/src/test/java/org/apache/hadoop/ozone/MiniOzoneOMChaosCluster.java
copy
hadoop-ozone/fault-injection-test/mini-chaos-tests/src/test/java/org/apache/hadoop/ozone/loadgenerators/{FilesystemLoadGenerator.java
=> ReadOnlyLoadGenerator.java} (55%)
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/Test2BlockOutputStream.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerListVolumes.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/ozShell/TestS3Shell.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/recon/TestReconTasks.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/node/TestSCMNodeMetrics.java
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/{ozShell =>
shell}/TestOzoneDatanodeShell.java (99%)
rename
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/{ozShell =>
shell}/TestOzoneShellHA.java (93%)
create mode 100644 hadoop-ozone/native-client/README.md
create mode 100644 hadoop-ozone/native-client/libo3fs-examples/libo3fs_read.c
create mode 100644 hadoop-ozone/native-client/libo3fs-examples/libo3fs_write.c
create mode 100644 hadoop-ozone/native-client/libo3fs/o3fs.c
copy
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineProvider.java
=> hadoop-ozone/native-client/libo3fs/o3fs.h (56%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java
(95%)
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManager.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManagerImpl.java
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/S3SecretManagerImpl.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/OmBucketInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/OmKeyInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/OmMultipartKeyInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/OmPrefixInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/OmVolumeArgsCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/RepeatedOmKeyInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/S3SecretValueCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/TokenIdentifierCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/UserVolumeInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/om/codec/package-info.java
(100%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMTrashRecoverRequest.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/bucket/S3BucketCreateRequest.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/bucket/S3BucketDeleteRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMTrashRecoverResponse.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/bucket/S3BucketCreateResponse.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/bucket/S3BucketDeleteResponse.java
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/security/AWSV4AuthValidator.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/security/OzoneBlockTokenSecretManager.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/security/OzoneDelegationTokenSecretManager.java
(88%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretKey.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretManager.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretStore.java
(100%)
copy
hadoop-ozone/ozone-manager/src/{test/java/org/apache/hadoop/ozone/web/ozShell
=> main/java/org/apache/hadoop/ozone/security}/package-info.java (91%)
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/AddAclBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/CreateBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/DeleteBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/GetAclBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/InfoBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/ListBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/RemoveAclBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/SetAclBucketHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/AddAclKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/DeleteKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/GetAclKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/GetKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/InfoKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/ListKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/PutKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/RemoveAclKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/SetAclKeyHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/GetS3SecretHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/S3BucketMapping.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/CancelTokenHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/GetTokenHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/PrintTokenHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/RenewTokenHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/AddAclVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/CreateVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/DeleteVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/GetAclVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/ListVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/RemoveAclVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/SetAclVolumeHandler.java
delete mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestS3BucketManager.java
rename hadoop-ozone/{common =>
ozone-manager}/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmMultipartKeyInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmPrefixInfoCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/test/java/org/apache/hadoop/ozone/om/codec/TestS3SecretValueCodec.java
(100%)
rename hadoop-ozone/{common =>
ozone-manager}/src/test/java/org/apache/hadoop/ozone/om/codec/package-info.java
(100%)
delete mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/bucket/TestS3BucketCreateRequest.java
delete mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/bucket/TestS3BucketDeleteRequest.java
rename
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/{s3/bucket/TestS3BucketRequest.java
=> security/TestOMDelegationTokenRequest.java} (60%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/security/TestOMGetDelegationTokenRequest.java
rename
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/{response/s3/bucket
=> request/security}/package-info.java (87%)
delete mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/bucket/TestS3BucketCreateResponse.java
delete mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/bucket/TestS3BucketDeleteResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/security/TestOMDelegationTokenResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/security/TestOMGetDelegationTokenResponse.java
copy
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/{web/ozShell
=> om/response/security}/package-info.java (87%)
rename hadoop-ozone/{common =>
ozone-manager}/src/test/java/org/apache/hadoop/ozone/security/TestAWSV4AuthValidator.java
(100%)
create mode 100644
hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/codegen/ReconSqlDbConfig.java
create mode 100644
hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/codegen/SqlDbUtils.java
create mode 100644
hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/schema/ContainerSchemaDefinition.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/metrics/OzoneManagerSyncMetrics.java
rename {hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/conf =>
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/metrics}/package-info.java
(81%)
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/persistence/ContainerSchemaManager.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/persistence/DerbyDataSourceProvider.java
copy
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/persistence/{DefaultDataSourceProvider.java
=> SqliteDataSourceProvider.java} (51%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/bucket/package-info.java
=>
recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconDBDefinition.java}
(65%)
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskConfig.java
rename
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/{AbstractOMMetadataManagerTest.java
=> OMMetadataManagerTestUtils.java} (79%)
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/ReconTestInjector.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/persistence/AbstractReconSqlDBTest.java
delete mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/persistence/AbstractSqlDatabaseTest.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/persistence/TestReconWithDifferentSqlDBs.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/persistence/TestSqlSchemaSetup.java
delete mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/types/GuiceInjectorUtilsForTests.java
create mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/TracingFilter.java
create mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadRequestUnmarshaller.java
copy
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineProvider.java
=>
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/BadRequestExceptionMapper.java
(52%)
create mode 100644
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestCompleteMultipartUploadRequestUnmarshaller.java
copy
hadoop-ozone/{recon/src/test/java/org/apache/hadoop/ozone/recon/GuiceInjectorUtilsForTestsImpl.java
=>
tools/src/main/java/org/apache/hadoop/ozone/debug/ChunkDataNodeDetails.java}
(59%)
copy
hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOMMetadataManager.java
=> tools/src/main/java/org/apache/hadoop/ozone/debug/ChunkDetails.java} (55%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/ChunkKeyHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/ContainerChunkInfo.java
copy
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeCommands.java
=>
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/OzoneDebug.java
(52%)
copy
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/debug}/package-info.java (91%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/HadoopNestedDirGenerator.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/LeaderAppendLogEntryGenerator.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/Handler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/ListOptions.java
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/OzoneAddress.java (89%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/OzoneShell.java (77%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/Shell.java (67%)
copy
hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOMMetadataManager.java
=> tools/src/main/java/org/apache/hadoop/ozone/shell/StoreTypeOption.java}
(54%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/acl/AclHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/acl/AclOption.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=> tools/src/main/java/org/apache/hadoop/ozone/shell/acl/GetAclHandler.java}
(64%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys
=> tools/src/main/java/org/apache/hadoop/ozone/shell/acl}/package-info.java
(88%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/AddAclBucketHandler.java}
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=>
tools/src/main/java/org/apache/hadoop/ozone/shell}/bucket/BucketCommands.java
(95%)
copy
hadoop-ozone/{recon/src/test/java/org/apache/hadoop/ozone/recon/GuiceInjectorUtilsForTestsImpl.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/BucketHandler.java}
(66%)
copy
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeCommands.java
=>
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/BucketUri.java
(50%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/CreateBucketHandler.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/DeleteBucketHandler.java}
(55%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/GetAclBucketHandler.java}
(60%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/InfoBucketHandler.java}
(56%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/ListBucketHandler.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/RemoveAclBucketHandler.java}
(50%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/SetAclBucketHandler.java}
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume
=> tools/src/main/java/org/apache/hadoop/ozone/shell/bucket}/package-info.java
(88%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/keys/AddAclKeyHandler.java}
(51%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/keys/DeleteKeyHandler.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/keys/GetAclKeyHandler.java}
(61%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/keys/GetKeyHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/keys/InfoKeyHandler.java
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/keys/KeyCommands.java
(95%)
copy
hadoop-ozone/{recon/src/test/java/org/apache/hadoop/ozone/recon/GuiceInjectorUtilsForTestsImpl.java
=> tools/src/main/java/org/apache/hadoop/ozone/shell/keys/KeyHandler.java}
(67%)
copy
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeCommands.java
=>
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/keys/KeyUri.java
(50%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/keys/ListKeyHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/keys/PutKeyHandler.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/keys/RemoveAclKeyHandler.java}
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=>
tools/src/main/java/org/apache/hadoop/ozone/shell}/keys/RenameKeyHandler.java
(55%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/keys/SetAclKeyHandler.java}
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket
=> tools/src/main/java/org/apache/hadoop/ozone/shell/keys}/package-info.java
(88%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/package-info.java (86%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/s3/GetS3SecretHandler.java}
(54%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/s3/S3Handler.java (64%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/s3/S3Shell.java (80%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/s3/package-info.java (94%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/token/CancelTokenHandler.java}
(56%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/token/GetTokenHandler.java
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/Handler.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/token/PrintTokenHandler.java}
(52%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/token/RenewTokenHandler.java}
(55%)
copy
hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOMMetadataManager.java
=> tools/src/main/java/org/apache/hadoop/ozone/shell/token/RenewerOption.java}
(57%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/token/TokenCommands.java
(95%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/token/TokenHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/token/TokenOption.java
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/main/java/org/apache/hadoop/ozone/shell}/token/package-info.java
(86%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/volume/AddAclVolumeHandler.java}
(50%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/volume/CreateVolumeHandler.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/volume/DeleteVolumeHandler.java}
(57%)
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/volume/GetAclVolumeHandler.java}
(60%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=>
tools/src/main/java/org/apache/hadoop/ozone/shell}/volume/InfoVolumeHandler.java
(54%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/volume/ListVolumeHandler.java
copy
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/volume/RemoveAclVolumeHandler.java}
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/ObjectPrinter.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/volume/SetAclVolumeHandler.java}
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=>
tools/src/main/java/org/apache/hadoop/ozone/shell}/volume/UpdateVolumeHandler.java
(55%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell
=>
tools/src/main/java/org/apache/hadoop/ozone/shell}/volume/VolumeCommands.java
(95%)
rename
hadoop-ozone/{recon/src/test/java/org/apache/hadoop/ozone/recon/GuiceInjectorUtilsForTestsImpl.java
=>
tools/src/main/java/org/apache/hadoop/ozone/shell/volume/VolumeHandler.java}
(66%)
copy
hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeCommands.java
=>
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/volume/VolumeUri.java
(50%)
rename
hadoop-ozone/{ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys
=> tools/src/main/java/org/apache/hadoop/ozone/shell/volume}/package-info.java
(88%)
rename
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/test/java/org/apache/hadoop/ozone/shell}/TestOzoneAddress.java
(98%)
rename
hadoop-ozone/{ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell
=> tools/src/test/java/org/apache/hadoop/ozone/shell}/package-info.java (94%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
