This is an automated email from the ASF dual-hosted git repository.
szetszwo pushed a change to branch HDDS-4454
in repository https://gitbox.apache.org/repos/asf/ozone.git
omit 185a28cedf HDDS-6592. [Ozone-Streaming] Fix
ContainerStateMachine#applyTransaction assert error (#3315)
omit 01ffcc2ccb HDDS-5666. Add option to createKey via streaming api in
Freon (#2574)
omit f5175dbd2d HDDS-6500. [Ozone-Streaming] Buffer the PutBlockRequest at
the end of the stream. (#3229)
omit bfb9216b5e HDDS-6137. [Ozone-Streaming] Refactor KeyDataStreamOutput.
(#3195)
omit a194579f9c HDDS-5798. [Ozone-Streaming] Setup TlsConf parameters.
(#3207)
omit df4c12abf5 HDDS-6461. Update Ratis version to 2.3.0-da5d868-SNAPSHOT.
(#3205)
omit ad5668e280 HDDS-6388. [Ozone-Streaming] Streaming write support both
pipeline model and star model (#3145)
omit aace239b72 HDDS-6355. [Ozone-Streaming] Fix CheckStyle problem (#3119)
omit f5290d312c HDDS-6229. [Ozone-Streaming] Data Channel abstraction on
datanode (#3023)
omit b09515f7bb HDDS-6282. Fix BlockDataStreamOutput#doFlushIfNeeded NPE
(#3060)
omit 84eb68320c HDDS-5487. [Ozone-Streaming] BlockDataStreamOutput support
FlushDelay. (#3002)
omit a793f3a003 HDDS-6298. Add XceiverServerRatis stream config (#3070)
omit 5dce655fdf HDDS-6138.[Ozone-Streaming] Define a limit on the size of
the retry bufferList. (#2946)
omit 6f32ef6bdc HDDS-6281. Update ratis version to 2.3.0-94db58b-SNAPSHOT
version (#3059)
omit f1dba712e3 HDDS-6178. [Ozone-Streaming] Fix NPE in HDDS-6139. (#2984)
omit b36004cca7 HDDS-6139. [Ozone-Streaming] Fix incorrect computation of
totalAckDataLength. (#2978)
omit 6cd0e89dad HDDS-6130. [Ozone-Streaming] When releaseBuffers will get
“Couldn 't find the required future” (#2939)
omit ad505bc02e HDDS-6039. Define a minimum packet size during streaming
writes. (#2883)
omit 2ef3b47fcf HDDS-5851. [Ozone-Streaming] Define a PutBlock/maxBuffer
fixed boundary for streaming writes. (#2866)
omit 49564be02a HDDS-5743. [Ozone-Streaming] Add option to write files via
streaming api in ofs and o3fs. (#2770)
omit d159210cbb HDDS-5879. [Ozone-Streaming] OzoneBucket add the
createMultipartStreamKey method (#2760)
omit 4871da7d8e HDDS-5961. [Ozone-Streaming] update the usage space of
Containers in the stream write (#2833)
omit ad9d281225 HDDS-5987. [Ozone-Streaming] Add XceiverClientRatis stream
config (#2841)
omit 659397a1da HDDS-5763. Provide an Executor for each LocalStream in
ContainerStateMachine (#2782)
omit 54cd1fe8cd HDDS-5895. [Ozone-Streaming] Make
raft.server.data-stream.client.pool.size configurable (#2766)
omit 37f5385c87 HDDS-5674.[Ozone-Streaming] Handle client retries on
exception (#2701)
omit 4e6fd1f578 HDDS-5849. [Ozone-Streaming]Write exceptions occur after
checksum is enabled (#2729)
omit a96e868eb8 HDDS-5486. [Ozone-Streaming] Streaming supports writing in
Pipline mode (#2682)
omit 0309d29aae HDDS-5742. Avoid unnecessary Bytebuffer conversions (#2673)
omit 05b1910e2a HDDS-5705. [Ozone-Streaming] Change ByteBufStreamOutput to
ByteBufferStreamOutput (#2603)
omit b19f749926 HDDS-5599. [Ozone-Streaming]drop BufferPool and
ChunkBuffer to avoid buffer copying (#2557)
omit a4ef7a6d50 HDDS-5488. [Ozone-Streaming] Add a new
BlockOutputStream/KeyOutputStream to support streaming api (#2495)
omit 968d1f1ac8 HDDS-5480. [Ozone-Streaming] Client and server should
support stream setup. (#2452)
omit 6f9ef072bc HDDS-5481. Fix stream() and link() method in
ContainerStateMachine. (#2451)
omit 9518dcf568 HDDS-5452. Add link method to ContainerStateMachine for
Ratis streaming (#2422)
omit 89dccf224f HDDS-5366. [Ozone-Streaming] Implement stream method to
ContainerStateMachine. (#2358). Contributed by mingchao zhao
add 1357a33df9 HDDS-6719. Verify build from source release tarball in CI
(#3396)
add 94dd1ea7c5 HDDS-6771. EC: ReplicationManager - make
ContainerReplicaPendingOps into a SCM service (#3445)
add 5ed0e0a9b4 HDDS-6323. Close RocksObject(s). (#3091)
add ddbe71d3f9 HDDS-6681. Post-Finalize behaviour for Bucket Layout
feature. (#3411)
add 2222be3303 HDDS-6788. Sort Ozone list Status output for FSO buckets.
(#3444)
add 66093e52c6 HDDS-6789. Omit download progress in builds (#3447)
add fad347c3c0 HDDS-6817. Bump rocksdbjni to 7.2.2 (#3463)
add 74a301cf06 HDDS-4945. Initial prototype for MultiTenant support for
Ozone (#2403)
add ab84d9ea47 HDDS-5485. Add new OM DB tables for AssignUserToTenant
(#2453)
add c826ec2532 Merge remote-tracking branch 'asf/master' into HDDS-4944
add 9787f70164 Merge remote-tracking branch 'asf/master' into HDDS-4944
add 2c95164bf2 HDDS-5647. [Multi-Tenant] Implement AssignUserToTenant
(#2564)
add e5fb5b02eb HDDS-5476. [Multi-Tenant] Support Ozone s3 authentication
with arbitrary accessId that is not same as the kerberos ID (#2635)
add 85167656ff HDDS-5750. [Multi-Tenant] GetS3Secret should retrieve
secret from new tables as well (#2649)
add 7f818ab446 Merge remote-tracking branch 'asf/master' into HDDS-4944
add 9fc34dc7ff HDDS-5754. [Multi-Tenant] Implement GetUserInfo (#2692)
add 7d63fe2d1f HDDS-5645. Chroot S3 requests for a tenant to their
corresponding volume. (#2697)
add 8ecff6fd1a HDDS-5776. [Multi-Tenant] Implement AssignTenantAdmin,
RevokeTenantAdmin, ListTenant, RevokeAccessID (#2734)
add dae8c0df17 HDDS-5834 [Multi-Tenant] Implement ListUsersInTenant,
Remove in memory maps. (#2759)
add 72d67018e6 HDDS-5939. [Multi-Tenant] Implement `ozone tenant user
getsecret` that does not generate secret when accessId does not exist (#2804)
add 19116fe1f1 HDDS-5972. [Multi-Tenant] Implement SetSecret: `ozone
tenant user setsecret` and `ozone s3 setsecret` (#2845)
add a19d252f98 HDDS-5942. Move Ranger REST API interactions under same
interface as Ranger client. (#2805)
add 8d1d957ee0 HDDS-6022. [Multi-Tenant] Implement DeleteTenant: `ozone
tenant delete` (#2857)
add f5581ce819 Merge remote-tracking branch 'asf/master' into HDDS-4944
add bb67e13048 HDDS-6004. Use changes from HDDS-5881 for volume chroot.
(#2924)
add f2fdf04666 Merge remote-tracking branch 'asf/master' into HDDS-4944
add 8d2fb136cd HDDS-6044. [Multi-Tenant] Implement Cross-Tenant Bucket
Sharing (#2936)
add 83a8c5f14b HDDS-6084. [Multi-Tenant] Handle upgrades to version
supporting S3 multi-tenancy (#3018)
add 2e6754f6cc HDDS-6214. [Multi-Tenant] Fix KMS Encryption/Decryption
(#3010)
add edcb2e1927 Merge remote-tracking branch 'asf/master' into HDDS-4944
add 52318302a6 HDDS-6063. [Multi-Tenant] Use VOLUME_LOCK in read and write
requests, and some minor refactoring (#3051)
add ecb685a6cf HDDS-6275. [Multi-Tenant] Add feature documentation and CLI
quick start guide (#3095)
add 43deaa8424 HDDS-6366. [Multi-Tenant] Disallow specifying custom
accessId in OzoneManager (#3166)
add f99130b6f9 HDDS-6396. [Multi-Tenant] Merge and cleanup tenant
group/role/policy tables, refactor protobuf messages and `isTenantAdmin` (#3177)
add c5bd28610a HDDS-6575. [Multi-Tenant] Clean up unused
tenantDefaultPolicyName field in CreateTenantRequest protobuf message (#3300)
add d621c8231c HDDS-6387. [Multi-Tenant] Refactor OMMultiTenantManager and
OMTenantRequestHelper (#3264)
add 9280b67553 HDDS-6375. [Multi-Tenant] Implement tenant request metrics
(#3256)
add 7719e23eec Merge remote-tracking branch 'asf/master' into HDDS-4944
add f127fa9944 HDDS-6576. [Multi-Tenant] Update documentation around
Ranger policy creation on bucket sharing (#3303)
add 6044ef607d HDDS-6566. [Multi-Tenant] Fix a permission check bug that
prevents non-delegated admins from assigning/revoking users to/from the tenant
(#3288)
add 42d30d37f1 HDDS-6625. [Multi-Tenant] Follow-up: Set owner of buckets
created via S3 Gateway to actual user (#3327)
add c55bd0290f HDDS-6612. [Multi-Tenant] Add a config key to enable or
disable S3 Multi-Tenancy feature (#3397)
add f0b3013091 HDDS-6340. [Multi-Tenant] Add tenant CLI option to print
results in JSON (#3381)
add 917b079a61 HDDS-5836. [Multi-Tenant] Use Ranger java client. (#3408)
add 35043683a6 HDDS-6371. [Multi-Tenant] Provide OM DB to Apache Ranger
Sync mechanism (#3131)
add 92daac20f0 Merge remote-tracking branch 'asf/master' into HDDS-4944
add dcd17390be HDDS-6701. [Multi-Tenant] Add proper locking between Ranger
background sync service and tenant requests; bug fixes (#3450)
add 1acaa82f34 Merge remote-tracking branch 'asf/master' into HDDS-4944
add 0242a823f9 Merge remote-tracking branch 'asf/HDDS-4944'
add b3ed6f403e HDDS-6816. SCMContainerPlacementRackScatter should use
original required node num to validate placement policy (#3462)
add 948249635c HDDS-6780. Bump axios to 0.21.4 (#3438)
add f51a0d543f HDDS-6804. use common backgroundScmService for
backgroundPipelieScruber (#3459)
add 3a4d2d4723 HDDS-6768. Add Volume and Bucket ID to the key path for FSO
Objects (#3449)
add af3a80d041 HDDS-6808. EC: Fix datanode exclusion check in client
(#3460)
add e8abd0f6e2 HDDS-6280. Support Container Balancer HA (#3423)
add e021a06b4c HDDS-6474. Add test to cover the FSO bucket list status
with beyond batch boundary and cache. (#3379). Contributed by aswinshakil
add 24ca66b067 HDDS-6705 Add metrics for volume statistics including disk
capacity, usage, Reserved (#3430)
add 0bc3321950 HDDS-5920. Allow to add ColumnFamilyOptions to tables.
(#2790)
add fcf74fcd96 HDDS-5921. [Merge rocksdb in datanode]Decouple
DatanodeStore impl classes from container. (#2791)
add 83bb7034b4 HDDS-5923. Add DBHandle abstraction over
ReferenceCountedDB. (#2794)
add db4734237b HDDS-6404. Format table key according to schema in
KeyValueContainerData. (#3172)
add 482886d77b HDDS-6428. Add prefix iterator support to RDBTable. (#3176)
add d693014b02 HDDS-6486. [Merge rocksdb in datanode] Add new container
schema v3 definitions. (#3253)
add 4e7a0c395b HDDS-6540. Add a Cache for per-disk RocksDB Instance.
(#3283)
add 7c8d51dca5 HDDS-6541. [Merge rocksdb in datanode] Per-disk DB location
management. (#3292)
add 07ad6080f4 Update non-rolling-upgrade test mlv to the newest 4.
add de18c59022 HDDS-6542. [Merge rocksdb in datanode] KeyValueContainer
operation adaptation for schema v3 containers. (#3346)
add 562a242095 HDDS-6543. [Merge rocksdb in datanode] BlockDeletingService
adaptation for schema v3 containers. (#3385)
add 3998c1da1f HDDS-6745: [Merge rocksdb in datanode] Flush RocksDB WAL
when closing container in Schema V3. (#3419)
add 4a94b609fa HDDS-6544. [Merge rocksdb in datanode] New container
replication operations for schema v3 container. (#3398)
add 3178ba5a94 HDDS-6597. [Merge rocksdb in datanode] Non-rolling upgrade
supports container Schema V3. (#3392)
add 78cff9c031 HDDS-6754. [Merge rocksdb in datanode] Add compatibility
test between schema v3 & v2. (#3422)
add 9b3de4101b HDDS-6747. Support configurable separator between
container ID and container meta data key (#3420)
add 0e7b095c30 HDDS-6791: Add smoke test for Merge RocksDB in datanode
feature. (#3448)
add 48525d793d HDDS-6792: [Merge RocksDB in Datanode] Fix issues reportd
by sonar. (#3446)
add a6b5174fbb HDDS-6790: Add document for Merge RocksDB in datanode
feature. (#3451)
add f94b7eb12d Merge remote-tracking branch 'origin/HDDS-3630'
add 55c3848624 Merge remote-tracking branch 'origin/HDDS-3630'
add 57b4a8a61d HDDS-4859. [FSO]ListKeys: seek all the files/dirs from
startKey to keyPrefix (#3466)
add c923a266cb HDDS-6819. Add LEGACY to AllowedBucketLayouts in
CreateBucketHandler (#3473)
add fdf7376d3f Merge remote-tracking branch 'origin/master' into HDDS-3630
add 687c6cac32 HDDS-6820. Bucket Layout Post-Finalization Validators for
ACL Requests. (#3472)
add b80d354ed2 HDDS-6823. Intermittent failure in
TestOzoneECClient#testExcludeOnDNMixed (#3476)
add 15257c7b3c HDDS-6795: EC: PipelineStateMap#addPipeline should not have
precondition checks post db updates (#3453)
add 88f4101317 HDDS-6764: EC: DN ability to create RECOVERING containers
for EC reconstruction. (#3458)
add db867de625 HDDS-6828. Revert RockDB version pending leak fixes (#3475)
add e1fb4a50bb HDDS-6810. Add a optional flag to trigger listStatus as
part of listKeys for FSO buckets. (#3461)
add a5858ddadb HDDS-6811. Bucket create message with layout type (#3479)
add 6c577cdef1 HDDS-6824. Revert changes made in proto.lock by HDDS-6768.
(#3480)
add b7bf8a4260 HDDS-6683. Refactor OM server bucket layout configuration
usage (#3477)
add 4a0d01e649 HDDS-5821 Container cache management for closing RockDB
(#3426)
add de3868308f HDDS-6759: Add listblock API in MockDatanodeStorage (#3452)
add 414d272ac9 HDDS-6706. Exposing Volume Information Metrics to the
DataNode UI (#3478)
add f3dc1580c5 HDDS-6091. Add file checksum to OmKeyInfo (#3201)
add 50e8e49a74 HDDS-4364: [FSO]List FileStatus : startKey can be a
non-existed path (#3481)
add 3397db917e HDDS-6686. Do Leadship check before SASL token
verification. (#3382)
add 1bf161464a HDDS-6751. Migrate parameterized tests in hdds-server-scm
to JUnit5 (#3431)
add 039e8a8f5a HDDS-6665. EC: Extend BlockReconstructedInputStreams to
recover parity block buffers as well if missing (#3457)
add 35c9ba0ec4 HDDS-6838. [FSO] Sets OM default to LEGACY for an older
client's bucket create request (#3487)
add bedc7ad045 HDDS-6760. [SCM HA finalization] Support for happy path
finalization (#3465)
add 7847e663c4 HDDS-6836. Fix Ozone http header configuration (#3486)
add 7981060f1e HDDS-6827. Need proper error message when "RATIS"
replication-type is passed with EC codec (#3491)
add 59180681b0 HDDS-6815: EC: getFileCheckSum should return null EC files
until ECFileChecksum implemented. (#3489)
add 1d7ad4ccd7 HDDS-6844. Reduce the number of watch requests in
XceiverClientRatis. (#3493)
new f73ee65af4 HDDS-5366. [Ozone-Streaming] Implement stream method to
ContainerStateMachine. (#2358). Contributed by mingchao zhao
new 117771b8bb HDDS-5452. Add link method to ContainerStateMachine for
Ratis streaming (#2422)
new 0eee8906e2 HDDS-5481. Fix stream() and link() method in
ContainerStateMachine. (#2451)
new 4c93359a61 HDDS-5480. [Ozone-Streaming] Client and server should
support stream setup. (#2452)
new 84407f0c17 HDDS-5488. [Ozone-Streaming] Add a new
BlockOutputStream/KeyOutputStream to support streaming api (#2495)
new 948e3455db HDDS-5599. [Ozone-Streaming]drop BufferPool and
ChunkBuffer to avoid buffer copying (#2557)
new f7e599046d HDDS-5705. [Ozone-Streaming] Change ByteBufStreamOutput to
ByteBufferStreamOutput (#2603)
new 796ef0ec4c HDDS-5742. Avoid unnecessary Bytebuffer conversions (#2673)
new 55dab067dc HDDS-5486. [Ozone-Streaming] Streaming supports writing in
Pipline mode (#2682)
new 6ef1b39dac HDDS-5849. [Ozone-Streaming]Write exceptions occur after
checksum is enabled (#2729)
new 801b88264d HDDS-5674.[Ozone-Streaming] Handle client retries on
exception (#2701)
new 1cb2e2123b HDDS-5895. [Ozone-Streaming] Make
raft.server.data-stream.client.pool.size configurable (#2766)
new 12cd5d167c HDDS-5763. Provide an Executor for each LocalStream in
ContainerStateMachine (#2782)
new e5f3bb2736 HDDS-5987. [Ozone-Streaming] Add XceiverClientRatis stream
config (#2841)
new e350e01658 HDDS-5961. [Ozone-Streaming] update the usage space of
Containers in the stream write (#2833)
new 085bbf1438 HDDS-5879. [Ozone-Streaming] OzoneBucket add the
createMultipartStreamKey method (#2760)
new 657915b506 HDDS-5743. [Ozone-Streaming] Add option to write files via
streaming api in ofs and o3fs. (#2770)
new 5804f2214c HDDS-5851. [Ozone-Streaming] Define a PutBlock/maxBuffer
fixed boundary for streaming writes. (#2866)
new d2a51a0579 HDDS-6039. Define a minimum packet size during streaming
writes. (#2883)
new a794067581 HDDS-6130. [Ozone-Streaming] When releaseBuffers will get
“Couldn 't find the required future” (#2939)
new 7da72fd374 HDDS-6139. [Ozone-Streaming] Fix incorrect computation of
totalAckDataLength. (#2978)
new fec03b7eaa HDDS-6178. [Ozone-Streaming] Fix NPE in HDDS-6139. (#2984)
new 918cec55cd HDDS-6281. Update ratis version to 2.3.0-94db58b-SNAPSHOT
version (#3059)
new 32598ca3a6 HDDS-6138.[Ozone-Streaming] Define a limit on the size of
the retry bufferList. (#2946)
new 630ea3a95d HDDS-6298. Add XceiverServerRatis stream config (#3070)
new 7caa82ca36 HDDS-5487. [Ozone-Streaming] BlockDataStreamOutput support
FlushDelay. (#3002)
new 1ae8d86946 HDDS-6282. Fix BlockDataStreamOutput#doFlushIfNeeded NPE
(#3060)
new d82f075dad HDDS-6229. [Ozone-Streaming] Data Channel abstraction on
datanode (#3023)
new 1fab6bb867 HDDS-6355. [Ozone-Streaming] Fix CheckStyle problem (#3119)
new fde368193b HDDS-6388. [Ozone-Streaming] Streaming write support both
pipeline model and star model (#3145)
new d35335d22f HDDS-6461. Update Ratis version to 2.3.0-da5d868-SNAPSHOT.
(#3205)
new ba69a4a838 HDDS-5798. [Ozone-Streaming] Setup TlsConf parameters.
(#3207)
new 6c0f95f967 HDDS-6137. [Ozone-Streaming] Refactor KeyDataStreamOutput.
(#3195)
new 96d5eb06ad HDDS-6500. [Ozone-Streaming] Buffer the PutBlockRequest at
the end of the stream. (#3229)
new 1063516b9d HDDS-5666. Add option to createKey via streaming api in
Freon (#2574)
new 19b8f37a9a HDDS-6592. [Ozone-Streaming] Fix
ContainerStateMachine#applyTransaction assert error (#3315)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (185a28cedf)
\
N -- N -- N refs/heads/HDDS-4454 (19b8f37a9a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 36 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/post-commit.yml | 22 +-
dev-support/ci/selective_ci_checks.bats | 8 +-
dev-support/ci/selective_ci_checks.sh | 6 +-
hadoop-hdds/client/pom.xml | 4 +
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 111 +-
.../hadoop/hdds/scm/client/HddsClientUtils.java | 5 +-
.../io/ECBlockReconstructedStripeInputStream.java | 460 +++++---
.../TestECBlockReconstructedStripeInputStream.java | 372 ++++---
.../java/org/apache/hadoop/hdds/HddsUtils.java | 11 +
.../hadoop/hdds/client/ReplicationConfig.java | 7 +-
.../hadoop/hdds/conf/OzoneConfiguration.java | 7 +-
.../org/apache/hadoop/hdds/ratis/RatisHelper.java | 2 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 11 +
.../hdds/scm/storage/ContainerProtocolCalls.java | 37 +-
.../hadoop/hdds/upgrade/HDDSLayoutFeature.java | 4 +-
.../org/apache/hadoop/ozone/ClientVersion.java | 4 +
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 5 +
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 98 +-
.../upgrade/AbstractLayoutVersionManager.java | 41 +-
.../ozone/upgrade/BasicUpgradeFinalizer.java | 88 +-
.../DefaultUpgradeFinalizationExecutor.java | 16 +-
.../hadoop/ozone/upgrade/LayoutVersionManager.java | 7 +
.../upgrade/UpgradeFinalizationExecutor.java} | 34 +-
.../common/src/main/resources/ozone-default.xml | 64 ++
.../org/apache/hadoop/ozone/TestOzoneConsts.java | 35 +
.../ozone/container/ContainerTestHelper.java | 13 +-
.../InjectedUpgradeFinalizationExecutor.java | 5 +-
.../ozone/upgrade/TestBasicUpgradeFinalizer.java | 27 +-
.../TestDefaultUpgradeFinalizationExecutor.java | 5 +-
.../ozone/upgrade/TestUpgradeFinalizerActions.java | 2 +-
.../hadoop/hdds/conf/ConfigurationSource.java | 32 +-
.../hadoop/hdds/conf/ConfigurationSourceTest.java | 50 +
.../apache/hadoop/ozone/HddsDatanodeService.java | 6 +-
.../ozone/container/common/impl/ContainerSet.java | 5 +
.../container/common/impl/HddsDispatcher.java | 11 +-
.../container/common/interfaces/DBHandle.java | 48 +
.../ozone/container/common/interfaces/Handler.java | 7 +
.../common/statemachine/DatanodeConfiguration.java | 66 ++
.../common/statemachine/DatanodeStateMachine.java | 11 +-
.../commandhandler/DeleteBlocksCommandHandler.java | 90 +-
.../FinalizeNewLayoutVersionCommandHandler.java | 2 +-
.../states/endpoint/VersionEndpointTask.java | 68 +-
.../container/common/utils/ContainerCache.java | 30 +-
.../container/common/utils/DatanodeStoreCache.java | 88 ++
.../container/common/utils/HddsVolumeUtil.java | 230 +---
.../hadoop/ozone/container/common/utils/RawDB.java | 42 +
.../container/common/utils/ReferenceCountedDB.java | 37 +-
.../container/common/utils/StorageVolumeUtil.java | 202 ++++
.../ozone/container/common/volume/DbVolume.java | 153 +++
...dataVolumeFactory.java => DbVolumeFactory.java} | 35 +-
.../ozone/container/common/volume/HddsVolume.java | 425 ++++----
.../container/common/volume/HddsVolumeFactory.java | 37 +-
.../container/common/volume/MetadataVolume.java | 11 +
.../common/volume/MetadataVolumeFactory.java | 2 +-
.../container/common/volume/MutableVolumeSet.java | 11 +
.../container/common/volume/StorageVolume.java | 278 ++++-
.../common/volume/StorageVolumeFactory.java | 41 +-
.../container/common/volume/VolumeInfoMetrics.java | 137 +++
.../container/keyvalue/KeyValueContainer.java | 124 ++-
.../container/keyvalue/KeyValueContainerCheck.java | 24 +-
.../container/keyvalue/KeyValueContainerData.java | 109 +-
.../KeyValueContainerMetadataInspector.java | 62 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 23 +-
.../container/keyvalue/TarContainerPacker.java | 15 +-
.../container/keyvalue/helpers/BlockUtils.java | 179 ++-
.../helpers/KeyValueContainerLocationUtil.java | 12 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 194 ++--
.../container/keyvalue/impl/BlockManagerImpl.java | 57 +-
.../background/BlockDeletingService.java | 95 +-
.../metadata/AbstractDatanodeDBDefinition.java | 11 +-
.../container/metadata/AbstractDatanodeStore.java | 37 +-
.../metadata/DatanodeSchemaOneDBDefinition.java | 6 +-
.../metadata/DatanodeSchemaThreeDBDefinition.java | 158 +++
.../metadata/DatanodeSchemaTwoDBDefinition.java | 6 +-
.../ozone/container/metadata/DatanodeStore.java | 18 +-
.../metadata/DatanodeStoreSchemaOneImpl.java | 7 +-
.../metadata/DatanodeStoreSchemaThreeImpl.java | 130 +++
.../metadata/DatanodeStoreSchemaTwoImpl.java | 13 +-
.../ozone/container/metadata/DatanodeTable.java | 34 +-
.../container/metadata/DeleteTransactionStore.java | 29 +
.../metadata/SchemaOneDeletedBlocksTable.java | 8 +-
.../ozone/container/ozoneimpl/OzoneContainer.java | 33 +-
.../upgrade/DataNodeUpgradeFinalizer.java | 19 +-
.../upgrade/DatanodeSchemaV3FinalizeAction.java | 82 ++
.../ScmHAFinalizeUpgradeActionDatanode.java | 2 +-
.../upgrade/VersionedDatanodeFeatures.java | 28 +-
.../webapps/hddsDatanode/dn-overview.html | 30 +
.../src/main/resources/webapps/hddsDatanode/dn.js | 24 +-
.../ozone/container/common/ContainerTestUtils.java | 32 +
.../container/common/TestBlockDeletingService.java | 167 +--
.../ozone/container/common/TestContainerCache.java | 79 +-
.../container/common/TestDatanodeStoreCache.java | 78 ++
.../common/TestKeyValueContainerData.java | 20 +-
.../TestSchemaOneBackwardsCompatibility.java | 150 +--
.../TestSchemaTwoBackwardsCompatibility.java | 367 +++++++
.../common/helpers/TestDatanodeVersionFile.java | 18 +-
.../common/impl/TestContainerPersistence.java | 30 +-
.../statemachine/TestDatanodeConfiguration.java | 11 +
.../container/common/utils/TestHddsVolumeUtil.java | 238 ++++
.../common/utils/TestStorageVolumeUtil.java | 99 ++
.../container/common/volume/TestDbVolume.java | 172 +++
.../container/common/volume/TestHddsVolume.java | 164 ++-
.../container/common/volume/TestStorageVolume.java | 83 ++
.../common/volume/TestVolumeSetDiskChecks.java | 30 +
.../keyvalue/ContainerTestVersionInfo.java | 79 ++
.../keyvalue/TestKeyValueBlockIterator.java | 82 +-
.../container/keyvalue/TestKeyValueContainer.java | 53 +-
.../keyvalue/TestKeyValueContainerCheck.java | 19 +-
.../TestKeyValueContainerIntegrityChecks.java | 44 +-
.../TestKeyValueContainerMetadataInspector.java | 20 +-
.../container/keyvalue/TestTarContainerPacker.java | 82 +-
.../keyvalue/impl/TestBlockManagerImpl.java | 23 +-
.../container/ozoneimpl/TestContainerReader.java | 85 +-
.../container/ozoneimpl/TestOzoneContainer.java | 99 +-
.../upgrade/TestDatanodeUpgradeToSchemaV3.java | 759 +++++++++++++
.../upgrade/TestDatanodeUpgradeToScmHA.java | 5 +
.../docs/content/design/dn-merge-rocksdb.md | 29 +
.../docs/content/feature/Nonrolling-Upgrade.md | 1 +
hadoop-hdds/docs/content/feature/PrefixFSO.md | 2 +-
.../feature/S3-Multi-Tenancy-Access-Control.md | 93 ++
.../docs/content/feature/S3-Multi-Tenancy-Setup.md | 93 ++
.../docs/content/feature/S3-Multi-Tenancy.md | 75 ++
.../docs/content/feature/S3-Tenant-Commands.md | 458 ++++++++
.../docs/content/feature/dn-merge-rocksdb.md | 70 ++
.../themes/ozonedoc/layouts/_default/section.html | 3 +-
.../rawcoder/ByteBufferDecodingState.java | 17 +-
.../hadoop/hdds/scm/metadata/SCMMetadataStore.java | 6 +
.../hadoop/hdds/server/http/HttpServer2.java | 29 +-
.../hadoop/hdds/utils/DBCheckpointServlet.java | 2 +-
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 7 +
.../hadoop/hdds/utils/MetadataKeyFilters.java | 1 +
.../hdds/utils/db/DBColumnFamilyDefinition.java | 13 +
.../org/apache/hadoop/hdds/utils/db/DBStore.java | 9 +-
.../hadoop/hdds/utils/db/DBStoreBuilder.java | 2 +-
.../db/{StringCodec.java => DumpFileLoader.java} | 40 +-
.../db/{StringCodec.java => DumpFileWriter.java} | 44 +-
...tringCodec.java => FixedLengthStringCodec.java} | 14 +-
.../hdds/utils/db/FixedLengthStringUtils.java | 57 +
.../hadoop/hdds/utils/db/RDBBatchOperation.java | 37 +-
.../hadoop/hdds/utils/db/RDBCheckpointManager.java | 33 +-
.../hadoop/hdds/utils/db/RDBSstFileLoader.java | 62 ++
.../hadoop/hdds/utils/db/RDBSstFileWriter.java | 98 ++
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 188 +---
.../hadoop/hdds/utils/db/RDBStoreIterator.java | 54 +-
.../org/apache/hadoop/hdds/utils/db/RDBTable.java | 220 ++--
.../apache/hadoop/hdds/utils/db/RocksDatabase.java | 504 +++++++++
.../apache/hadoop/hdds/utils/db/StringCodec.java | 2 +-
.../org/apache/hadoop/hdds/utils/db/Table.java | 43 +-
.../apache/hadoop/hdds/utils/db/TableConfig.java | 7 +-
.../apache/hadoop/hdds/utils/db/TypedTable.java | 44 +-
.../hadoop/hdds/utils/db/TestDBStoreBuilder.java | 79 ++
.../hdds/utils/db/TestFixedLengthStringUtils.java} | 43 +-
.../apache/hadoop/hdds/utils/db/TestRDBStore.java | 8 +
.../hadoop/hdds/utils/db/TestRDBStoreIterator.java | 54 +
.../hadoop/hdds/utils/db/TestRDBTableStore.java | 228 +++-
.../src/main/proto/DatanodeClientProtocol.proto | 4 +
.../interface-client/src/main/proto/hdds.proto | 20 +
.../src/main/proto/SCMRatisProtocol.proto | 1 +
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 1 +
hadoop-hdds/server-scm/pom.xml | 5 +
.../scm/container/balancer/ContainerBalancer.java | 291 +++--
.../balancer/ContainerBalancerConfiguration.java | 84 ++
.../IllegalContainerBalancerStateException.java | 5 +-
...lidContainerBalancerConfigurationException.java | 11 +-
.../SCMContainerPlacementRackScatter.java | 38 +-
.../BackgroundSCMService.java} | 213 ++--
.../org/apache/hadoop/hdds/scm/ha/SCMContext.java | 37 +-
.../org/apache/hadoop/hdds/scm/ha/SCMService.java | 2 +-
.../hadoop/hdds/scm/ha/SCMServiceException.java} | 40 +-
.../hadoop/hdds/scm/ha/SCMServiceManager.java | 6 +-
.../apache/hadoop/hdds/scm/ha/StatefulService.java | 23 +-
.../scm/ha/StatefulServiceStateManagerImpl.java | 14 +
.../hadoop/hdds/scm/ha/io/ByteStringCodec.java | 38 +-
.../apache/hadoop/hdds/scm/ha/io/CodecFactory.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/IntegerCodec.java | 31 +-
.../hadoop/hdds/scm/metadata/SCMDBDefinition.java | 14 +-
.../hdds/scm/metadata/SCMMetadataStoreImpl.java | 12 +
.../hdds/scm/node/HealthyReadOnlyNodeHandler.java | 51 +-
.../hadoop/hdds/scm/node/NodeStateManager.java | 49 +-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 62 +-
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 22 +-
.../hdds/scm/pipeline/PipelineManagerImpl.java | 32 +-
.../hadoop/hdds/scm/server/SCMCertStore.java | 6 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 24 +-
.../hadoop/hdds/scm/server/SCMConfigurator.java | 21 +
.../hdds/scm/server/StorageContainerManager.java | 116 +-
.../scm/server/upgrade/FinalizationCheckpoint.java | 73 ++
.../scm/server/upgrade/FinalizationManager.java | 54 +
.../server/upgrade/FinalizationManagerImpl.java | 197 ++++
.../server/upgrade/FinalizationStateManager.java | 47 +
.../upgrade/FinalizationStateManagerImpl.java | 222 ++++
.../upgrade/SCMUpgradeFinalizationContext.java | 144 +++
.../scm/server/upgrade/SCMUpgradeFinalizer.java | 181 +--
.../ScmHAUnfinalizedStateValidationAction.java | 10 +-
.../ScmOnFinalizeActionForDatanodeSchemaV2.java | 5 +-
.../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 17 +-
.../scm/TestStorageContainerManagerHttpServer.java | 49 +-
.../container/balancer/TestContainerBalancer.java | 157 ++-
.../TestSCMContainerPlacementRackAware.java | 223 ++--
.../TestSCMContainerPlacementRackScatter.java | 208 ++--
.../hdds/scm/ha/TestBackgroundSCMService.java | 113 ++
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 2 +-
.../hadoop/hdds/scm/node/TestNodeStateManager.java | 57 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 104 +-
.../pipeline/TestBackgroundPipelineScrubber.java | 109 --
.../TestPipelineDatanodesIntersection.java | 58 +-
.../hdds/scm/server/TestSCMContainerMetrics.java | 3 +-
.../scm/upgrade/FinalizationManagerTestImpl.java | 54 +
.../upgrade/FinalizationStateManagerTestImpl.java} | 35 +-
.../TestSCMHAUnfinalizedStateValidationAction.java | 84 +-
.../hdds/scm/upgrade/TestScmFinalization.java | 333 ++++++
.../apache/hadoop/ozone/client/ObjectStore.java | 173 ++-
.../apache/hadoop/ozone/client/OzoneBucket.java | 181 ++-
.../apache/hadoop/ozone/client/OzoneVolume.java | 30 +
.../org/apache/hadoop/ozone/client/TenantArgs.java | 82 ++
.../org/apache/hadoop/ozone/client/VolumeArgs.java | 2 +
.../client/checksum/BaseFileChecksumHelper.java | 63 +-
.../checksum/ReplicatedFileChecksumHelper.java | 9 +
.../hadoop/ozone/client/io/ECKeyOutputStream.java | 3 +-
.../ozone/client/protocol/ClientProtocol.java | 150 ++-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 238 +++-
.../hadoop/ozone/client/MockDatanodeStorage.java | 27 +-
.../ozone/client/MockXceiverClientFactory.java | 60 +-
.../hadoop/ozone/client/MockXceiverClientSpi.java | 12 +
.../hadoop/ozone/client/TestOzoneECClient.java | 71 +-
.../checksum/TestReplicatedFileChecksumHelper.java | 30 +
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 13 +
.../org/apache/hadoop/ozone/audit/OMAction.java | 15 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 41 +
.../hadoop/ozone/om/exceptions/OMException.java | 16 +-
.../hadoop/ozone/om/helpers/BucketLayout.java | 17 +
.../hadoop/ozone/om/helpers/DeleteTenantState.java | 91 ++
.../hadoop/ozone/om/helpers/OmDBAccessIdInfo.java | 136 +++
.../hadoop/ozone/om/helpers/OmDBTenantState.java | 205 ++++
.../ozone/om/helpers/OmDBUserPrincipalInfo.java | 93 ++
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 41 +-
.../hadoop/ozone/om/helpers/OmRangerSyncArgs.java | 67 ++
.../hadoop/ozone/om/helpers/OmTenantArgs.java | 92 ++
.../hadoop/ozone/om/helpers/OmTenantUserArgs.java | 36 +-
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 55 +-
.../hadoop/ozone/om/helpers/OzoneFSUtils.java | 27 +
.../hadoop/ozone/om/helpers/OzoneFileStatus.java | 4 +-
.../hadoop/ozone/om/helpers/RepeatedOmKeyInfo.java | 3 +-
.../hadoop/ozone/om/helpers/S3SecretValue.java | 1 +
.../hadoop/ozone/om/helpers/S3VolumeContext.java | 94 ++
.../hadoop/ozone/om/helpers/TenantStateList.java | 72 ++
.../ozone/om/helpers/TenantUserInfoValue.java | 75 ++
.../hadoop/ozone/om/helpers/TenantUserList.java | 70 ++
.../hadoop/ozone/om/multitenant/AccessPolicy.java | 152 +++
.../ozone/om/multitenant/AccountNameSpace.java | 77 ++
.../ozone/om/multitenant/BucketNameSpace.java | 94 ++
.../ozone/om/multitenant/OzoneOwnerPrincipal.java | 39 +
.../hadoop/ozone/om/multitenant/OzoneTenant.java | 96 ++
.../om/multitenant/OzoneTenantRolePrincipal.java | 41 +
.../ozone/om/multitenant/RangerAccessPolicy.java | 316 ++++++
.../apache/hadoop/ozone/om/multitenant/Tenant.java | 59 +
.../om/multitenant/impl/AccountNameSpaceImpl.java | 53 +
.../impl/SingleVolumeTenantNamespace.java | 83 ++
.../ozone/om/multitenant/impl/package-info.java | 33 +-
.../hadoop/ozone/om/multitenant/package-info.java | 33 +-
.../ozone/om/protocol/OzoneManagerProtocol.java | 144 +++
.../apache/hadoop/ozone/om/protocol/S3Auth.java | 18 +-
...OzoneManagerProtocolClientSideTranslatorPB.java | 302 ++++-
.../apache/hadoop/ozone/protocolPB/OMPBHelper.java | 185 +++-
.../ozone/om/exceptions/TestResultCodes.java | 3 +-
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 5 +-
hadoop-ozone/dev-support/checks/build.sh | 2 +-
hadoop-ozone/dev-support/checks/checkstyle.sh | 2 +-
hadoop-ozone/dev-support/checks/findbugs.sh | 2 +-
hadoop-ozone/dev-support/checks/junit.sh | 2 +-
hadoop-ozone/dev-support/checks/rat.sh | 2 +-
.../dist/src/main/compose/compatibility/test.sh | 1 +
.../main/compose/ozonesecure/docker-compose.yaml | 16 +-
.../src/main/compose/ozonesecure/docker-config | 13 +
.../ozonesecure/mockserverInitialization.json | 98 ++
.../non-rolling-upgrade/1.1.0-1.2.0/callback.sh | 2 +-
.../non-rolling-upgrade/1.2.1-1.3.0/callback.sh | 4 +-
hadoop-ozone/dist/src/main/license/jar-report.txt | 24 +
.../dn-one-rocksdb.robot} | 11 +-
.../dist/src/main/smoketest/ec/basic.robot | 20 +-
.../smoketest/security/ozone-secure-tenant.robot | 138 +++
.../smoketest/security/ozone-secure-token.robot | 1 +
.../dist/src/main/smoketest/upgrade/generate.robot | 31 +-
.../dist/src/main/smoketest/upgrade/validate.robot | 22 +
hadoop-ozone/dist/src/shell/ozone/ozone | 5 +
.../ozone/TestDirectoryDeletingServiceWithFSO.java | 12 +-
.../fs/ozone/TestOzoneFileSystemWithFSO.java | 45 +-
.../fs/ozone/TestRootedOzoneFileSystemWithFSO.java | 26 +
.../hdds/scm/storage/TestContainerCommandsEC.java | 101 +-
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 115 +-
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 9 +
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 3 +-
.../java/org/apache/hadoop/ozone/TestDataUtil.java | 4 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 12 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 17 +-
.../ozone/TestStorageContainerManagerHelper.java | 80 +-
.../client/rpc/TestFailureHandlingByClient.java | 14 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 16 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 27 +-
.../client/rpc/TestValidateBCSIDOnRestart.java | 18 +-
.../commandhandler/TestBlockDeletion.java | 25 +-
.../TestCloseContainerByPipeline.java | 6 +-
.../ozone/om/TestBucketLayoutWithOlderClient.java | 126 +++
.../hadoop/ozone/om/TestListKeysWithFSO.java | 497 +++++++++
.../org/apache/hadoop/ozone/om/TestListStatus.java | 210 ++++
.../ozone/om/TestOMStartupWithBucketLayout.java | 8 +-
.../hadoop/ozone/om/TestObjectStoreWithFSO.java | 34 +-
...estMultiTenantAccessAuthorizerRangerPlugin.java | 269 +++++
.../om/multitenant/TestMultiTenantVolume.java | 264 +++++
.../om/multitenant/TestRangerBGSyncService.java | 722 ++++++++++++
.../hadoop/ozone/om/multitenant/package-info.java | 34 +-
.../hadoop/ozone/scm/TestFailoverWithSCMHA.java | 93 ++
.../hadoop/ozone/shell/TestOzoneShellHA.java | 72 +-
.../ozone/shell/TestOzoneShellHAWithFSO.java | 52 +
.../hadoop/ozone/shell/TestOzoneTenantShell.java | 1085 ++++++++++++++++++
.../src/main/proto/OmClientProtocol.proto | 298 ++++-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 44 +-
.../ozone/om/codec/OmDBAccessIdInfoCodec.java | 57 +
.../ozone/om/codec/OmDBTenantStateCodec.java | 57 +
.../ozone/om/codec/OmDBUserPrincipalInfoCodec.java | 60 +
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 16 +
hadoop-ozone/ozone-manager/pom.xml | 39 +
.../hadoop/ozone/om/DirectoryDeletingService.java | 59 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 12 +-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 133 ++-
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 174 +++
.../hadoop/ozone/om/OMMultiTenantManager.java | 321 ++++++
.../hadoop/ozone/om/OMMultiTenantManagerImpl.java | 1149 ++++++++++++++++++++
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 135 ++-
.../org/apache/hadoop/ozone/om/OzoneAclUtils.java | 25 +
.../hadoop/ozone/om/OzoneListStatusHelper.java | 566 ++++++++++
.../org/apache/hadoop/ozone/om/OzoneManager.java | 393 ++++++-
.../hadoop/ozone/om/S3SecretManagerImpl.java | 14 +-
.../java/org/apache/hadoop/ozone/om/TenantOp.java | 86 ++
.../hadoop/ozone/om/TrashOzoneFileSystem.java | 4 +
.../hadoop/ozone/om/codec/OMDBDefinition.java | 38 +-
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 21 +
.../ozone/om/multitenant/AuthorizerLock.java | 87 ++
.../ozone/om/multitenant/AuthorizerLockImpl.java | 186 ++++
.../ozone/om/multitenant/CachedTenantState.java | 88 ++
.../multitenant/MultiTenantAccessAuthorizer.java | 282 +++++
.../MultiTenantAccessAuthorizerDummyPlugin.java | 194 ++++
.../MultiTenantAccessAuthorizerRangerPlugin.java | 850 +++++++++++++++
.../multitenant/MultiTenantAccessController.java | 414 +++++++
.../om/multitenant/OMRangerBGSyncService.java | 844 ++++++++++++++
.../RangerClientMultiTenantAccessController.java | 323 ++++++
.../RangerRestMultiTenantAccessController.java | 670 ++++++++++++
.../hadoop/ozone/om/multitenant/package-info.java | 22 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 3 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 82 +-
.../hadoop/ozone/om/request/OMClientRequest.java | 20 +-
.../om/request/bucket/OMBucketCreateRequest.java | 51 +-
.../om/request/bucket/OMBucketDeleteRequest.java | 34 +
.../om/request/file/OMDirectoryCreateRequest.java | 33 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 11 +-
.../ozone/om/request/file/OMFileCreateRequest.java | 35 +-
.../request/file/OMFileCreateRequestWithFSO.java | 17 +-
.../ozone/om/request/file/OMFileRequest.java | 91 +-
.../om/request/key/OMAllocateBlockRequest.java | 29 +
.../request/key/OMAllocateBlockRequestWithFSO.java | 15 +-
....java => OMDirectoriesPurgeRequestWithFSO.java} | 23 +-
.../ozone/om/request/key/OMKeyCommitRequest.java | 29 +
.../om/request/key/OMKeyCommitRequestWithFSO.java | 16 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 29 +
.../om/request/key/OMKeyCreateRequestWithFSO.java | 23 +-
.../ozone/om/request/key/OMKeyDeleteRequest.java | 35 +
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 8 +-
.../ozone/om/request/key/OMKeyRenameRequest.java | 35 +
.../om/request/key/OMKeyRenameRequestWithFSO.java | 14 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 6 +-
.../ozone/om/request/key/OMKeysDeleteRequest.java | 38 +-
.../ozone/om/request/key/OMKeysRenameRequest.java | 35 +
.../om/request/key/OmKeysDeleteRequestWithFSO.java | 25 +-
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 5 +-
.../om/request/key/acl/OMKeyAddAclRequest.java | 41 +
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 41 +
.../om/request/key/acl/OMKeySetAclRequest.java | 41 +
.../S3InitiateMultipartUploadRequest.java | 30 +
.../S3InitiateMultipartUploadRequestWithFSO.java | 14 +-
.../multipart/S3MultipartUploadAbortRequest.java | 29 +
.../S3MultipartUploadAbortRequestWithFSO.java | 15 +-
.../S3MultipartUploadCommitPartRequest.java | 29 +
.../S3MultipartUploadCommitPartRequestWithFSO.java | 13 +-
.../S3MultipartUploadCompleteRequest.java | 38 +-
.../S3MultipartUploadCompleteRequestWithFSO.java | 13 +-
.../om/request/s3/security/OMSetSecretRequest.java | 186 ++++
.../om/request/s3/security/S3GetSecretRequest.java | 153 ++-
.../request/s3/security/S3RevokeSecretRequest.java | 17 +-
.../request/s3/security/S3SecretRequestHelper.java | 107 ++
.../tenant/OMSetRangerServiceVersionRequest.java | 84 ++
.../s3/tenant/OMTenantAssignAdminRequest.java | 253 +++++
.../tenant/OMTenantAssignUserAccessIdRequest.java | 373 +++++++
.../request/s3/tenant/OMTenantCreateRequest.java | 352 ++++++
.../request/s3/tenant/OMTenantDeleteRequest.java | 247 +++++
.../s3/tenant/OMTenantRevokeAdminRequest.java | 242 +++++
.../tenant/OMTenantRevokeUserAccessIdRequest.java | 269 +++++
.../ozone/om/request/s3/tenant/package-info.java | 33 +-
.../om/request/validation/RequestValidations.java | 2 +-
.../om/request/volume/OMVolumeDeleteRequest.java | 10 +
.../ozone/om/request/volume/OMVolumeRequest.java | 4 +-
.../file/OMDirectoryCreateResponseWithFSO.java | 13 +-
.../response/file/OMFileCreateResponseWithFSO.java | 7 +-
.../key/OMDirectoriesPurgeResponseWithFSO.java | 129 +++
.../response/key/OMKeyDeleteResponseWithFSO.java | 11 +-
.../response/key/OMKeysDeleteResponseWithFSO.java | 12 +-
.../response/key/OMPathsPurgeResponseWithFSO.java | 125 ---
.../response/key/acl/OMKeyAclResponseWithFSO.java | 9 +-
.../S3InitiateMultipartUploadResponseWithFSO.java | 9 +-
...ecretResponse.java => OMSetSecretResponse.java} | 36 +-
.../response/s3/security/S3GetSecretResponse.java | 6 +
.../tenant/OMSetRangerServiceVersionResponse.java | 72 ++
.../s3/tenant/OMTenantAssignAdminResponse.java | 75 ++
.../tenant/OMTenantAssignUserAccessIdResponse.java | 105 ++
.../response/s3/tenant/OMTenantCreateResponse.java | 94 ++
.../response/s3/tenant/OMTenantDeleteResponse.java | 84 ++
.../s3/tenant/OMTenantRevokeAdminResponse.java | 75 ++
.../tenant/OMTenantRevokeUserAccessIdResponse.java | 93 ++
.../ozone/om/response/s3/tenant/package-info.java | 33 +-
.../om/upgrade/DisallowedUntilLayoutVersion.java | 2 +-
.../hadoop/ozone/om/upgrade/OMLayoutFeature.java | 3 +-
.../ozone/om/upgrade/OMLayoutFeatureAspect.java | 17 +-
.../ozone/om/upgrade/OMUpgradeFinalizer.java | 8 +-
.../protocolPB/OMAdminProtocolServerSideImpl.java | 8 +-
.../OMInterServiceProtocolServerSideImpl.java | 13 +-
...OzoneManagerProtocolServerSideTranslatorPB.java | 27 +-
.../protocolPB/OzoneManagerRequestHandler.java | 322 +++++-
.../OzoneDelegationTokenSecretManager.java | 23 +
.../hadoop/ozone/security/S3SecurityUtil.java | 18 +-
.../ozone/security/acl/OzoneNativeAuthorizer.java | 3 +
.../src/main/resources/META-INF/aop.xml | 24 +
.../hadoop/ozone/om/TestAuthorizerLockImpl.java | 156 +++
.../hadoop/ozone/om/TestOMMultiTenantManager.java | 175 +++
.../ozone/om/TestOMMultiTenantManagerImpl.java | 159 +++
.../hadoop/ozone/om/TestOmMetadataManager.java | 8 +-
.../InMemoryMultiTenantAccessController.java | 154 +++
.../TestMultiTenantAccessController.java | 397 +++++++
.../ozone/om/request/OMRequestTestUtils.java | 215 +++-
.../request/TestBucketLayoutAwareOMKeyFactory.java | 9 +-
.../request/bucket/TestOMBucketCreateRequest.java | 17 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 3 +
.../file/TestOMDirectoryCreateRequestWithFSO.java | 91 +-
.../file/TestOMFileCreateRequestWithFSO.java | 25 +-
.../key/TestOMAllocateBlockRequestWithFSO.java | 16 +-
.../om/request/key/TestOMKeyAclRequestWithFSO.java | 8 +-
.../om/request/key/TestOMKeyCommitRequest.java | 6 +-
.../request/key/TestOMKeyCommitRequestWithFSO.java | 12 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 44 +-
.../request/key/TestOMKeyDeleteRequestWithFSO.java | 8 +-
.../request/key/TestOMOpenKeysDeleteRequest.java | 57 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 24 +-
.../TestS3MultipartUploadAbortRequest.java | 2 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 10 +-
.../TestS3MultipartUploadCommitPartRequest.java | 5 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 18 +-
...estS3MultipartUploadCompleteRequestWithFSO.java | 26 +-
.../s3/security/TestS3GetSecretRequest.java | 450 ++++++++
.../tenant/TestSetRangerServiceVersionRequest.java | 110 ++
.../file/TestOMDirectoryCreateResponseWithFSO.java | 51 +-
.../file/TestOMFileCreateResponseWithFSO.java | 8 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 5 +-
.../om/response/key/TestOMKeyCommitResponse.java | 4 +-
.../key/TestOMKeyCommitResponseWithFSO.java | 18 +-
.../key/TestOMKeyCreateResponseWithFSO.java | 8 +-
.../om/response/key/TestOMKeyDeleteResponse.java | 20 +-
.../key/TestOMKeyDeleteResponseWithFSO.java | 5 +-
.../ozone/om/response/key/TestOMKeyResponse.java | 32 +-
.../response/key/TestOMOpenKeysDeleteResponse.java | 14 +-
...stS3InitiateMultipartUploadResponseWithFSO.java | 10 +-
.../s3/multipart/TestS3MultipartResponse.java | 64 +-
.../TestS3MultipartUploadAbortResponse.java | 8 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 13 +-
...S3MultipartUploadCommitPartResponseWithFSO.java | 26 +-
...stS3MultipartUploadCompleteResponseWithFSO.java | 33 +-
.../om/upgrade/TestOMLayoutFeatureAspect.java | 1 +
.../ozone/om/upgrade/TestOMUpgradeFinalizer.java | 7 +-
.../TestOzoneDelegationTokenSecretManager.java | 40 +
.../apache/hadoop/fs/ozone/OzoneClientUtils.java | 18 +-
.../hadoop/fs/ozone/TestOzoneClientUtils.java | 32 +
.../hadoop/ozone/recon/api/NSSummaryEndpoint.java | 31 +-
.../scm/ReconStorageContainerManagerFacade.java | 2 +-
.../spi/impl/OzoneManagerServiceProviderImpl.java | 4 +-
.../webapps/recon/ozone-recon-web/package.json | 2 +-
.../webapps/recon/ozone-recon-web/pnpm-lock.yaml | 34 +-
.../ozone/recon/OMMetadataManagerTestUtils.java | 15 +-
.../ozone/recon/api/TestNSSummaryEndpoint.java | 26 +-
.../impl/TestOzoneManagerServiceProviderImpl.java | 11 +-
.../ozone/recon/tasks/TestNSSummaryTask.java | 18 +-
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 6 +-
.../hadoop/ozone/s3/OzoneClientProducer.java | 8 +-
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 11 +-
.../hadoop/ozone/client/ObjectStoreStub.java | 10 +-
.../hadoop/ozone/client/OzoneBucketStub.java | 8 +-
.../hadoop/ozone/s3/TestS3GatewayAuditLog.java | 6 +-
.../s3/endpoint/TestMultipartUploadWithCopy.java | 6 +-
.../hadoop/ozone/s3/endpoint/TestObjectHead.java | 6 +-
.../ozone/s3/endpoint/TestPermissionCheck.java | 12 +-
.../hadoop/ozone/debug/DBDefinitionFactory.java | 14 +-
.../org/apache/hadoop/ozone/debug/DBScanner.java | 3 +-
.../apache/hadoop/ozone/debug/PrefixParser.java | 29 +-
.../ozone/debug/container/ContainerCommands.java | 3 +-
.../containergenerator/GeneratorDatanode.java | 8 +-
.../ozone/shell/bucket/CreateBucketHandler.java | 36 +-
.../hadoop/ozone/shell/s3/GetS3SecretHandler.java | 13 +-
.../org/apache/hadoop/ozone/shell/s3/S3Shell.java | 1 +
...3SecretHandler.java => SetS3SecretHandler.java} | 35 +-
.../ozone/shell/tenant/GetUserInfoHandler.java | 110 ++
.../shell/tenant/TenantAssignAdminHandler.java | 68 ++
.../tenant/TenantAssignUserAccessIdHandler.java | 83 ++
.../shell/tenant/TenantBucketLinkHandler.java | 74 ++
.../ozone/shell/tenant/TenantCreateHandler.java | 55 +
.../ozone/shell/tenant/TenantDeleteHandler.java | 73 ++
.../ozone/shell/tenant/TenantGetSecretHandler.java | 53 +
.../hadoop/ozone/shell/tenant/TenantHandler.java | 54 +
.../ozone/shell/tenant/TenantListHandler.java | 72 ++
.../ozone/shell/tenant/TenantListUsersHandler.java | 81 ++
.../shell/tenant/TenantRevokeAdminHandler.java | 61 ++
.../tenant/TenantRevokeUserAccessIdHandler.java | 45 +
.../ozone/shell/tenant/TenantSetSecretHandler.java | 55 +
.../{s3/S3Shell.java => tenant/TenantShell.java} | 30 +-
.../ozone/shell/tenant/TenantUserCommands.java | 76 ++
.../hadoop/ozone/shell/tenant/package-info.java | 21 +
.../ozone/debug/TestDBDefinitionFactory.java | 13 +-
pom.xml | 2 +-
523 files changed, 35345 insertions(+), 4583 deletions(-)
copy
hadoop-hdds/{container-service/src/main/resources/webapps/hddsDatanode/dn.js =>
common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeFinalizationExecutor.java}
(60%)
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/TestOzoneConsts.java
create mode 100644
hadoop-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/ConfigurationSourceTest.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/DBHandle.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/DatanodeStoreCache.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/RawDB.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/DbVolume.java
copy
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/{MetadataVolumeFactory.java
=> DbVolumeFactory.java} (64%)
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoMetrics.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeSchemaThreeDBDefinition.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeStoreSchemaThreeImpl.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DeleteTransactionStore.java
create mode 100644
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeSchemaV3FinalizeAction.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestDatanodeStoreCache.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaTwoBackwardsCompatibility.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/utils/TestHddsVolumeUtil.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/utils/TestStorageVolumeUtil.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestDbVolume.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestStorageVolume.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/ContainerTestVersionInfo.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDatanodeUpgradeToSchemaV3.java
create mode 100644 hadoop-hdds/docs/content/design/dn-merge-rocksdb.md
create mode 100644
hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Access-Control.md
create mode 100644 hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Setup.md
create mode 100644 hadoop-hdds/docs/content/feature/S3-Multi-Tenancy.md
create mode 100644 hadoop-hdds/docs/content/feature/S3-Tenant-Commands.md
create mode 100644 hadoop-hdds/docs/content/feature/dn-merge-rocksdb.md
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/{StringCodec.java
=> DumpFileLoader.java} (58%)
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/{StringCodec.java
=> DumpFileWriter.java} (59%)
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/{StringCodec.java
=> FixedLengthStringCodec.java} (81%)
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/FixedLengthStringUtils.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBSstFileLoader.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBSstFileWriter.java
create mode 100644
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RocksDatabase.java
copy
hadoop-hdds/{server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/IllegalContainerBalancerStateException.java
=>
framework/src/test/java/org/apache/hadoop/hdds/utils/db/TestFixedLengthStringUtils.java}
(51%)
rename
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/{pipeline/BackgroundPipelineScrubber.java
=> ha/BackgroundSCMService.java} (53%)
copy
hadoop-hdds/{framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMServiceException.java}
(55%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/DisallowedUntilLayoutVersion.java
=>
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/io/ByteStringCodec.java
(55%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/DisallowedUntilLayoutVersion.java
=>
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/io/IntegerCodec.java
(56%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/FinalizationCheckpoint.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/FinalizationManager.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/FinalizationManagerImpl.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/FinalizationStateManager.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/FinalizationStateManagerImpl.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/SCMUpgradeFinalizationContext.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/ha/TestBackgroundSCMService.java
delete mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestBackgroundPipelineScrubber.java
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/FinalizationManagerTestImpl.java
copy
hadoop-hdds/{container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolume.java
=>
server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/FinalizationStateManagerTestImpl.java}
(53%)
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/TestScmFinalization.java
create mode 100644
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/TenantArgs.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/DeleteTenantState.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDBAccessIdInfo.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDBTenantState.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDBUserPrincipalInfo.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmRangerSyncArgs.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmTenantArgs.java
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmTenantUserArgs.java
(55%)
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/S3VolumeContext.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/TenantStateList.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/TenantUserInfoValue.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/TenantUserList.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/AccessPolicy.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/AccountNameSpace.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/BucketNameSpace.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/OzoneOwnerPrincipal.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/OzoneTenant.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/OzoneTenantRolePrincipal.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/RangerAccessPolicy.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/Tenant.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/impl/AccountNameSpaceImpl.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/impl/SingleVolumeTenantNamespace.java
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/impl/package-info.java
(54%)
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/multitenant/package-info.java
(54%)
create mode 100644
hadoop-ozone/dist/src/main/compose/ozonesecure/mockserverInitialization.json
copy hadoop-ozone/dist/src/main/smoketest/{upgrade/validate.robot =>
compatibility/dn-one-rocksdb.robot} (74%)
create mode 100644
hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-tenant.robot
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestBucketLayoutWithOlderClient.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestListKeysWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestListStatus.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/multitenant/TestMultiTenantAccessAuthorizerRangerPlugin.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/multitenant/TestMultiTenantVolume.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/multitenant/TestRangerBGSyncService.java
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/multitenant/package-info.java
(54%)
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHAWithFSO.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneTenantShell.java
create mode 100644
hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/codec/OmDBAccessIdInfoCodec.java
create mode 100644
hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/codec/OmDBTenantStateCodec.java
create mode 100644
hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/codec/OmDBUserPrincipalInfoCodec.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMMultiTenantManager.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMMultiTenantManagerImpl.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneListStatusHelper.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/TenantOp.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/AuthorizerLock.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/AuthorizerLockImpl.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/CachedTenantState.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/MultiTenantAccessAuthorizer.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/MultiTenantAccessAuthorizerDummyPlugin.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/MultiTenantAccessAuthorizerRangerPlugin.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/MultiTenantAccessController.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/OMRangerBGSyncService.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/RangerClientMultiTenantAccessController.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/RangerRestMultiTenantAccessController.java
copy
hadoop-hdds/container-service/src/main/resources/webapps/hddsDatanode/dn.js =>
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/multitenant/package-info.java
(60%)
rename
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/{OMPathsPurgeRequestWithFSO.java
=> OMDirectoriesPurgeRequestWithFSO.java} (69%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/security/OMSetSecretRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/security/S3SecretRequestHelper.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMSetRangerServiceVersionRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMTenantAssignAdminRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMTenantAssignUserAccessIdRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMTenantCreateRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMTenantDeleteRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMTenantRevokeAdminRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/OMTenantRevokeUserAccessIdRequest.java
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/tenant/package-info.java
(54%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMDirectoriesPurgeResponseWithFSO.java
delete mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMPathsPurgeResponseWithFSO.java
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/security/{S3GetSecretResponse.java
=> OMSetSecretResponse.java} (66%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMSetRangerServiceVersionResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMTenantAssignAdminResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMTenantAssignUserAccessIdResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMTenantCreateResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMTenantDeleteResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMTenantRevokeAdminResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/OMTenantRevokeUserAccessIdResponse.java
copy
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/StringCodec.java
=>
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/tenant/package-info.java
(54%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/resources/META-INF/aop.xml
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestAuthorizerLockImpl.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOMMultiTenantManager.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOMMultiTenantManagerImpl.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/multitenant/InMemoryMultiTenantAccessController.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/multitenant/TestMultiTenantAccessController.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/security/TestS3GetSecretRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/tenant/TestSetRangerServiceVersionRequest.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/s3/{GetS3SecretHandler.java
=> SetS3SecretHandler.java} (66%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/GetUserInfoHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantAssignAdminHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantAssignUserAccessIdHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantBucketLinkHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantCreateHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantDeleteHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantGetSecretHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantListHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantListUsersHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantRevokeAdminHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantRevokeUserAccessIdHandler.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantSetSecretHandler.java
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/{s3/S3Shell.java
=> tenant/TenantShell.java} (67%)
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantUserCommands.java
create mode 100644
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/package-info.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
