This is an automated email from the ASF dual-hosted git repository.
rakeshr pushed a change to branch HDDS-2939
in repository https://gitbox.apache.org/repos/asf/ozone.git.
discard d74b856 HDDS-4332: ListFileStatus - do lookup in directory and file
tables (#1503)
omit 51bb909 HDDS-4266: CreateFile : store parent dir entries into
DirTable and file entry into separate FileTable (#1473)
omit 6f2fa22 HDDS-2949: mkdir : store directory entries in a separate
table (#1404)
add 2650723 HDDS-4343. ReplicationManager.handleOverReplicatedContainer()
does not handle unhealthyReplicas properly. (#1495)
add 814428e HDDS-3995. Fix s3g met NPE exception while write file by
multiPartUpload (#1499)
add 342bf6d HDDS-4327. Potential resource leakage using BatchOperation.
(#1493)
add aff7c52 HDDS-4269. Ozone DataNode thinks a volume is failed if an
unexpected file is in the HDDS root directory. (#1490)
add 8fab5f2 HDDS-2922. Balance ratis leader distribution in datanodes
(#1371)
add 25e2046 HDDS-4297. Allow multiple transactions per container to be
sent for deletion by SCM.
add 9baaf64 HDDS-4158. Provide a class type for Java based configuration
(#1407)
add 049793d HDDS-4301. SCM CA certificate does not encode KeyUsage
extension properly (#1468)
add 10df489 HDDS-4328. Provide fallback cache restore key (#1505)
add 4e1d2ef HDDS-4306. Ozone checkstyle rule can't be imported to
IntelliJ. (#1472)
add 6ba64bf HDDS-4359. Expose VolumeIOStats in DN JMX (#1506)
add c1069a6 HDDS-4362. Change hadoop32 test to use 3.2 image (#1521)
add 6112603 HDDS-4088. Adding Owner info for Authorizer plugin to honor
owner access rights (#1395)
add 808aabd HDDS-3882. Update modification time when updating
volume/bucket/key ACLs (#1301)
add 18711f3 HDDS-4388. Make writeStateMachineTimeout retry count
proportional to node failure timeout (#1519)
add 5e5635f HDDS-4386: Each EndpointStateMachine uses its own thread pool
to talk with SCM/Recon (#1518)
add c67aa2f HDDS-3731. [doc]add storage space quota document. (#1516)
add 3861e77 HDDS-4117. Normalize Keypath for listKeys. (#1451)
add a9c3d8c HDDS-4405. Proxy failover is logging with out trying all OMS.
(#1531)
add df43d85 HDDS-4366. SCM deletion service should delete configured
number of blocks every interval. (#1513)
add b42449e HDDS-4400. Make raft log directory deletion configurable
during pipeline remove. (#1527)
add 1ac1e24 HDDS-3959. Avoid HddsProtos.PipelineID#toString (#1525)
add d8f4dc1 HDDS-4345. Replace the deprecated Lock method (#1497)
add 8d3d6e4 HDDS-4391. UnixPath.toUri() is expensive. (#1536)
new 88a4108 HDDS-2949: mkdir : store directory entries in a separate
table (#1404)
new da1cb75 HDDS-4266: CreateFile : store parent dir entries into
DirTable and file entry into separate FileTable (#1473)
new 181bb34 HDDS-4332: ListFileStatus - do lookup in directory and file
tables (#1503)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d74b856)
\
N -- N -- N refs/heads/HDDS-2939 (181bb34)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/post-commit.yml | 3 +
.../hadoop/hdds/scm/storage/CommitWatcher.java | 2 +-
.../org/apache/hadoop/hdds/ratis/RatisHelper.java | 19 ++
.../java/org/apache/hadoop/hdds/scm/ScmConfig.java | 20 ++
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 3 +
.../hadoop/hdds/scm/exceptions/SCMException.java | 3 +-
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 42 ++-
.../common/src/main/resources/ozone-default.xml | 18 ++
.../hadoop/hdds/conf/SimpleConfiguration.java | 13 +
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 7 +-
.../org/apache/hadoop/hdds/conf/ConfigTag.java | 3 +-
.../org/apache/hadoop/hdds/conf/ConfigType.java | 3 +-
.../hdds/conf/ConfigurationReflectionUtil.java | 12 +
.../common/statemachine/EndpointStateMachine.java | 16 ++
.../ClosePipelineCommandHandler.java | 15 +-
.../CreatePipelineCommandHandler.java | 34 ++-
.../commandhandler/DeleteBlocksCommandHandler.java | 65 ++---
.../states/datanode/RunningDatanodeState.java | 8 +-
.../common/transport/server/XceiverServerSpi.java | 10 +-
.../transport/server/ratis/XceiverServerRatis.java | 64 ++++-
.../container/common/utils/HddsVolumeUtil.java | 5 +
.../ozone/container/common/volume/HddsVolume.java | 8 +-
.../container/common/volume/VolumeIOStats.java | 72 +++--
.../container/keyvalue/impl/BlockManagerImpl.java | 70 ++---
.../background/BlockDeletingService.java | 45 ++--
.../protocol/commands/CreatePipelineCommand.java | 62 ++++-
.../TestCreatePipelineCommandHandler.java | 7 +-
hadoop-hdds/dev-support/checkstyle/checkstyle.xml | 4 +-
hadoop-hdds/docs/content/feature/Quota.md | 74 +++++
hadoop-hdds/docs/content/feature/Quota.zh.md | 67 +++++
.../hdds/conf/DatanodeRatisServerConfig.java | 17 ++
.../certificates/utils/CertificateSignRequest.java | 2 +-
.../certificates/utils/SelfSignedCertificate.java | 4 +-
.../interface-client/src/main/proto/hdds.proto | 1 +
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 1 +
.../src/main/proto/ScmServerProtocol.proto | 1 +
.../block/DatanodeDeletedBlockTransactions.java | 132 +++------
.../hadoop/hdds/scm/block/DeletedBlockLog.java | 4 +-
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 73 +++--
.../hdds/scm/block/SCMBlockDeletingService.java | 130 ++++-----
.../hdds/scm/container/ReplicationManager.java | 3 +-
.../hdds/scm/container/SCMContainerManager.java | 28 +-
.../hdds/scm/pipeline/PipelineStateManager.java | 3 +-
.../hdds/scm/pipeline/RatisPipelineProvider.java | 29 +-
.../algorithms/DefaultLeaderChoosePolicy.java} | 26 +-
.../choose/algorithms/LeaderChoosePolicy.java | 55 ++++
.../algorithms/LeaderChoosePolicyFactory.java | 75 ++++++
.../algorithms/MinLeaderCountChoosePolicy.java | 91 +++++++
.../leader/choose/algorithms}/package-info.java | 6 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 100 +++----
.../choose/algorithms/TestLeaderChoosePolicy.java | 74 +++++
hadoop-ozone/common/pom.xml | 5 +
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 39 +++
.../ozone/om/ha/OMFailoverProxyProvider.java | 2 +-
...OzoneManagerProtocolClientSideTranslatorPB.java | 4 +-
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 10 +
.../hadoop/ozone/security/acl/RequestContext.java | 48 +++-
.../hadoop/ozone/om/lock/TestOzoneManagerLock.java | 52 ++--
.../dist/src/main/compose/ozone-mr/hadoop32/.env | 6 +-
.../dist/src/main/compose/ozonesecure-mr/.env | 3 +-
.../compose/ozonesecure-mr/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozonesecure-mr/test.sh | 1 +
.../fs/ozone/TestOzoneFSWithObjectStoreCreate.java | 71 ++++-
.../hdds/scm/pipeline/TestLeaderChoosePolicy.java | 216 +++++++++++++++
.../TestRatisPipelineCreateAndDestroy.java | 2 +-
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 4 +-
.../commandhandler/TestBlockDeletion.java | 7 +-
.../container/metrics/TestContainerMetrics.java | 14 +
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 9 +-
.../ozone/om/TestOzoneManagerListVolumes.java | 36 +--
.../ozone/om/TestOzoneManagerRocksDBLogging.java | 2 +-
.../src/main/proto/OmClientProtocol.proto | 3 +
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 32 +--
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 67 +++--
.../org/apache/hadoop/ozone/om/OzoneManager.java | 87 ++++--
.../apache/hadoop/ozone/om/PrefixManagerImpl.java | 24 +-
.../hadoop/ozone/om/S3SecretManagerImpl.java | 8 +-
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 36 +--
.../hadoop/ozone/om/request/OMClientRequest.java | 46 ++--
.../om/request/bucket/acl/OMBucketAclRequest.java | 15 ++
.../request/bucket/acl/OMBucketAddAclRequest.java | 15 ++
.../bucket/acl/OMBucketRemoveAclRequest.java | 15 ++
.../request/bucket/acl/OMBucketSetAclRequest.java | 15 ++
.../ozone/om/request/key/OMKeyCommitRequest.java | 4 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 34 +++
.../ozone/om/request/key/OMKeysDeleteRequest.java | 4 +-
.../ozone/om/request/key/OMKeysRenameRequest.java | 9 +-
.../ozone/om/request/key/acl/OMKeyAclRequest.java | 15 ++
.../om/request/key/acl/OMKeyAddAclRequest.java | 15 ++
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 15 ++
.../om/request/key/acl/OMKeySetAclRequest.java | 15 ++
.../request/key/acl/prefix/OMPrefixAclRequest.java | 7 +
.../hadoop/ozone/om/request/util/ObjectParser.java | 6 +-
.../om/request/volume/OMVolumeSetOwnerRequest.java | 2 +-
.../om/request/volume/acl/OMVolumeAclRequest.java | 14 +
.../request/volume/acl/OMVolumeAddAclRequest.java | 15 ++
.../volume/acl/OMVolumeRemoveAclRequest.java | 15 ++
.../request/volume/acl/OMVolumeSetAclRequest.java | 15 ++
.../ozone/security/acl/OzoneNativeAuthorizer.java | 44 ++-
.../ozone/om/request/TestOMRequestUtils.java | 60 +++++
.../bucket/acl/TestOMBucketAddAclRequest.java | 119 ++++++++
.../bucket/acl/TestOMBucketRemoveAclRequest.java | 137 ++++++++++
.../bucket/acl/TestOMBucketSetAclRequest.java | 125 +++++++++
.../ozone/om/request/bucket/acl/package-info.java | 4 +-
.../ozone/om/request/key/TestOMKeyAclRequest.java | 155 ++++++++++-
.../key/TestOMKeyPurgeRequestAndResponse.java | 15 +-
.../volume/acl/TestOMVolumeAddAclRequest.java | 7 +
.../volume/acl/TestOMVolumeRemoveAclRequest.java | 8 +
.../volume/acl/TestOMVolumeSetAclRequest.java | 7 +
.../bucket/TestOMBucketCreateResponse.java | 8 +
.../bucket/TestOMBucketDeleteResponse.java | 8 +
.../bucket/TestOMBucketSetPropertyResponse.java | 9 +
.../file/TestOMDirectoryCreateResponse.java | 8 +
.../ozone/om/response/key/TestOMKeyResponse.java | 3 +
.../s3/multipart/TestS3MultipartResponse.java | 9 +
.../security/TestOMDelegationTokenResponse.java | 8 +
.../volume/TestOMVolumeCreateResponse.java | 8 +
.../volume/TestOMVolumeDeleteResponse.java | 8 +
.../volume/TestOMVolumeSetOwnerResponse.java | 9 +
.../volume/TestOMVolumeSetQuotaResponse.java | 9 +
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 298 +++++++++++++++++++++
.../recon/tasks/TestContainerKeyMapperTask.java | 2 +-
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 5 +-
pom.xml | 4 +-
124 files changed, 3113 insertions(+), 680 deletions(-)
create mode 100644 hadoop-hdds/docs/content/feature/Quota.md
create mode 100644 hadoop-hdds/docs/content/feature/Quota.zh.md
copy
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/{choose/algorithms/RandomPipelineChoosePolicy.java
=> leader/choose/algorithms/DefaultLeaderChoosePolicy.java} (54%)
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/leader/choose/algorithms/LeaderChoosePolicy.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/leader/choose/algorithms/LeaderChoosePolicyFactory.java
create mode 100644
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/leader/choose/algorithms/MinLeaderCountChoosePolicy.java
copy hadoop-hdds/{client/src/test/java/org/apache/hadoop/hdds/scm/storage =>
server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/leader/choose/algorithms}/package-info.java
(88%)
create mode 100644
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/leader/choose/algorithms/TestLeaderChoosePolicy.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestLeaderChoosePolicy.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/acl/TestOMBucketAddAclRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/acl/TestOMBucketRemoveAclRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/acl/TestOMBucketSetAclRequest.java
copy hadoop-ozone/ozone-manager/src/{main =>
test}/java/org/apache/hadoop/ozone/om/request/bucket/acl/package-info.java (92%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestVolumeOwner.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
