This is an automated email from the ASF dual-hosted git repository.
smengcl pushed a change to branch HDDS-13323-sts
in repository https://gitbox.apache.org/repos/asf/ozone.git
from 3a56b7faa35 HDDS-15325. [STS] Polaris Smoke Test (#10315)
add af3e34377b0 HDDS-16076. Separate test class for
testInstallSnapshotDuringBootstrapping. (#10931)
add 0a9bd0d999a HDDS-16067. Intermittent timeout in
testNodeWithOpenPipelineCanBeDecommissionedAndRecommissioned (#10939)
add 5425ee3f86c HDDS-15781. Adding Spankind information to ozone sh
commands (#10695)
add 0110ff57df4 HDDS-15396. Grafana dashboard for disk balancer (#10609).
add bd0f23b49ba HDDS-16119. Fix DatanodeStorageMetrics metrics-system
deadlock and mini-cluster source leak (#10988)
add 5be6afae7af HDDS-15389. Top-level delete entry retention utilities for
snapshot diff. (#10750)
add 5e99e226bb0 HDDS-16081. TestOMRatisSnapshots fails due to leader
applied index taken before flush (#10959)
add 09a7d615dec HDDS-15982. Fix UnsupportedOperationException and add
idempotent deletion in RangerClientMultiTenantAccessController (#10872)
add 07f069b72fc HDDS-16015. Make listStatusLight skip SCM pipeline
refresh. (#10894)
add 6384c643fb3 HDDS-14742. Clean up orphan files from a terminated
compaction. (#10821)
add b4a4227cf4f HDDS-14876. Make snapshot diff progress more granular
(#10901)
add 776ce13a798 HDDS-16113. Add regression tests for snapshot
defragmentation cleanup (#10978)
add a1ce3b9d900 HDDS-15774. Make SCM Ratis roles string parsing IPv6-safe
(#10831)
add 5dfed2ce426 HDDS-15775. Support IPv6 literal authorities in OzoneFS
URI parsing (#10933)
add 752e96c2fb3 HDDS-15776. Make S3 Gateway VirtualHostStyleFilter
IPv6-safe (#10809)
add 0f812bcbb30 HDDS-15542. Closed-container reconciliation advances BCSID
past a hole, masking missing chunks (#10496)
add 9065c84364d HDDS-15761. Create test for OMMXBean (#10960)
add 59351d77aba HDDS-16114. Avoid misleading Ozone mount warning in
acceptance test results (#10979)
add 1db063b63ce HDDS-16072. Implement GetObjectAttributes: ETag,
ObjectSize, StorageClass, and ObjectParts count (#10930).
add 291633510ce HDDS-15961. Resolve linked bucket source properties
consistently (#10859)
add a7f5cbc832c HDDS-16117. Failed FSO multipart complete leaks bucket
namespace quota (#10984)
add 2fbc910aa29 HDDS-11620. Log SCM finalization completion (#10963)
add 6a4513ddaec HDDS-15826. Recon: add cycle guard to NSSummary /du tree
walks (#10723)
add 5672fc71c5d HDDS-8082. Check for unnecessary newlines in config
defaults (#10840)
add 7f68eb6f36d HDDS-16125. Copy bucket snapshot in quota repair response
to fix cache aliasing (#11004)
add bc21296f297 HDDS-16118. Release snapshot DB handles before
KeyDeletingService submits OM requests (#10987)
add 7693b1979f8 HDDS-15895. Make ServerNotLeaderException suggested leader
flow IPv6-safe (#10807)
add 0c349a3f541 HDDS-15356. Make multi-buffer chunk checksum
allocation-free (#10350)
add ce13cdece18 HDDS-15911. Recon AI Assistant: add conversation memory
for follow-up questions. (#10808)
add 749da668af1 HDDS-16158. Fix sendDeleteKeysRequestAndClearList
Authorization and FSO Trash Handling (#10996)
add fb2cc733688 HDDS-16169. Add old updateContainerState to handle upgrade
path. (#11012)
add 711476cb0c4 HDDS-16089. Improve MPU table evaluation efficiency in
KeyLifeCycleService (#10949)
add 50978e3ad7c HDDS-15422. Stream read seek should not close stream
(#10415)
add c0c3241ff57 HDDS-16115. Reject invalid lifecycle rule Status values
(#10980)
add 1a4486a271b HDDS-16088. Reject lifecycle
AbortIncompleteMultipartUpload rules whose daysAfterInitiation exceeds the MPU
cleanup threshold (#10982)
add 32b6c97db1f HDDS-16071. Add S3 Bucket Lifecycle CRUD integration tests
to AbstractS3SDKV2Tests (#10986)
add a552e060433 HDDS-16183. Avoid the per-group list copy in
OMKeyRequest.sumBlockLengths (#11026)
add db3fff45c51 HDDS-16198. Refactor the admin permission check for
LifecycleService. (#11034)
add 6a903990741 Improve listSnapshotDiffJobs
add b58885e7baf Adding OM tests.
add 54881878681 Add missing check for GetContainerChecksumInfo.
add afa7f0e667c HDDS-13242. getTrashRoots should not be bounded by
ozone.fs.listing.page.size.max (#11002)
add 3af0d141eb1 HDDS-16197. Remove all bucket tagging outstanding TODOs
(#11036)
add a9b67b11ccb HDDS-16150. Fix error code for invalid S3 Lifecycle
Expiration Days from InvalidRequest to InvalidArgument (#11003)
add 2c02f432d30 HDDS-12208. Give `ozone debug replicas verify` options for
OM container location cache behavior. (#11015)
new 2b5240bf92f Merge remote-tracking branch 'upstream/master' into
HDDS-13323-sts
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
hadoop-hdds/client/pom.xml | 4 +
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 5 +-
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 3 +-
.../hdds/scm/storage/MultipartInputStream.java | 4 +-
.../hdds/scm/storage/StreamBlockInputStream.java | 186 ++--
.../java/org/apache/hadoop/hdds/HddsUtils.java | 64 ++
.../hdds/ratis/ServerNotLeaderException.java | 5 +-
.../apache/hadoop/hdds/scm/net/HostAndPort.java | 3 +-
.../hadoop/hdds/tracing/GrpcServerInterceptor.java | 4 +-
.../{SkipTracing.java => InternalSpanKind.java} | 6 +-
.../apache/hadoop/hdds/tracing/TraceAllMethod.java | 9 +-
.../apache/hadoop/hdds/tracing/TracingUtil.java | 56 +-
.../org/apache/hadoop/ozone/common/Checksum.java | 186 ++--
.../apache/hadoop/ozone/common/ChecksumCache.java | 117 ++-
.../hadoop/ozone/common/utils/BufferUtils.java | 23 +
.../common/src/main/resources/ozone-default.xml | 16 +-
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 62 ++
...TestServerNotLeaderExceptionMessageParsing.java | 22 +
.../hadoop/hdds/scm/net/TestHostAndPort.java | 6 +
.../hadoop/hdds/tracing/TestTraceAllMethod.java | 20 +
.../hdds/tracing/TestTracingAnnotations.java | 88 ++
.../hadoop/hdds/tracing/TestTracingUtil.java | 77 +-
.../apache/hadoop/ozone/common/TestChecksum.java | 17 +
.../hadoop/ozone/common/TestChecksumCache.java | 73 +-
.../ozone/common/TestChecksumMultiBuffer.java | 97 ++
.../hadoop/ozone/common/utils/TestBufferUtils.java | 81 ++
.../checksum/ContainerChecksumTreeManager.java | 12 +-
.../container/checksum/ContainerDiffReport.java | 13 +
.../common/transport/server/XceiverServerGrpc.java | 4 +-
.../transport/server/ratis/XceiverServerRatis.java | 4 +-
.../common/volume/DatanodeStorageMetrics.java | 39 +-
.../container/common/volume/MutableVolumeSet.java | 37 +
.../ozone/container/keyvalue/KeyValueHandler.java | 177 +++-
.../container/keyvalue/impl/BlockManagerImpl.java | 10 +
.../keyvalue/interfaces/BlockManager.java | 10 +
.../common/volume/TestDatanodeStorageMetrics.java | 52 +-
.../container/common/volume/TestVolumeSet.java | 27 +
.../TestReconcileChunksPerBlockHoleBcsId.java | 522 ++++++++++
.../keyvalue/impl/TestFilePerBlockStrategy.java | 10 +-
.../docs/content/tools/debug/DebugReplicas.md | 6 +
.../scm/proxy/SCMFailoverProxyProviderBase.java | 95 +-
.../server/OzoneProtocolMessageDispatcher.java | 3 +-
.../TestSCMFailoverProxyProviderRefreshWired.java | 167 ++-
.../ozone/rocksdiff/RocksDBCheckpointDiffer.java | 48 +-
.../rocksdiff/TestRocksDBCheckpointDiffer.java | 55 +-
.../hdds/scm/container/ContainerStateManager.java | 6 +
.../scm/container/ContainerStateManagerImpl.java | 30 +
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 31 +-
.../ha/invoker/ContainerStateManagerInvoker.java | 25 +-
.../hdds/scm/server/StorageContainerManager.java | 2 +-
.../scm/server/upgrade/SCMUpgradeFinalizer.java | 1 +
.../hadoop/hdds/scm/ha/TestSCMRatisServerImpl.java | 62 ++
.../hdds/scm/upgrade/TestScmFinalization.java | 50 +-
.../hdds/scm/cli/SafeModeCheckSubcommand.java | 49 +-
.../admin/scm/GetScmRatisRolesSubcommand.java | 51 +-
.../ozone/scm/TestGetScmRatisRolesSubcommand.java | 65 ++
.../ozone/debug/replicas/ReplicasVerify.java | 7 +-
.../ozone/debug/replicas/TestReplicasVerify.java | 54 +
hadoop-ozone/cli-shell/pom.xml | 4 +
.../java/org/apache/hadoop/ozone/shell/Shell.java | 3 +-
.../ozone/shell/snapshot/SnapshotDiffHandler.java | 7 +
.../hadoop/ozone/client/OzoneClientFactory.java | 5 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 6 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 11 +-
.../hadoop/ozone/om/helpers/SnapshotDiffJob.java | 3 +-
...OzoneManagerProtocolClientSideTranslatorPB.java | 14 +-
.../ozone/snapshot/SnapshotDiffResponse.java | 28 +-
.../java/org/apache/hadoop/ozone/TestOmUtils.java | 19 +
.../hadoop/ozone/om/helpers/TestOmBucketInfo.java | 21 +
.../ozone/snapshot/TestSnapshotDiffResponse.java | 40 +-
.../grafana/dashboards/Ozone - Disk Balancer.json | 1096 ++++++++++++++++++++
hadoop-ozone/dist/src/main/smoketest/testlib.sh | 3 +-
.../ozone/s3/awssdk/v1/AbstractS3SDKV1Tests.java | 4 +-
.../ozone/s3/awssdk/v2/AbstractS3SDKV2Tests.java | 257 ++++-
.../hadoop/conf/ConfigurationFieldsTests.java | 1 -
.../scm/node/TestDecommissionAndMaintenance.java | 7 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 58 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 58 ++
.../rpc/read/TestStreamBlockInputStream.java | 56 +-
.../ozone/client/rpc/read/TestStreamRead.java | 1 +
.../dn/TestDatanodeStorageMetricsIntegration.java | 9 +-
.../org/apache/hadoop/ozone/om/OMMXBeanTests.java | 88 ++
.../ozone/om/TestGetDBUpdatesAuthorization.java | 137 +++
.../TestOMInstallSnapshotDuringBootstrapping.java | 207 ++++
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 172 +--
.../hadoop/ozone/om/snapshot/OmSnapshotTests.java | 26 +
.../java/org/apache/ozone/test/NonHATests.java | 9 +
.../src/main/proto/OmClientProtocol.proto | 21 +-
.../interface-client/src/main/resources/proto.lock | 18 +
.../RangerClientMultiTenantAccessController.java | 98 +-
...lientMultiTenantAccessControllerMockClient.java | 153 +++
.../org/apache/hadoop/ozone/om/KeyManager.java | 28 +
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 33 +-
.../apache/hadoop/ozone/om/OmMetadataReader.java | 37 +-
.../org/apache/hadoop/ozone/om/OmSnapshot.java | 32 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 57 +-
.../hadoop/ozone/om/TrashOzoneFileSystem.java | 34 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 14 +-
.../ozone/om/request/key/OMKeysDeleteRequest.java | 2 +-
.../OMLifecycleConfigurationSetRequest.java | 50 +-
.../lifecycle/OMLifecycleSaveScanStateRequest.java | 2 +-
.../OMLifecycleSetServiceStatusRequest.java | 2 +-
.../S3MultipartUploadCompleteRequest.java | 7 +
.../om/request/volume/OMQuotaRepairRequest.java | 9 +-
.../ozone/om/service/KeyDeletingService.java | 15 +-
.../ozone/om/service/KeyLifecycleService.java | 187 ++--
.../ozone/om/snapshot/SnapshotDiffManager.java | 168 ++-
.../diff/delta/CompositeDeltaDiffComputer.java | 3 +
.../om/snapshot/filter/ReclaimableFilter.java | 15 +-
.../protocolPB/OzoneManagerRequestHandler.java | 6 +
.../hadoop/ozone/om/TestBucketManagerImpl.java | 132 +++
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 55 +-
.../hadoop/ozone/om/TestTrashOzoneFileSystem.java | 133 +++
.../om/request/key/TestOMKeysDeleteRequest.java | 53 +
.../TestOMLifecycleConfigurationRequest.java | 45 +-
.../TestOMLifecycleConfigurationSetRequest.java | 63 ++
.../TestOMLifecycleSaveScanStateRequest.java | 8 +-
...=> TestOMLifecycleSetServiceStatusRequest.java} | 61 +-
.../request/volume/TestOMQuotaRepairRequest.java | 95 ++
.../ozone/om/service/TestKeyDeletingService.java | 245 +++++
.../ozone/om/service/TestKeyLifecycleService.java | 204 ++++
.../ozone/om/snapshot/TestSnapshotDiffManager.java | 160 ++-
.../om/snapshot/TestSnapshotDiffManagerMXBean.java | 4 +-
.../snapshot/defrag/TestSnapshotDefragService.java | 38 +-
.../protocolPB/TestOzoneManagerRequestHandler.java | 65 ++
.../fs/ozone/BasicRootedOzoneFileSystem.java | 29 +-
.../hadoop/fs/ozone/TestBasicOzoneFileSystems.java | 49 +
.../org/apache/hadoop/ozone/recon/ReconServer.java | 10 +
.../org/apache/hadoop/ozone/recon/ReconUtils.java | 55 +-
.../ozone/recon/api/handlers/EntityHandler.java | 122 ++-
.../ozone/recon/chatbot/ChatbotConfigKeys.java | 15 +
.../recon/chatbot/agent/ChatHistoryBuilder.java | 175 ++++
.../ozone/recon/chatbot/agent/ChatbotAgent.java | 90 +-
.../ozone/recon/chatbot/api/ChatbotEndpoint.java | 63 +-
.../ozone/recon/metrics/NSSummaryMetrics.java | 91 ++
.../recon/spi/ReconNamespaceSummaryManager.java | 6 +
.../spi/impl/ReconNamespaceSummaryManagerImpl.java | 21 +-
.../impl/StorageContainerServiceProviderImpl.java | 3 +-
.../recon-tool-selection-prompt-preamble.txt | 7 +
.../ozone-recon-web/src/v2/hooks/useChat.hook.tsx | 28 +-
.../ozone-recon-web/src/v2/types/chatbot.types.ts | 8 +
.../apache/hadoop/ozone/recon/TestReconUtils.java | 40 +
.../api/handlers/TestEntityHandlerCycleGuard.java | 196 ++++
.../chatbot/agent/TestChatHistoryBuilder.java | 289 ++++++
.../agent/TestChatbotAgentExecutionPolicy.java | 62 ++
.../recon/chatbot/api/TestChatbotEndpoint.java | 14 +-
.../ozone/recon/metrics/TestNSSummaryMetrics.java | 60 ++
.../org/apache/hadoop/ozone/audit/S3GAction.java | 4 +-
.../hadoop/ozone/s3/VirtualHostStyleFilter.java | 37 +-
.../ozone/s3/endpoint/BucketLifecycleHandler.java | 19 +-
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 19 +
.../s3/endpoint/GetObjectAttributesResponse.java | 206 ++++
.../ozone/s3/endpoint/ObjectAttributesHandler.java | 173 +++
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 21 +-
.../s3/endpoint/S3LifecycleConfiguration.java | 16 +-
.../hadoop/ozone/s3/metrics/S3GatewayMetrics.java | 22 +
.../org/apache/hadoop/ozone/s3/util/S3Consts.java | 4 +
.../ozone/s3/TestVirtualHostStyleFilter.java | 58 ++
.../ozone/s3/endpoint/EndpointTestUtils.java | 14 +
.../ozone/s3/endpoint/TestObjectAttributesGet.java | 176 ++++
.../endpoint/TestS3LifecycleConfigurationPut.java | 110 +-
161 files changed, 9056 insertions(+), 916 deletions(-)
copy
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/{SkipTracing.java
=> InternalSpanKind.java} (86%)
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/tracing/TestTracingAnnotations.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/common/TestChecksumMultiBuffer.java
create mode 100644
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/common/utils/TestBufferUtils.java
create mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestReconcileChunksPerBlockHoleBcsId.java
create mode 100644
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/replicas/TestReplicasVerify.java
create mode 100644
hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone - Disk
Balancer.json
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/OMMXBeanTests.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestGetDBUpdatesAuthorization.java
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMInstallSnapshotDuringBootstrapping.java
create mode 100644
hadoop-ozone/multitenancy-ranger/src/test/java/org/apache/hadoop/ozone/om/multitenant/TestRangerClientMultiTenantAccessControllerMockClient.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestTrashOzoneFileSystem.java
copy
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/{TestOMLifecycleSaveScanStateRequest.java
=> TestOMLifecycleSetServiceStatusRequest.java} (55%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/volume/TestOMQuotaRepairRequest.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/chatbot/agent/ChatHistoryBuilder.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/metrics/NSSummaryMetrics.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/handlers/TestEntityHandlerCycleGuard.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/chatbot/agent/TestChatHistoryBuilder.java
create mode 100644
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/metrics/TestNSSummaryMetrics.java
create mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/GetObjectAttributesResponse.java
create mode 100644
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectAttributesHandler.java
create mode 100644
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectAttributesGet.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
