This is an automated email from the ASF dual-hosted git repository.
sammichen pushed a change to branch HDDS-10685
in repository https://gitbox.apache.org/repos/asf/ozone.git
discard fdef943fcc HDDS-11622. Support domain socket creation (#7397)
add a737fc3122 HDDS-11619. Remove dependency on hadoop-shaded-guava (#7373)
add c9956a1641 HDDS-11601. Intermittent failure in acceptance balancer
test. (#7343)
add c7a196f313 HDDS-11635. Memory leak when using Ozone FS via Hadoop
FileContext API (#7382)
add 5b065d8ada HDDS-11638. Bump cyclonedx-maven-plugin to 2.9.0 (#7383)
add 58d144369c HDDS-10240. Cleanup zero-copy EC (#7340)
add 6f9db61895 HDDS-11200. Hsync client-side metrics (#7371)
add 3bc3b8ad84 HDDS-11621. Fix missing HADOOP_ variables in MR acceptance
test (#7375)
add 76ec9b9d71 HDDS-11639. Upgrade ozone-runner to Rocky Linux 9.3 (#7391)
add 786bb49f88 HDDS-11642. MutableQuantiles should be stopped (#7392)
add 0fb5e50a5e HDDS-11587. Ozone Manager not processing file put requests
with multi-tenancy enabled (#7316)
add 67e526111e HDDS-11563. Display OM/SCM service ID as Namespace in web
UI (#7321)
add c055036b38 HDDS-11637. Compile failure is ignored in build check
(#7389)
add efe589297c HDDS-11641. Allow testing Hadoop with custom docker images
(#7393)
add 2b1524b596 HDDS-11609. Switch to Recon v2 UI as the default UI (#7358)
add c0458398a8 HDDS-11623. Improve OM Ratis Configuration change log
message (#7388)
add 2547ac0438 HDDS-11652. Fix SCM start command in SCM-HA doc (#7398)
add 0415c0b4f7 HDDS-11649. Recon ListKeys API: Simplify filter predicates
(#7395)
add 18f6e8ab53 HDDS-11311. Added Compatibility test for HSync (#7400)
add 4e603aa93c HDDS-11462. Enhancing DataNode I/O Monitoring Capabilities.
(#7206)
add 5d18b9cdec HDDS-11659. Improve HSync compatibility test (#7404)
add a7e301472f HDDS-11660. Recon List Key API: Reduce object creation and
buffering memory (#7405)
add ef2bf98c37 HDDS-11669. In OmUtils.normalizeKey isDebugEnabled should
be evaluated first (#7411)
add 6dd566f08c HDDS-11583. Use ozone-runner from GitHub in CI (#7409)
add cf7924526f HDDS-11674. Bump junit to 5.11.3 (#7415)
add 358534b83a HDDS-11675. Bump maven-site-plugin to 3.21.0 (#7414)
add 2797c45a6c HDDS-11677. Bump sqlite-jdbc to 3.47.0.0 (#7413)
add 952e0ec385 HDDS-11681. Bump Bouncy Castle to 1.79 (#7387)
add cb81f0cc57 HDDS-11683. Skip shade in most integration checks (#7422)
add 5663971efe HDDS-11665. Minor optimizations on the write path (#7407)
add 27fcd0cb57 HDDS-11685. Use ozone-testkrb5 from GitHub (#7424)
add 47c2409d2f HDDS-64. OzoneClientException should extend IOException.
(#7403)
add 3a18a9d6e5 HDDS-11664. Hadoop download failure not reported as error
(#7421)
add a33d8a3e66 HDDS-10166. Replace GenericTestUtils temporary directories
with `@TempDir` (#7399)
add ee63232860 HDDS-11684. Remove suppression of HiddenField (#7423)
add 8e4a508155 HDDS-11668. Recon List Keys API: Reuse key prefix if
parentID is the same (#7410)
add d09e6d4da3 HDDS-11646. Mark TestXceiverClientMetrics#testMetrics as
flaky
add 8e617dc10c HDDS-11672. Mark
TestSnapshotBackgroundServices#testCompactionLogBackgroundService as flaky
add 9dd6a8301b HDDS-11645. Mark
TestReconScmSnapshot#testExplicitRemovalOfNode as flaky
add aa6da3e1ff HDDS-9781. Limited maxOpenFiles, disabled
enableCompactionDag, and createCheckpointDirs when creating OMMetadataManager
instance for bootstrapping (#7095)
add 10cac80f5c HDDS-11686. Use ozone image from GitHub in CI (#7425)
new 15ee286339 HDDS-11622. Support domain socket creation (#7397)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (fdef943fcc)
\
N -- N -- N refs/heads/HDDS-10685 (15ee286339)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/ci.yml | 5 +
.github/workflows/intermittent-test-check.yml | 2 +-
.../hadoop/hdds/scm/ContainerClientMetrics.java | 53 +++
.../hadoop/hdds/scm/XceiverClientMetrics.java | 2 +
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 2 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 17 +-
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 3 +
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 3 -
.../common/ChunkBufferImplWithByteBuffer.java | 4 +-
.../ozone/container/common/helpers/BlockData.java | 2 +-
.../hadoop/ozone/grpc/metrics/GrpcMetrics.java | 3 +
.../org/apache/hadoop/ozone/util/MetricUtil.java | 16 +
.../hadoop/ozone/util/PerformanceMetrics.java | 15 +-
.../ozone/util/PerformanceMetricsInitializer.java | 9 +-
.../common/src/main/resources/ozone-default.xml | 20 +-
.../java/org/apache/hadoop/ozone/DNMXBean.java | 28 ++
.../java/org/apache/hadoop/ozone/DNMXBeanImpl.java | 49 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 28 +-
.../container/common/helpers/ContainerMetrics.java | 9 +-
.../ozone/container/common/impl/ContainerSet.java | 15 +
.../transport/server/GrpcXceiverService.java | 19 +-
.../common/transport/server/XceiverServerGrpc.java | 9 +-
.../ozone/container/common/volume/HddsVolume.java | 20 +-
.../container/common/volume/VolumeIOStats.java | 57 ++-
.../container/common/volume/VolumeInfoMetrics.java | 8 +
.../ozoneimpl/BackgroundContainerDataScanner.java | 1 +
.../container/ozoneimpl/ContainerController.java | 10 +
.../ozoneimpl/ContainerDataScannerMetrics.java | 11 +
.../ozone/container/ozoneimpl/OzoneContainer.java | 16 +
.../replication/GrpcReplicationService.java | 35 +-
.../container/replication/ReplicationServer.java | 29 +-
.../replication/SendContainerRequestHandler.java | 6 +-
.../webapps/hddsDatanode/dn-overview.html | 28 +-
.../resources/webapps/hddsDatanode/dn-scanner.html | 47 ++
.../src/main/resources/webapps/hddsDatanode/dn.js | 100 ++++-
.../main/resources/webapps/hddsDatanode/index.html | 9 +-
.../resources/webapps/hddsDatanode/iostatus.html | 76 ++++
.../TestVolumeIOStatsWithPrometheusSink.java | 9 +-
.../replication/TestGrpcReplicationService.java | 8 +-
.../TestGrpcReplicationServiceWithZeroCopy.java | 31 --
hadoop-hdds/docs/content/feature/SCM-HA.md | 4 +-
.../hdds/security/x509/certificate/CertInfo.java | 1 -
.../hadoop/hdds/server/ServiceRuntimeInfo.java | 9 +
.../src/main/resources/webapps/static/ozone.css | 21 +-
.../src/main/resources/webapps/static/ozone.js | 14 +-
.../resources/webapps/static/templates/jvm.html | 13 +-
.../resources/webapps/static/templates/menu.html | 2 +
.../webapps/static/templates/overview.html | 6 +-
hadoop-hdds/hadoop-dependency-client/pom.xml | 12 -
hadoop-hdds/hadoop-dependency-server/pom.xml | 12 -
.../hdds/scm/server/StorageContainerManager.java | 4 +
.../server/upgrade/FinalizationManagerImpl.java | 17 +-
.../upgrade/SCMUpgradeFinalizationContext.java | 9 +-
.../org/apache/ozone/test/GenericTestUtils.java | 15 +
.../hadoop/ozone/client/OzoneClientException.java | 8 +-
.../org/apache/hadoop/ozone/client/TenantArgs.java | 9 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 8 +-
.../hadoop/ozone/client/TestOzoneClient.java | 2 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 2 +-
.../hadoop/ozone/om/helpers/DeleteTenantState.java | 1 -
.../hadoop/ozone/om/helpers/OmDBAccessIdInfo.java | 1 -
.../hadoop/ozone/om/helpers/OmDBTenantState.java | 1 -
.../ozone/om/helpers/OmDBUserPrincipalInfo.java | 1 -
.../hadoop/ozone/om/helpers/OmRangerSyncArgs.java | 1 -
.../hadoop/ozone/om/helpers/OmTenantArgs.java | 1 -
.../hadoop/ozone/om/helpers/S3VolumeContext.java | 1 -
hadoop-ozone/dev-support/checks/_lib.sh | 6 +-
hadoop-ozone/dev-support/checks/acceptance.sh | 5 +-
hadoop-ozone/dev-support/checks/build.sh | 2 +
hadoop-ozone/dist/pom.xml | 6 +-
.../dist/src/main/compose/common/hadoop-test.sh | 16 +-
.../dist/src/main/compose/common/hadoop.conf | 3 -
.../src/main/compose/ozone-balancer/docker-config | 1 +
.../src/main/compose/ozone-balancer/test-ec.sh | 4 +-
.../src/main/compose/ozone-balancer/test-ratis.sh | 2 +-
hadoop-ozone/dist/src/main/compose/ozone-ha/.env | 1 +
.../dist/src/main/compose/ozonesecure-mr/.env | 1 +
hadoop-ozone/dist/src/main/compose/testlib.sh | 8 +-
.../dist/src/main/compose/upgrade/compose/ha/.env | 2 +-
.../compose/upgrade/compose/ha/docker-compose.yaml | 2 +-
.../src/main/compose/upgrade/compose/non-ha/.env | 2 +-
.../upgrade/compose/non-ha/docker-compose.yaml | 2 +-
.../src/main/compose/upgrade/compose/om-ha/.env | 2 +-
.../upgrade/compose/om-ha/docker-compose.yaml | 2 +-
.../dist/src/main/compose/upgrade/testlib.sh | 2 +-
hadoop-ozone/dist/src/main/compose/xcompat/.env | 2 +
.../dist/src/main/compose/xcompat/clients.yaml | 10 +-
.../dist/src/main/compose/xcompat/old-cluster.yaml | 2 +-
hadoop-ozone/dist/src/main/keytabs/HTTP.keytab | Bin 580 -> 580 bytes
hadoop-ozone/dist/src/main/keytabs/dn.keytab | Bin 576 -> 576 bytes
hadoop-ozone/dist/src/main/keytabs/hadoop.keytab | Bin 146 -> 146 bytes
hadoop-ozone/dist/src/main/keytabs/httpfs.keytab | Bin 616 -> 616 bytes
hadoop-ozone/dist/src/main/keytabs/jhs.keytab | Bin 142 -> 142 bytes
hadoop-ozone/dist/src/main/keytabs/nm.keytab | Bin 138 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/om.keytab | Bin 576 -> 576 bytes
hadoop-ozone/dist/src/main/keytabs/recon.keytab | Bin 606 -> 606 bytes
hadoop-ozone/dist/src/main/keytabs/rm.keytab | Bin 138 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/s3g.keytab | Bin 586 -> 586 bytes
hadoop-ozone/dist/src/main/keytabs/scm.keytab | Bin 586 -> 586 bytes
hadoop-ozone/dist/src/main/keytabs/testuser.keytab | Bin 908 -> 908 bytes
.../dist/src/main/keytabs/testuser2.keytab | Bin 920 -> 920 bytes
.../src/main/smoketest/balancer/testBalancer.robot | 9 +-
.../src/main/smoketest/compatibility/read.robot | 6 +
.../src/main/smoketest/compatibility/setup.robot | 1 +
.../src/main/smoketest/compatibility/write.robot | 10 +
.../main/smoketest/om-ratis/testOMAdminCmd.robot | 2 +-
.../dist/src/main/smoketest/s3/objectcopys3a.robot | 8 +-
.../hadoop/hdds/scm/TestXceiverClientManager.java | 22 +-
.../hadoop/hdds/scm/TestXceiverClientMetrics.java | 11 +-
.../org/apache/hadoop/ozone/TestBlockTokens.java | 8 +-
.../apache/hadoop/ozone/TestBlockTokensCLI.java | 7 +-
.../apache/hadoop/ozone/TestDelegationToken.java | 5 +-
.../apache/hadoop/ozone/TestOMSortDatanodes.java | 8 +-
.../client/rpc/AbstractTestECKeyOutputStream.java | 492 ---------------------
.../ozone/client/rpc/OzoneRpcClientTests.java | 15 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 6 -
.../rpc/TestContainerReplicationEndToEnd.java | 7 +-
.../client/rpc/TestContainerStateMachine.java | 6 -
.../TestContainerStateMachineFailureOnRead.java | 5 -
.../rpc/TestContainerStateMachineFlushDelay.java | 6 -
.../client/rpc/TestDeleteWithInAdequateDN.java | 7 -
.../ozone/client/rpc/TestECKeyOutputStream.java | 466 ++++++++++++++++++-
.../rpc/TestECKeyOutputStreamWithZeroCopy.java | 31 --
.../client/rpc/TestOzoneAtRestEncryption.java | 6 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 7 +-
.../transport/server/ratis/TestCSMMetrics.java | 11 +-
.../container/metrics/TestContainerMetrics.java | 21 +-
.../ozoneimpl/TestSecureOzoneContainer.java | 7 +-
.../container/server/TestContainerServer.java | 14 +-
.../server/TestSecureContainerServer.java | 19 +-
.../ozone/freon/TestHadoopDirTreeGenerator.java | 16 +-
.../ozone/freon/TestHadoopNestedDirGenerator.java | 15 +-
.../ozone/freon/TestOmBucketReadWriteFileOps.java | 15 +-
.../ozone/freon/TestOmBucketReadWriteKeyOps.java | 14 +-
.../hadoop/ozone/fsck/TestContainerMapper.java | 12 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 9 +-
.../ozone/om/TestOmContainerLocationCache.java | 9 +-
.../ozone/om/service/TestRangerBGSyncService.java | 5 +-
.../snapshot/TestSnapshotBackgroundServices.java | 2 +
.../hadoop/ozone/recon/TestReconScmSnapshot.java | 2 +
.../hadoop/ozone/shell/TestOzoneShellHA.java | 25 +-
.../hadoop/ozone/shell/TestOzoneTenantShell.java | 15 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 42 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 5 +
.../RangerClientMultiTenantAccessController.java | 12 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 11 +-
.../apache/hadoop/ozone/om/TestOMDBDefinition.java | 44 +-
.../java/org/apache/hadoop/fs/ozone/BasicOzFs.java | 9 +
.../main/java/org/apache/hadoop/fs/ozone/OzFs.java | 9 +
.../org/apache/hadoop/fs/ozone/RootedOzFs.java | 9 +
.../main/java/org/apache/hadoop/fs/ozone/OzFs.java | 9 +
.../org/apache/hadoop/fs/ozone/RootedOzFs.java | 9 +
.../main/java/org/apache/hadoop/fs/ozone/OzFs.java | 9 +
.../org/apache/hadoop/fs/ozone/RootedOzFs.java | 9 +
.../org/apache/hadoop/ozone/recon/ReconUtils.java | 86 +++-
.../hadoop/ozone/recon/api/NodeEndpoint.java | 50 +--
.../ozone/recon/api/OMDBInsightEndpoint.java | 171 ++++---
.../hadoop/ozone/recon/api/types/AclMetadata.java | 17 +-
.../recon/api/types/ClusterStateResponse.java | 1 -
.../ozone/recon/api/types/DatanodeMetadata.java | 33 +-
.../recon/api/types/KeyEntityInfoProtoWrapper.java | 145 ++++++
.../ozone/recon/api/types/ListKeysResponse.java | 6 +-
.../recon/api/types/NamespaceSummaryResponse.java | 15 +-
.../hadoop/ozone/recon/api/types/ParamInfo.java | 14 +
.../ozone/recon/api/types/PipelineMetadata.java | 1 -
.../recon/recovery/ReconOMMetadataManager.java | 12 +
.../recon/recovery/ReconOmMetadataManagerImpl.java | 9 +
.../ozone/recon/upgrade/UpgradeActionRecon.java | 15 +-
.../webapps/recon/ozone-recon-web/src/app.tsx | 69 +--
.../src/v2/pages/notFound/notFound.tsx | 103 +++++
.../recon/ozone-recon-web/src/v2/routes-v2.tsx | 9 +
.../ozone/recon/api/TestOmDBInsightEndPoint.java | 18 +-
.../hadoop/ozone/s3/metrics/S3GatewayMetrics.java | 16 +-
.../ozone/admin/om/GetServiceRolesSubcommand.java | 3 -
.../ozone/admin/om/UpdateRangerSubcommand.java | 4 -
.../apache/hadoop/ozone/debug/ChunkKeyHandler.java | 3 +-
.../ozone/debug/CompactionLogDagPrinter.java | 3 +-
.../apache/hadoop/ozone/debug/ReadReplicas.java | 3 +-
.../java/org/apache/hadoop/ozone/freon/Freon.java | 3 +-
...{SameKeyReader.java => OzoneClientCreator.java} | 54 ++-
.../org/apache/hadoop/ozone/shell/Handler.java | 4 +-
.../apache/hadoop/ozone/shell/OzoneAddress.java | 10 +-
.../ozone/shell/bucket/SetEncryptionKey.java | 3 +-
.../shell/bucket/SetReplicationConfigHandler.java | 3 +-
.../ozone/shell/bucket/UpdateBucketHandler.java | 3 +-
.../hadoop/ozone/shell/keys/CatKeyHandler.java | 3 +-
.../ozone/shell/keys/ChecksumKeyHandler.java | 3 +-
.../hadoop/ozone/shell/keys/CopyKeyHandler.java | 3 +-
.../hadoop/ozone/shell/keys/DeleteKeyHandler.java | 3 +-
.../hadoop/ozone/shell/keys/GetKeyHandler.java | 2 +-
.../hadoop/ozone/shell/keys/ListKeyHandler.java | 3 +-
.../hadoop/ozone/shell/keys/PutKeyHandler.java | 3 +-
.../hadoop/ozone/shell/keys/RenameKeyHandler.java | 3 +-
.../hadoop/ozone/shell/keys/RewriteKeyHandler.java | 3 +-
.../apache/hadoop/ozone/shell/s3/S3Handler.java | 2 +-
.../shell/snapshot/RenameSnapshotHandler.java | 3 +-
.../hadoop/ozone/shell/tenant/TenantHandler.java | 2 +-
.../ozone/shell/token/CancelTokenHandler.java | 3 +-
.../hadoop/ozone/shell/token/GetTokenHandler.java | 2 +-
.../ozone/shell/token/RenewTokenHandler.java | 3 +-
.../TestGenerateOzoneRequiredConfigurations.java | 16 +-
.../shell/TestOzoneAddressClientCreation.java | 24 +-
.../ozone/shell/keys/TestChecksumKeyHandler.java | 3 +-
pom.xml | 16 +-
204 files changed, 2160 insertions(+), 1437 deletions(-)
create mode 100644
hadoop-hdds/container-service/src/main/resources/webapps/hddsDatanode/dn-scanner.html
create mode 100644
hadoop-hdds/container-service/src/main/resources/webapps/hddsDatanode/iostatus.html
delete mode 100644
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestGrpcReplicationServiceWithZeroCopy.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/AbstractTestECKeyOutputStream.java
delete mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestECKeyOutputStreamWithZeroCopy.java
create mode 100644
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/KeyEntityInfoProtoWrapper.java
create mode 100644
hadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/v2/pages/notFound/notFound.tsx
copy
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/{SameKeyReader.java
=> OzoneClientCreator.java} (51%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
