This is an automated email from the ASF dual-hosted git repository.
aengineer pushed a change to branch ozone-0.4.1
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from 0b92dec HDDS-1858. mTLS support for Ozone is not correct. Contributed
by Siddharth Wagle.
add d545f9c HADOOP-16437 documentation typo fix:
fs.s3a.experimental.input.fadvise
add 4e66cb9 HDDS-1653. Add option to "ozone scmcli printTopology" to
order the output acccording to topology layer. Contributed by Xiaoyu Yao.
(#1067)
add 6282c02 HDFS-14577. RBF: FederationUtil#newInstance should allow
constructor without context. Contributed by CR Hota.
add cd967c7 HADOOP-15847. S3Guard testConcurrentTableCreations to set R/W
capacity == 0
add 7f1b76c HADOOP-13868. [s3a] New default for S3A multi-part
configuration (#1125)
add 69a46a9 HDDS-1713. ReplicationManager fail to find proper node
topology based… (#1112)
add f5b2f75 HDDS-1795. Implement S3 Delete Bucket request to use Cache
and DoubleBuffer. (#1097)
add 0b45293 HADOOP-16404. ABFS default blocksize change(256MB from 512MB)
add e60f5e2 HADOOP-16440. Distcp can not preserve timestamp with -delete
option. Contributed by ludun.
add 4aa76e3 HDDS-1805. Implement S3 Initiate MPU request to use Cache and
DoubleBuffer. (#1108)
add acdb0a1 HDDS-1841. Fix TestSecureContainerServer. (#1136)
add 340bbaf HDDS-1840. Fix TestSecureOzoneContainer. (#1135)
add cdc36fe HDDS-1649. On installSnapshot notification from OM leader,
download checkpoint and reload OM state (#948)
add c958edd HDDS-1811. Prometheus metrics are broken.
add 2ea71d9 HDDS-1686. Remove check to get from openKeyTable in acl
implementatio… (#966)
add f3f2f34 HDDS-1585. Add LICENSE.txt and NOTICE.txt to Ozone Recon Web
add d70ec4b HDDS-1799. Add goofyfs to the ozone-runner docker image
add d59f271 HDDS-1803. shellcheck.sh does not work on Mac
add c533b79 HDDS-1710. Publish JVM metrics via Hadoop metrics
Signed-off-by: Anu Engineer <[email protected]>
add ee87e9a MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete
failing in our internal build
add 377f95b HDFS-13693. Remove unnecessary search in
INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
add eb36b09 HADOOP-16443. Improve help text for setfacl --set option.
add 4317d33 HADOOP-16380. S3Guard to determine empty directory status for
all non-root directories.
add 278390f HDDS-1710. Fix checkstyle errors
add fbe2fc6 HDDS-1846. Default value for checksum bytes is different in
ozone-site.xml and code.
add ecb9f81 HDDS-1845. Fix
OMVolumeSetQuota|OwnerRequest#validateAndUpdateCache return response. (#1141)
add a1251ad HADOOP-16431. Remove useless log in IOUtils.java and
ExceptionDiags.java.
add cf9ff08 HDDS-1848. Fix TestOzoneManagerHA and
TestOzoneManagerSnapShotProvider. (#1148)
add 7b21977 HADOOP-16433. S3Guard: Filter expired entries and tombstones
when listing with MetadataStore.listChildren().
add cb69700 HDDS-1853. Fix failing blockade test-cases. (#1151)
add 1d98a21 HDDS-1819. Implement S3 Commit MPU request to use Cache and
DoubleBuffer. (#1140)
add 2546e6e HDDS-1817. GetKey fails with IllegalArgumentException.
add 9b8b3ac HADOOP-16451. Update jackson-databind to 2.9.9.1. Contributed
by Siyao Meng.
add 62deab1 HDFS-14647. NPE during secure namenode startup. Contributed
by Fengnan Li.
add ba43233 HDDS-1749 : Ozone Client should randomize the list of nodes
in pipeline for reads. (#1124)
add 3c4159f HDDS-1842. Implement S3 Abort MPU request to use Cache and
DoubleBuffer. (#1155)
add b41ef61 HDDS-1858. mTLS support for Ozone is not correct. Contributed
by Siddharth Wagle.
new 20b2cd6 Merge branch 'trunk' into ozone-0.4.1
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/hadoop/fs/shell/AclCommands.java | 6 +-
.../main/java/org/apache/hadoop/io/IOUtils.java | 2 -
.../src/main/resources/core-default.xml | 4 +-
.../src/site/markdown/FileSystemShell.md | 2 +-
.../src/site/markdown/filesystem/filesystem.md | 44 ++-
.../filesystem/fsdatainputstreambuilder.md | 2 +-
.../AbstractContractRootDirectoryTest.java | 2 +-
.../hadoop/fs/contract/ContractTestUtils.java | 30 +-
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 4 +
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 1 +
.../common/src/main/resources/ozone-default.xml | 8 +
.../hadoop/hdds/scm/cli/TopologySubcommand.java | 60 +++-
.../server/federation/router/FederationUtil.java | 13 +-
.../federation/router/TestFederationUtil.java | 69 ++++
.../hdfs/server/namenode/FSImageFormatPBINode.java | 4 +-
.../hdfs/server/namenode/INodeDirectory.java | 16 +
.../hadoop/hdfs/server/namenode/NameNode.java | 4 +
.../hadoop/hdfs/server/common/TestJspHelper.java | 34 ++
.../test/java/org/apache/hadoop/hdfs/NNBench.java | 2 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 1 -
.../org/apache/hadoop/ozone/audit/OMAction.java | 2 +
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 3 +
.../hadoop/ozone/om/exceptions/OMException.java | 3 +-
.../ozone/om/protocol/OzoneManagerHAProtocol.java | 18 +-
.../src/main/proto/OzoneManagerProtocol.proto | 9 +-
.../dist/src/main/compose/ozone-topology/test.sh | 2 +
.../dist/src/main/smoketest/topology/scmcli.robot | 32 ++
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 7 +
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 54 ++-
.../ozoneimpl/TestSecureOzoneContainer.java | 2 +
.../server/TestSecureContainerServer.java | 2 +
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 189 ++++++++++
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 7 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 11 -
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 10 +-
.../hadoop/ozone/om/OMDBCheckpointServlet.java | 2 +-
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 22 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 381 +++++++++++++++------
.../ozone/om/ratis/OzoneManagerDoubleBuffer.java | 3 +
.../ozone/om/ratis/OzoneManagerRatisServer.java | 15 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 130 +++----
.../om/ratis/utils/OzoneManagerRatisUtils.java | 12 +
.../request/s3/bucket/S3BucketDeleteRequest.java | 193 +++++++++++
.../S3InitiateMultipartUploadRequest.java | 213 ++++++++++++
.../multipart/S3MultipartUploadAbortRequest.java | 173 ++++++++++
.../S3MultipartUploadCommitPartRequest.java | 217 ++++++++++++
.../om/request/s3/multipart/package-info.java} | 21 +-
.../om/request/volume/OMVolumeSetOwnerRequest.java | 4 +-
.../om/request/volume/OMVolumeSetQuotaRequest.java | 3 +-
.../response/s3/bucket/S3BucketCreateResponse.java | 8 +-
...teResponse.java => S3BucketDeleteResponse.java} | 54 +--
.../S3InitiateMultipartUploadResponse.java | 80 +++++
.../multipart/S3MultipartUploadAbortResponse.java | 83 +++++
.../S3MultipartUploadCommitPartResponse.java | 109 ++++++
.../om/response/s3/multipart/package-info.java} | 22 +-
.../om/snapshot/OzoneManagerSnapshotProvider.java | 2 +-
.../OzoneManagerHARequestHandlerImpl.java | 18 +-
.../protocolPB/OzoneManagerRequestHandler.java | 35 --
.../ozone/om/request/TestOMRequestUtils.java | 85 +++++
.../s3/bucket/TestS3BucketDeleteRequest.java | 167 +++++++++
.../TestS3InitiateMultipartUploadRequest.java | 153 +++++++++
.../s3/multipart/TestS3MultipartRequest.java | 178 ++++++++++
.../TestS3MultipartUploadAbortRequest.java | 158 +++++++++
.../TestS3MultipartUploadCommitPartRequest.java | 209 +++++++++++
.../s3/multipart/package-info.java} | 20 +-
.../ozone/om/response/TestOMResponseUtils.java | 42 +++
.../s3/bucket/TestS3BucketCreateResponse.java | 41 +--
.../s3/bucket/TestS3BucketDeleteResponse.java | 73 ++++
.../TestS3InitiateMultipartUploadResponse.java | 63 ++++
.../s3/multipart/TestS3MultipartResponse.java | 143 ++++++++
.../TestS3MultipartUploadAbortResponse.java | 129 +++++++
.../multipart/package-info.java} | 20 +-
hadoop-project/pom.xml | 3 +-
.../java/org/apache/hadoop/fs/s3a/Constants.java | 4 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 27 +-
.../hadoop/fs/s3a/s3guard/DDBPathMetadata.java | 11 +-
.../hadoop/fs/s3a/s3guard/DirListingMetadata.java | 41 ++-
.../fs/s3a/s3guard/DynamoDBMetadataStore.java | 23 +-
.../hadoop/fs/s3a/s3guard/LocalMetadataStore.java | 50 +--
.../apache/hadoop/fs/s3a/s3guard/PathMetadata.java | 62 +++-
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 13 +-
.../src/site/markdown/tools/hadoop-aws/index.md | 4 +-
.../site/markdown/tools/hadoop-aws/performance.md | 4 +-
.../src/site/markdown/tools/hadoop-aws/testing.md | 6 +-
.../tools/hadoop-aws/troubleshooting_s3a.md | 2 +-
.../fs/contract/s3a/ITestS3AContractRootDir.java | 9 +-
.../hadoop/fs/s3a/ITestS3GuardEmptyDirs.java | 155 ++++++++-
.../fs/s3a/ITestS3GuardOutOfBandOperations.java | 46 ++-
.../org/apache/hadoop/fs/s3a/ITestS3GuardTtl.java | 70 ++++
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 26 ++
.../fs/s3a/s3guard/ITestDynamoDBMetadataStore.java | 2 +-
.../fs/s3a/s3guard/ITestS3GuardConcurrentOps.java | 10 +
.../s3a/s3guard/ITestS3GuardDDBRootOperations.java | 2 +-
.../fs/s3a/s3guard/MetadataStoreTestBase.java | 5 +-
.../fs/s3a/s3guard/TestDirListingMetadata.java | 44 ++-
.../fs/s3a/s3guard/TestLocalMetadataStore.java | 40 +++
.../apache/hadoop/fs/s3a/s3guard/TestS3Guard.java | 14 +-
.../hadoop-aws/src/test/resources/core-site.xml | 4 +-
.../constants/FileSystemConfigurations.java | 2 +-
.../TestAbfsConfigurationFieldsValidation.java | 6 +
.../apache/hadoop/tools/mapred/CopyCommitter.java | 14 +-
.../hadoop/tools/mapred/TestCopyCommitter.java | 80 +++++
.../hadoop/fs/swift/http/ExceptionDiags.java | 2 -
.../pom.xml | 1 -
104 files changed, 4153 insertions(+), 569 deletions(-)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestFederationUtil.java
create mode 100644 hadoop-ozone/dist/src/main/smoketest/topology/scmcli.robot
create mode 100644
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMRatisSnapshots.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/bucket/S3BucketDeleteRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3InitiateMultipartUploadRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadAbortRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCommitPartRequest.java
copy
hadoop-ozone/ozone-manager/src/{test/java/org/apache/hadoop/ozone/om/response/TestOMResponseUtils.java
=>
main/java/org/apache/hadoop/ozone/om/request/s3/multipart/package-info.java}
(56%)
copy
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/bucket/{S3BucketCreateResponse.java
=> S3BucketDeleteResponse.java} (53%)
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3InitiateMultipartUploadResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadAbortResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCommitPartResponse.java
copy
hadoop-ozone/ozone-manager/src/{test/java/org/apache/hadoop/ozone/om/response/TestOMResponseUtils.java
=>
main/java/org/apache/hadoop/ozone/om/response/s3/multipart/package-info.java}
(56%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/bucket/TestS3BucketDeleteRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3InitiateMultipartUploadRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadAbortRequest.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCommitPartRequest.java
copy
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/{response/TestOMResponseUtils.java
=> request/s3/multipart/package-info.java} (56%)
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/bucket/TestS3BucketDeleteResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3InitiateMultipartUploadResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartResponse.java
create mode 100644
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadAbortResponse.java
copy
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/{TestOMResponseUtils.java
=> s3/multipart/package-info.java} (56%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
