This is an automated email from the ASF dual-hosted git repository.
hexiaoqiao pushed a change to branch HDFS-17531
in repository https://gitbox.apache.org/repos/asf/hadoop.git
omit 4e0b405adec HDFS-17544. [ARR] The router client rpc protocol PB
supports asynchrony. (#6870). Contributed by Jian Zhang.
omit 6de630b013b HADOOP-19235. IPC client uses CompletableFuture to support
asynchronous operations. (#6888)
omit 78204d9ef03 HDFS-17543. [ARR] AsyncUtil makes asynchronous code more
concise and easier. (#6868). Contributed by Jian Zhang.
add bbb17e76a7a HADOOP-19178: [WASB Deprecation] Updating Documentation on
Upcoming Plans for Hadoop-Azure (#6862)
add 01d257d5aa9 HADOOP-19189. ITestS3ACommitterFactory failing (#6857)
add 10df59e4210 Revert "HADOOP-19071. Update maven-surefire-plugin from
3.0.0 to 3.2.5. (#6664)" (#6875)
add 2e1deee87a5 HADOOP-19137. [ABFS] Prevent ABFS initialization for
non-hierarchal-namespace account if Customer-provided-key configs given. (#6752)
add 776c0a3ab95 HDFS-17539. Make TestFileChecksum fields static (#6853)
add bb30545583c HADOOP-19163. Use hadoop-shaded-protobuf_3_25 (#6858)
add 005030f7a0d HADOOP-18610: [ABFS] OAuth2 Token Provider support for
Azure Workload Identity (#6787)
add 06dd3bfee80 HADOOP-19196. Allow base path to be deleted as well using
Bulk Delete. (#6872)
add 4b1b16a846b HDFS-17551. Fix unit test failure caused by HDFS-17464.
(#6883). Contributed by farmmamba.
add a1f5dc58651 Minor, fix cpu arch compare to use correct Dockerfile
(#6852)
add 2bde5ccb813 HADOOP-19192. Log level is WARN when fail to load native
hadoop libs (#6863)
add 240fddcf17f HADOOP-18931. FileSystem.getFileSystemClass() to log the
jar the .class came from (#6197)
add 2d5fa9e016d HADOOP-18508. S3A: Support parallel integration test runs
on same bucket (#5081)
add 2fbbfe3cc93 HDFS-17546. Implementing HostsFileReader timeout (#6873)
add 90024d8cb13 HDFS-17439. Support -nonSuperUser for
NNThroughputBenchmark: useful for testing auth frameworks such as Ranger (#6677)
add 9710a8d52f9 YARN-11701. [Federation] Enhance Federation Cache Clean
Conditions. (#6889) Contributed by Shilun Fan.
add 1e6411c9ec6 HDFS-17528. FsImageValidation: set txid when saving a new
image (#6828)
add 56c8aa5f1c4 HADOOP-19204. VectorIO regression: empty ranges are now
rejected (#6887)
add 6545b7eeef1 HDFS-17098. DatanodeManager does not handle null storage
type properly. (#6840). Contributed by ConfX.
add 8ac9c1839ac HADOOP-19203. WrappedIO BulkDelete API to raise IOEs as
UncheckedIOExceptions (#6885)
add d3b98cb1b23 HADOOP-19194:Add test to find unshaded dependencies in the
aws sdk (#6865)
add b4ddb2d3bba HDFS-13603: do not propagate ExecutionException and add
maxRetries limit to NameNode edek cache warmup (#6774)
add 134dcf166f5 YARN-11703. Validate accessibility of Node Manager working
directories (#6903)
add c33d8686060 HADOOP-19210. S3A: Speed up some slow unit tests (#6907)
add a57105462be HADOOP-19215. Fix unit tests testSlowConnection and
testBadSetup failed in TestRPC. (#6912). Contributed by farmmamba.
add ae76e9475cd HDFS-17564. EC: Fix the issue of inaccurate metrics when
decommission mark busy DN. (#6911). Contributed by Haiyang Hu.
add 4c55adbb6bc HADOOP-19205. S3A: initialization/close slower than with
v1 SDK (#6892)
add 25e28b41cc3 HADOOP-19216. Upgrade Guice from 4.0 to 5.1.0 to support
Java 17 (#6913). Contributed by Cheng Pan.
add 5a8f70a72eb HDFS-17559. Fix the uuid as null in NameNodeMXBean
(#6906). Contributed by Haiyang Hu.
add 8ca4627a0da HDFS-17557. Fix bug for
TestRedundancyMonitor#testChooseTargetWhenAllDataNodesStop (#6897). Contributed
by Haiyang Hu.
add b1d96f6101c HADOOP-19195. S3A: Upgrade aws sdk v2 to 2.25.53 (#6900)
add 783a8520292 HDFS-17555. Fix NumberFormatException of
NNThroughputBenchmark when configured dfs.blocksize. (#6894). Contributed by
wangzhongwei
add f211af30be9 HDFS-17534. RBF: Support leader follower mode for multiple
subclusters (#6861). Contributed by Yuanbo Liu.
add 8913d379fd2 HDFS-17566. Got wrong sorted block order when StorageType
is considered. (#6934). Contributed by Chenyu Zheng.
add 4f0ee9d67d9 HADOOP-19222. Switch yum repo baseurl due to CentOS 7
sunset (#6932) Contributed by Cheng Pan.
add 51cb858cc8c HADOOP-19208: [ABFS] Fixing logic to determine HNS nature
of account to avoid extra getAcl() calls (#6893)
add 57306566601 HDFS-17574. Make NNThroughputBenchmark support
human-friendly units about blocksize. (#6931). Contributed by wangzhongwei.
add 1360c7574ad HADOOP-19218 Avoid DNS lookup while creating IPC
Connection object (#6916). Contributed by Viraj Jasani.
add ebbe9628d34 HDFS-17558. RBF: Make maxSizeOfFederatedStateToPropagate
work on setResponseHeaderState. (#6902)
add 9a9ad6cc984 YARN-11705. Turn off Node Manager working directories
validation by default (#6948)
add 7638b4727e7 HDFS-17575. SaslDataTransferClient should use
SaslParticipant to create messages. (#6933)
add 9dad697dbca HADOOP-19227. ipc.Server accelerate token negotiation only
for the default mechanism. (#6949)
add a5eb5e96117 HDFS-17576. Support user defined auth Callback. (#6945)
add 1577f57d4c8 HADOOP-19228. ShellCommandFencer#setConfAsEnvVars should
also replace '-' with '_'. (#6936). Contributed by fuchaohong.
add e48cd0e987d Revert "HDFS-17575. SaslDataTransferClient should use
SaslParticipant to create messages. (#6933)"
add b60497ff41e HADOOP-19120. ApacheHttpClient adaptation in ABFS. (#6633)
add e000cbf2771 HADOOP-19218. Addendum. Update TestFSNamesystemLockReport
to exclude hostname resolution from regex. (#6951). Contributed by Viraj Jasani.
add e2a0dca43b5 HDFS-16690. Automatically format unformatted JNs with
JournalNodeSyncer (#6925). Contributed by Aswin M Prabhu.
add 4525c7e35ea HADOOP-19197. S3A: Support AWS KMS Encryption Context
(#6874)
add a5806a9e7bc HADOOP-19161. S3A: option "fs.s3a.performance.flags" to
take list of performance flags (#6789)
add 038636a1b52 HADOOP-19238. Fix create-release script for arm64 based
MacOS (#6962)
add 059e996c02d HDFS-17591. RBF: Router should follow X-FRAME-OPTIONS
protection setting (#6963)
add c593c17255c HADOOP-19237. Upgrade to dnsjava 3.6.1 due to CVEs (#6961)
add 2cf4d638af3 HADOOP-19245. S3ABlockOutputStream no longer sends
progress events in close() (#6974)
add b08d492abd9 HADOOP-19246. Update the yasm rpm download address (#6973)
add 59d5e0bb2e7 HADOOP-19244. Pullout arch-agnostic maven javadoc plugin
configurations in hadoop-common (#6970) Contributed by Cheng Pan.
add b189ef81978 HDFS-17575. SaslDataTransferClient should use
SaslParticipant to create messages. (#6954)
add 2a509117344 HADOOP-17609. Make SM4 support optional for OpenSSL native
code. (#3019)
add 321a6cc55ed HADOOP-19072. S3A: expand optimisations on stores with
"fs.s3a.performance.flags" for mkdir (#6543)
add 74ff00705cf HADOOP-19072. S3A: Override fs.s3a.performance.flags for
tests (ADDENDUM) (#6985)
add fa83c9a8050 HADOOP-19072 S3A: Override fs.s3a.performance.flags for
tests (ADDENDUM 2) (#6993)
add 55a576906dd HADOOP-19131. Assist reflection IO with WrappedOperations
class (#6686)
add fa9bb0d1ac4 HADOOP-19231. Add JacksonUtil to manage Jackson classes
(#6953)
add 5f93edfd707 HADOOP-19153. hadoop-common exports logback as a
transitive dependency (#6999)
add f00094203bf YARN-11709. NodeManager should be shut down or blacklisted
when it cacannot run program /var/lib/yarn-ce/bin/container-executor (#6960)
add bf804cb64be HADOOP-19250. Fix test
TestServiceInterruptHandling.testRegisterAndRaise (#6987)
add b5f88990b72 HADOOP-19136. Upgrade commons-io to 2.16.1. (#6704)
add 59dba6e1bd3 HADOOP-19134. Use StringBuilder instead of StringBuffer.
(#6692). Contributed by PJ Fanning
add e5b76dc99fd HADOOP-19180. EC: Fix calculation errors caused by special
index order (#6813). Contributed by zhengchenyu.
add df08e0de41a HDFS-17605. Reduce memory overhead of TestBPOfferService
(#6996)
add 2fd7cf53fac HADOOP-19253. Google GCS compilation fails due to VectorIO
changes (#7002)
add 33c9ecb6521 HADOOP-19249. KMSClientProvider raises NPE with unauthed
user (#6984)
add b15ed27cfbf HADOOP-19187: [ABFS][FNSOverBlob] AbfsClient Refactoring
to Support Multiple Implementation of Clients. (#6879)
add 012ae9d1aa0 HDFS-17606. Do not require implementing
CustomizedCallbackHandler. (#7005)
add 68fcd7234ca HADOOP-18542. Keep MSI tenant ID and client ID optional
(#4262)
add f6c45e0bcf4 HDFS-17546. Follow-up backport from branch3.3 (#6908)
add 6be04633b55 YARN-11711. Clean Up ServiceScheduler Code. (#6977)
Contributed by Shilun Fan.
add 5745a7dd754 HDFS-16084. Fix getJNIEnv crash due to incorrect state set
to tls var (#6969). Contributed by Kevin Cai.
add 89e38f08ae0 HDFS-17573. Allow turn on both FSImage parallelization and
compression (#6929). Contributed by Sung Dong Kim.
add e4ee3d560bd YARN-10345 HsWebServices containerlogs does not honor ACLs
for completed jobs (#7013)
add 0aab1a29764 HADOOP-19248. Protobuf code generate and replace should
happen together (#6975)
add 0837c84a9f2 Revert "HADOOP-19231. Add JacksonUtil to manage Jackson
classes (#6953)"
add a962aa37e00 HDFS-17599. EC: Fix the mismatch between locations and
indices for mover (#6980)
add b404c8c8f80 HADOOP-19252. Upgrade hadoop-thirdparty to 1.3.0 (#7007)
add 1655acc5e2d HADOOP-19250. [Addendum] Fix test
TestServiceInterruptHandling.testRegisterAndRaise. (#7008)
add 3bbfb2be089 HADOOP-19257. S3A:
ITestAssumeRole.testAssumeRoleBadInnerAuth failure (#7021)
add 94868446104 HADOOP-16928. Make javadoc work on Java 17 (#6976)
add 6c01490f14b YARN-11664. Remove HDFS Binaries/Jars Dependency From Yarn
(#6631)
add 1f302e83fd9 HADOOP-18938. S3A: Fix endpoint region parsing for vpc
endpoints. (#6466)
add 57e62ae07f1 Revert "YARN-11664. Remove HDFS Binaries/Jars Dependency
From Yarn (#6631)"
add a00b1c06f32 HADOOP-19269. Upgrade maven-shade-plugin 3.6.0 (#7029)
add 8c41fbcaf54 Revert "YARN-11709. NodeManager should be shut down or
blacklisted when it ca…" (#7028)
add 6881d12da4b HADOOP-19262: Upgrade wildfly-openssl:1.1.3.Final to
2.1.4.Final to support Java17+ (#7026)
add c9e9bce3610 YARN-11729. Broken 'AM Node Web UI' link on App details
page (#7030)
add c835adb3a8d HADOOP-19201 S3A. Support external-id in assume role
(#6876)
new 4bb587ea2ba HDFS-17543. [ARR] AsyncUtil makes asynchronous code more
concise and easier. (#6868). Contributed by Jian Zhang.
new 1f91aefc0f0 HADOOP-19235. IPC client uses CompletableFuture to support
asynchronous operations. (#6888)
new a5919024552 HDFS-17544. [ARR] The router client rpc protocol PB
supports asynchrony. (#6870). Contributed by Jian Zhang.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (4e0b405adec)
\
N -- N -- N refs/heads/HDFS-17531 (a5919024552)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
LICENSE-binary | 25 +-
dev-support/bin/create-release | 4 +-
dev-support/docker/Dockerfile_centos_7 | 16 +-
dev-support/docker/pkg-resolver/install-yasm.sh | 2 +-
.../pkg-resolver/set-vault-as-baseurl-centos.sh | 2 +-
.../resources/ensure-jars-have-correct-contents.sh | 2 +
.../hadoop-client-runtime/pom.xml | 3 +
.../dev-support/findbugsExcludeFile.xml | 6 +
hadoop-common-project/hadoop-common/pom.xml | 42 +-
.../java/org/apache/hadoop/conf/Configuration.java | 68 +-
.../org/apache/hadoop/crypto/OpensslCipher.java | 16 +
.../hadoop/crypto/OpensslSm4CtrCryptoCodec.java | 4 +
.../org/apache/hadoop/crypto/key/KeyShell.java | 2 +-
.../hadoop/crypto/key/kms/KMSClientProvider.java | 12 +-
.../org/apache/hadoop/fs/AbstractFileSystem.java | 2 +-
.../java/org/apache/hadoop/fs/BulkDeleteUtils.java | 8 +-
.../org/apache/hadoop/fs/ChecksumFileSystem.java | 2 +-
.../main/java/org/apache/hadoop/fs/ChecksumFs.java | 2 +-
.../org/apache/hadoop/fs/ClosedIOException.java | 22 +-
.../hadoop/fs/CommonConfigurationKeysPublic.java | 1 +
.../apache/hadoop/fs/CommonPathCapabilities.java | 16 +
.../src/main/java/org/apache/hadoop/fs/DF.java | 2 +-
.../org/apache/hadoop/fs/FSDataInputStream.java | 8 +
.../java/org/apache/hadoop/fs/FileContext.java | 32 +-
.../main/java/org/apache/hadoop/fs/FileSystem.java | 38 +-
.../main/java/org/apache/hadoop/fs/FileUtil.java | 2 +-
.../main/java/org/apache/hadoop/fs/Options.java | 65 +-
.../org/apache/hadoop/fs/RawLocalFileSystem.java | 2 +
.../java/org/apache/hadoop/fs/RemoteIterator.java | 4 +-
.../org/apache/hadoop/fs/VectoredReadUtils.java | 26 +-
.../java/org/apache/hadoop/fs/impl/FlagSet.java | 327 +++++
.../fs/statistics/FileSystemStatisticNames.java | 26 +-
.../hadoop/fs/statistics/StoreStatisticNames.java | 6 +
.../java/org/apache/hadoop/ha/HAServiceTarget.java | 2 +-
.../org/apache/hadoop/ha/ShellCommandFencer.java | 10 +-
.../java/org/apache/hadoop/io/EnumSetWritable.java | 18 +-
.../java/org/apache/hadoop/io/ObjectWritable.java | 4 +-
.../java/org/apache/hadoop/io/SequenceFile.java | 10 +-
.../io/compress/bzip2/CBZip2InputStream.java | 8 +-
.../io/compress/bzip2/CBZip2OutputStream.java | 82 +-
.../io/compress/zlib/BuiltInZlibDeflater.java | 2 +-
.../io/erasurecode/rawcoder/RSRawDecoder.java | 32 +-
.../org/apache/hadoop/io/file/tfile/Chunk.java | 4 +-
.../org/apache/hadoop/io/wrappedio/WrappedIO.java | 182 ++-
.../hadoop/io/wrappedio/WrappedStatistics.java | 357 ++++++
.../hadoop/io/wrappedio/impl/DynamicWrappedIO.java | 500 ++++++++
.../wrappedio/impl/DynamicWrappedStatistics.java | 678 ++++++++++
.../hadoop/io/wrappedio/impl/package-info.java | 18 +-
.../apache/hadoop/io/wrappedio/package-info.java | 28 +-
.../org/apache/hadoop/ipc/RpcClientException.java | 2 +-
.../java/org/apache/hadoop/ipc/RpcException.java | 2 +-
.../org/apache/hadoop/ipc/RpcServerException.java | 2 +-
.../main/java/org/apache/hadoop/ipc/Server.java | 17 +-
.../hadoop/ipc/UnexpectedServerException.java | 2 +-
.../org/apache/hadoop/metrics2/package-info.java | 26 +-
.../main/java/org/apache/hadoop/net/NetUtils.java | 6 +-
.../org/apache/hadoop/oncrpc/RpcDeniedReply.java | 2 +-
.../hadoop/security/AccessControlException.java | 6 +-
.../org/apache/hadoop/security/ProviderUtils.java | 2 +-
.../org/apache/hadoop/security/SaslConstants.java | 2 +-
.../org/apache/hadoop/security/SaslRpcClient.java | 11 +-
.../hadoop/security/alias/CredentialShell.java | 2 +-
.../security/authorize/AuthorizationException.java | 6 +-
.../hadoop/security/ssl/SSLHostnameVerifier.java | 8 +-
.../java/org/apache/hadoop/util/ClassUtil.java | 22 +-
.../apache/hadoop/util/ConfigurationHelper.java | 126 ++
.../apache/hadoop/util/GenericOptionsParser.java | 2 +-
.../apache/hadoop/util/InstrumentedReadLock.java | 2 +-
.../hadoop/util/InstrumentedReadWriteLock.java | 2 +-
.../apache/hadoop/util/InstrumentedWriteLock.java | 2 +-
.../main/java/org/apache/hadoop/util/Shell.java | 6 +-
.../apache/hadoop/util/ShutdownThreadsHelper.java | 16 +-
.../java/org/apache/hadoop/util/StringUtils.java | 4 +-
.../apache/hadoop/util/dynamic/BindingUtils.java | 214 ++++
.../hadoop/util/dynamic/DynConstructors.java | 273 ++++
.../org/apache/hadoop/util/dynamic/DynMethods.java | 544 ++++++++
.../package-info.java} | 25 +-
.../util/functional/BiFunctionRaisingIOE.java | 16 +
.../hadoop/util/functional/CallableRaisingIOE.java | 19 +
.../util/functional/CommonCallableSupplier.java | 5 +-
.../hadoop/util/functional/FunctionRaisingIOE.java | 15 +
.../hadoop/util/functional/FunctionalIO.java | 89 ++
.../apache/hadoop/util/functional/FutureIO.java | 50 +-
.../util/functional/LazyAtomicReference.java | 152 +++
.../functional/LazyAutoCloseableReference.java | 102 ++
.../org/apache/hadoop/util/functional/Tuples.java | 17 +
.../src/org/apache/hadoop/crypto/OpensslCipher.c | 26 +-
.../apache/hadoop/io/erasurecode/erasure_coder.c | 36 +-
.../apache/hadoop/io/erasurecode/erasure_coder.h | 1 -
.../hadoop/io/erasurecode/erasure_code_test.c | 80 +-
.../src/main/resources/core-default.xml | 10 +
.../src/site/markdown/Benchmarking.md | 2 +
.../src/site/markdown/NativeLibraries.md.vm | 2 +-
.../src/site/markdown/filesystem/bulkdelete.md | 1 +
.../site/markdown/filesystem/fsdatainputstream.md | 1 -
.../filesystem/fsdatainputstreambuilder.md | 95 +-
.../filesystem/fsdataoutputstreambuilder.md | 4 +-
.../org/apache/hadoop/crypto/TestCryptoCodec.java | 13 +-
...tCryptoStreamsWithOpensslSm4CtrCryptoCodec.java | 2 +
.../apache/hadoop/crypto/TestOpensslCipher.java | 10 +
.../apache/hadoop/fs/FSMainOperationsBaseTest.java | 6 +-
.../hadoop/fs/FileContextCreateMkdirBaseTest.java | 21 +-
.../fs/FileContextMainOperationsBaseTest.java | 8 +-
.../fs/TestFSMainOperationsLocalFileSystem.java | 26 +-
.../contract/AbstractContractBulkDeleteTest.java | 40 +-
.../fs/contract/AbstractContractMkdirTest.java | 7 +-
.../contract/AbstractContractVectoredReadTest.java | 11 +
.../hadoop/fs/contract/ContractTestUtils.java | 18 +
.../org/apache/hadoop/fs/impl/TestFlagSet.java | 431 +++++++
.../hadoop/fs/impl/TestVectoredReadUtils.java | 42 +-
.../java/org/apache/hadoop/fs/shell/TestCount.java | 2 +-
.../TestFSMainOperationsLocalFileSystem.java | 9 -
.../apache/hadoop/ha/TestShellCommandFencer.java | 2 +-
.../TestErasureCodingEncodeAndDecode.java | 108 ++
.../hadoop/io/wrappedio/impl/TestWrappedIO.java | 484 ++++++++
.../io/wrappedio/impl/TestWrappedStatistics.java | 496 ++++++++
.../org/apache/hadoop/ipc/MiniRPCBenchmark.java | 12 +-
.../test/java/org/apache/hadoop/ipc/TestIPC.java | 2 +-
.../test/java/org/apache/hadoop/ipc/TestRPC.java | 8 +-
.../alias/TestCredentialProviderFactory.java | 2 +-
.../launcher/TestServiceInterruptHandling.java | 9 +-
.../java/org/apache/hadoop/util/TestClassUtil.java | 44 +-
.../hadoop/util/TestConfigurationHelper.java | 174 +++
.../java/org/apache/hadoop/util/TestShell.java | 2 +-
.../apache/hadoop/util/dynamic/Concatenator.java | 85 ++
.../hadoop/util/dynamic/TestDynConstructors.java | 170 +++
.../apache/hadoop/util/dynamic/TestDynMethods.java | 320 +++++
.../hadoop/util/functional/TestFunctionalIO.java | 111 ++
.../hadoop/util/functional/TestLazyReferences.java | 263 ++++
.../util/functional/TestRemoteIterators.java | 2 +-
.../src/test/resources/log4j.properties | 4 +-
.../hadoop/crypto/key/kms/server/TestKMS.java | 2 +-
.../hadoop/registry/server/dns/RegistryDNS.java | 2 +-
.../registry/server/dns/TestRegistryDNS.java | 8 +-
.../hadoop/hdfs/client/HdfsClientConfigKeys.java | 3 +
.../hdfs/client/impl/BlockReaderLocalLegacy.java | 2 +-
.../datatransfer/sasl/DataTransferSaslUtil.java | 6 +-
.../datatransfer/sasl/SaslDataTransferClient.java | 9 +-
.../datatransfer/sasl/SaslParticipant.java | 16 +-
.../hadoop/hdfs/util/CombinedHostsFileReader.java | 38 +
.../src/main/native/libhdfs/jni_helper.c | 19 +-
.../src/main/native/libhdfspp/tests/CMakeLists.txt | 4 +
.../native/libhdfspp/tests/libhdfs_getjni_test.cc | 33 +-
.../MultipleDestinationMountTableResolver.java | 5 +-
.../resolver/order/DestinationOrder.java | 7 +-
.../resolver/order/LeaderFollowerResolver.java | 50 +
.../server/federation/router/RouterHttpServer.java | 11 +
.../federation/router/RouterStateIdContext.java | 8 +-
.../store/records/impl/pb/MountTablePBImpl.java | 4 +
.../hdfs/tools/federation/AddMountAttributes.java | 1 +
.../hadoop/hdfs/tools/federation/RouterAdmin.java | 10 +-
.../src/main/proto/FederationProtocol.proto | 1 +
.../src/site/markdown/HDFSRouterFederation.md | 4 +
.../resolver/TestMultipleDestinationResolver.java | 17 +
.../resolver/order/TestLeaderFollowerResolver.java | 66 +
.../federation/router/TestObserverWithRouter.java | 40 +
.../federation/router/TestRouterAdminCLI.java | 37 +-
.../router/TestRouterHttpServerXFrame.java | 65 +
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 8 +
.../sasl/CustomizedCallbackHandler.java | 60 +
.../datatransfer/sasl/SaslDataTransferServer.java | 42 +-
.../qjournal/protocol/InterQJournalProtocol.java | 10 +
...nterQJournalProtocolServerSideTranslatorPB.java | 16 +
.../InterQJournalProtocolTranslatorPB.java | 14 +
.../hdfs/qjournal/server/JournalNodeRpcServer.java | 28 +-
.../hdfs/qjournal/server/JournalNodeSyncer.java | 81 +-
.../hadoop/hdfs/server/balancer/Dispatcher.java | 8 +-
.../hdfs/server/blockmanagement/BlockManager.java | 4 +-
.../blockmanagement/BlockReconstructionWork.java | 2 +-
.../blockmanagement/CombinedHostFileManager.java | 11 +-
.../server/blockmanagement/DatanodeDescriptor.java | 2 +-
.../server/blockmanagement/DatanodeManager.java | 10 +-
.../server/blockmanagement/ErasureCodingWork.java | 8 +-
.../server/blockmanagement/ReplicationWork.java | 3 +-
.../org/apache/hadoop/hdfs/server/mover/Mover.java | 18 +-
.../server/namenode/EncryptionZoneManager.java | 2 +-
.../server/namenode/FSDirEncryptionZoneOp.java | 50 +-
.../hadoop/hdfs/server/namenode/FSImage.java | 13 +-
.../hdfs/server/namenode/FSImageFormatPBINode.java | 2 +
.../server/namenode/FSImageFormatProtobuf.java | 52 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 18 +-
.../hdfs/server/namenode/FsImageValidation.java | 7 +-
.../hadoop/hdfs/server/namenode/NameNode.java | 2 +-
.../hdfs/server/namenode/snapshot/DiffList.java | 2 +-
.../namenode/snapshot/FSImageFormatPBSnapshot.java | 4 +-
.../java/org/apache/hadoop/hdfs/tools/DFSck.java | 2 +-
.../offlineEditsViewer/OfflineEditsXmlLoader.java | 7 +-
.../FileDistributionCalculator.java | 18 +-
.../FileDistributionVisitor.java | 16 +-
.../offlineImageViewer/PBImageCorruption.java | 2 +-
.../src/main/proto/InterQJournalProtocol.proto | 7 +
.../src/main/resources/hdfs-default.xml | 36 +-
.../hadoop/fs/contract/hdfs/TestDFSWrappedIO.java | 39 +-
.../java/org/apache/hadoop/hdfs/DFSTestUtil.java | 2 +-
.../hadoop/hdfs/TestDecommissionWithStriped.java | 54 +
.../org/apache/hadoop/hdfs/TestFileChecksum.java | 55 +-
.../java/org/apache/hadoop/hdfs/TestHDFSTrash.java | 2 +-
.../java/org/apache/hadoop/hdfs/TestSafeMode.java | 2 +-
.../sasl/TestCustomizedCallbackHandler.java | 119 ++
.../datatransfer/sasl/TestSaslDataTransfer.java | 2 +-
.../hdfs/qjournal/server/TestJournalNodeSync.java | 44 +
.../server/blockmanagement/TestBlockManager.java | 8 +-
.../blockmanagement/TestRedundancyMonitor.java | 2 +-
.../hdfs/server/datanode/TestBPOfferService.java | 5 +-
.../server/datanode/TestReadOnlySharedStorage.java | 6 +-
.../datanode/fsdataset/impl/TestFsDatasetImpl.java | 2 +-
.../apache/hadoop/hdfs/server/mover/TestMover.java | 148 +++
.../server/namenode/NNThroughputBenchmark.java | 170 ++-
.../server/namenode/TestFSDirEncryptionZoneOp.java | 59 +
.../hadoop/hdfs/server/namenode/TestFSImage.java | 17 +-
.../server/namenode/TestFSImageWithSnapshot.java | 40 +-
...TestFSImageWithSnapshotParallelAndCompress.java | 58 +
.../namenode/TestFSNamesystemLockReport.java | 18 +-
.../server/namenode/TestNNThroughputBenchmark.java | 83 +-
.../hdfs/server/namenode/TestNameNodeMXBean.java | 64 +
.../web/resources/TestWebHdfsDataLocality.java | 2 +-
.../hdfs/util/TestCombinedHostsFileReader.java | 57 +
.../v2/app/rm/preemption/AMPreemptionPolicy.java | 2 +-
.../mapreduce/v2/app/webapp/AMWebServices.java | 15 +-
.../hadoop/mapreduce/v2/app/webapp/ConfBlock.java | 2 +-
.../hadoop/mapreduce/v2/app/TestRecovery.java | 2 +-
.../v2/app/webapp/TestAMWebServicesAttempts.java | 2 +-
.../v2/app/webapp/TestAMWebServicesJobs.java | 2 +-
.../org/apache/hadoop/mapred/LocalJobRunner.java | 4 +-
.../hadoop/mapreduce/v2/util/MRWebAppUtil.java | 2 +-
.../hadoop/mapred/TestLocalModeWithNewApis.java | 2 +-
.../org/apache/hadoop/mapred/FileInputFormat.java | 2 +-
.../org/apache/hadoop/mapred/FileOutputFormat.java | 16 +-
.../hadoop/mapred/InvalidInputException.java | 2 +-
.../java/org/apache/hadoop/mapred/JobConf.java | 4 +-
.../java/org/apache/hadoop/mapred/MapRunnable.java | 2 +-
.../org/apache/hadoop/mapred/MultiFileSplit.java | 2 +-
.../org/apache/hadoop/mapred/SortedRanges.java | 2 +-
.../java/org/apache/hadoop/mapred/TaskLog.java | 6 +-
.../org/apache/hadoop/mapred/jobcontrol/Job.java | 2 +-
.../hadoop/mapred/join/CompositeInputFormat.java | 12 +-
.../hadoop/mapred/join/CompositeRecordReader.java | 4 +-
.../hadoop/mapred/join/OverrideRecordReader.java | 2 +-
.../java/org/apache/hadoop/mapred/join/Parser.java | 2 +-
.../hadoop/mapred/lib/FieldSelectionMapReduce.java | 2 +-
.../hadoop/mapred/lib/TotalOrderPartitioner.java | 2 +-
.../main/java/org/apache/hadoop/mapreduce/Job.java | 2 +-
.../org/apache/hadoop/mapreduce/JobStatus.java | 2 +-
.../hadoop/mapreduce/TaskCompletionEvent.java | 2 +-
.../mapreduce/lib/aggregate/ValueHistogram.java | 4 +-
.../lib/fieldsel/FieldSelectionHelper.java | 8 +-
.../lib/input/CombineFileInputFormat.java | 2 +-
.../mapreduce/lib/input/CombineFileSplit.java | 4 +-
.../mapreduce/lib/input/FileInputFormat.java | 2 +-
.../mapreduce/lib/input/InvalidInputException.java | 2 +-
.../mapreduce/lib/jobcontrol/ControlledJob.java | 4 +-
.../mapreduce/lib/join/CompositeInputFormat.java | 12 +-
.../mapreduce/lib/join/CompositeRecordReader.java | 4 +-
.../mapreduce/lib/join/OverrideRecordReader.java | 2 +-
.../apache/hadoop/mapreduce/lib/join/Parser.java | 2 +-
.../hadoop/mapreduce/lib/join/TupleWritable.java | 4 +-
.../mapreduce/lib/output/FileOutputFormat.java | 8 +-
.../lib/partition/TotalOrderPartitioner.java | 10 +-
.../apache/hadoop/mapreduce/split/JobSplit.java | 2 +-
.../hadoop/mapreduce/task/reduce/Fetcher.java | 8 +-
.../task/reduce/ShuffleSchedulerImpl.java | 2 +-
.../org/apache/hadoop/mapreduce/tools/CLI.java | 2 +-
.../hadoop/mapred/TestFileOutputCommitter.java | 2 +-
.../lib/output/TestFileOutputCommitter.java | 2 +-
.../hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java | 2 +-
.../mapreduce/v2/hs/webapp/HsWebServices.java | 11 +-
.../hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java | 2 +-
.../v2/hs/webapp/TestHsWebServicesAcls.java | 46 +-
.../v2/hs/webapp/TestHsWebServicesAttempts.java | 2 +-
.../mapreduce/v2/hs/webapp/VerifyJobsUtils.java | 2 +-
.../org/apache/hadoop/RandomTextWriterJob.java | 2 +-
.../org/apache/hadoop/fs/AccumulatingReducer.java | 10 +-
.../java/org/apache/hadoop/fs/IOMapperBase.java | 4 +-
.../java/org/apache/hadoop/fs/JHLogAnalyzer.java | 44 +-
.../java/org/apache/hadoop/mapred/MRBench.java | 2 +-
.../mapred/TestConcatenatedCompressedInput.java | 2 +-
.../hadoop/mapred/TestFixedLengthInputFormat.java | 2 +-
.../mapred/TestMRCJCFileOutputCommitter.java | 2 +-
.../org/apache/hadoop/mapred/TestMapProgress.java | 2 +-
.../java/org/apache/hadoop/mapred/TestMapRed.java | 2 +-
.../apache/hadoop/mapred/TestMiniMRClasspath.java | 4 +-
.../mapred/TestMultipleTextOutputFormat.java | 8 +-
.../apache/hadoop/mapred/TestTextInputFormat.java | 2 +-
.../org/apache/hadoop/mapred/UtilsForTests.java | 6 +-
.../mapred/jobcontrol/JobControlTestUtils.java | 2 +-
.../apache/hadoop/mapreduce/MapReduceTestUtil.java | 6 +-
.../apache/hadoop/mapreduce/RandomTextWriter.java | 2 +-
.../lib/input/TestFixedLengthInputFormat.java | 2 +-
.../lib/output/TestMRCJCFileOutputCommitter.java | 2 +-
.../hadoop/mapreduce/v2/MiniMRYarnCluster.java | 2 +-
.../apache/hadoop/examples/RandomTextWriter.java | 2 +-
.../apache/hadoop/examples/dancing/Pentomino.java | 2 +-
.../org/apache/hadoop/examples/dancing/Sudoku.java | 2 +-
.../org/apache/hadoop/examples/pi/package.html | 71 +-
.../hadoop/examples/terasort/TeraScheduler.java | 4 +-
hadoop-project/pom.xml | 60 +-
.../hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java | 17 +
.../hadoop-aws/dev-support/findbugs-exclude.xml | 5 -
hadoop-tools/hadoop-aws/pom.xml | 28 +-
.../java/org/apache/hadoop/fs/s3a/Constants.java | 20 +
.../hadoop/fs/s3a/DefaultS3ClientFactory.java | 16 +-
.../apache/hadoop/fs/s3a/S3ABlockOutputStream.java | 3 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 344 +++--
.../org/apache/hadoop/fs/s3a/S3AInputPolicy.java | 24 +-
.../java/org/apache/hadoop/fs/s3a/S3AStore.java | 9 +-
.../java/org/apache/hadoop/fs/s3a/S3AUtils.java | 22 +-
.../org/apache/hadoop/fs/s3a/S3ClientFactory.java | 2 -
.../java/org/apache/hadoop/fs/s3a/Statistic.java | 16 +
.../hadoop/fs/s3a/api/PerformanceFlagEnum.java} | 38 +-
.../fs/s3a/auth/AssumedRoleCredentialProvider.java | 5 +
.../delegation/EncryptionSecretOperations.java | 16 +
.../fs/s3a/auth/delegation/EncryptionSecrets.java | 35 +-
.../fs/s3a/commit/AbstractS3ACommitterFactory.java | 5 +-
.../hadoop/fs/s3a/commit/S3ACommitterFactory.java | 7 +-
.../apache/hadoop/fs/s3a/impl/ClientManager.java | 40 +-
.../hadoop/fs/s3a/impl/ClientManagerImpl.java | 238 ++++
.../apache/hadoop/fs/s3a/impl/MkdirOperation.java | 77 +-
.../hadoop/fs/s3a/impl/RequestFactoryImpl.java | 14 +
.../apache/hadoop/fs/s3a/impl/S3AEncryption.java | 106 ++
.../apache/hadoop/fs/s3a/impl/S3AStoreBuilder.java | 21 +-
.../apache/hadoop/fs/s3a/impl/S3AStoreImpl.java | 121 +-
.../apache/hadoop/fs/s3a/impl/StoreContext.java | 19 +-
.../hadoop/fs/s3a/impl/StoreContextBuilder.java | 17 +-
.../apache/hadoop/fs/s3a/s3guard/S3GuardTool.java | 9 +-
.../markdown/tools/hadoop-aws/assumed_roles.md | 8 +
.../site/markdown/tools/hadoop-aws/encryption.md | 30 +
.../src/site/markdown/tools/hadoop-aws/index.md | 14 +
.../site/markdown/tools/hadoop-aws/performance.md | 129 +-
.../src/site/markdown/tools/hadoop-aws/testing.md | 50 +-
.../fs/contract/s3a/ITestS3AContractCreate.java | 10 +-
.../fs/contract/s3a/ITestS3AContractMkdir.java | 9 +
.../s3a/ITestS3AContractMkdirWithCreatePerf.java | 68 +
.../fs/contract/s3a/ITestS3AContractRootDir.java | 8 +
.../contract/s3a/ITestS3AContractVectoredRead.java | 4 +-
.../hadoop/fs/contract/s3a/ITestS3AWrappedIO.java | 25 +-
.../apache/hadoop/fs/s3a/AbstractS3AMockTest.java | 9 +
.../hadoop/fs/s3a/AbstractTestS3AEncryption.java | 2 +
.../hadoop/fs/s3a/ITestS3AConfiguration.java | 6 +-
.../hadoop/fs/s3a/ITestS3AEncryptionSSEC.java | 92 +-
...stS3AEncryptionSSEKMSWithEncryptionContext.java | 101 ++
.../hadoop/fs/s3a/ITestS3AEndpointRegion.java | 13 +-
.../hadoop/fs/s3a/ITestS3AFSMainOperations.java | 6 +-
.../hadoop/fs/s3a/ITestS3AFileOperationCost.java | 10 +-
.../hadoop/fs/s3a/ITestS3AFileSystemContract.java | 5 +-
.../apache/hadoop/fs/s3a/MockS3AFileSystem.java | 7 +
.../org/apache/hadoop/fs/s3a/S3ATestConstants.java | 12 +
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 36 +-
.../fs/s3a/TestS3AAWSCredentialsProvider.java | 8 +-
.../hadoop/fs/s3a/TestS3AEndpointParsing.java | 43 +
.../apache/hadoop/fs/s3a/TestSSEConfiguration.java | 69 +-
.../apache/hadoop/fs/s3a/auth/ITestAssumeRole.java | 3 +-
.../fs/s3a/auth/TestMarshalledCredentials.java | 3 +-
.../delegation/ITestSessionDelegationTokens.java | 6 +-
.../delegation/TestS3ADelegationTokenSupport.java | 24 +-
.../fs/s3a/commit/AbstractITCommitProtocol.java | 2 +-
.../fs/s3a/commit/ITestS3ACommitterFactory.java | 234 ++--
.../fs/s3a/commit/staging/StagingTestBase.java | 11 +-
.../fs/s3a/commit/terasort/ITestTerasortOnS3A.java | 22 +-
.../ITestS3AFileContextCreateMkdir.java | 9 +-
... ITestS3AFileContextCreateMkdirCreatePerf.java} | 32 +-
.../ITestS3AFileContextMainOperations.java | 37 +-
.../fs/s3a/fileContext/ITestS3AFileContextURI.java | 6 +-
.../fs/s3a/impl/ITestConnectionTimeouts.java | 4 +-
.../hadoop/fs/s3a/impl/TestClientManager.java | 379 ++++++
.../hadoop/fs/s3a/impl/TestOpenFileSupport.java | 43 +-
.../hadoop/fs/s3a/impl/TestRequestFactory.java | 2 +-
.../hadoop/fs/s3a/impl/TestS3AEncryption.java | 77 ++
.../fs/s3a/performance/AbstractS3ACostTest.java | 3 +-
.../fs/s3a/performance/ITestCreateFileCost.java | 17 +-
.../performance/ITestDirectoryMarkerListing.java | 4 +-
.../fs/s3a/performance/ITestS3ADeleteCost.java | 10 +-
.../fs/s3a/performance/ITestS3AOpenCost.java | 2 +
.../fs/s3a/scale/AbstractSTestS3AHugeFiles.java | 10 +
.../hadoop/fs/s3a/scale/ITestS3AConcurrentOps.java | 5 +-
.../hadoop/fs/s3a/scale/S3AScaleTestBase.java | 2 +-
.../hadoop/fs/s3a/test/StubS3ClientFactory.java | 122 ++
.../fs/s3a/tools/AbstractMarkerToolTest.java | 3 +-
.../s3a/tools/ITestMarkerToolRootOperations.java | 2 +
.../org/apache/hadoop/fs/s3a/yarn/ITestS3A.java | 41 +-
.../org/apache/hadoop/fs/sdk/TestAWSV2SDK.java | 94 ++
.../hadoop-aws/src/test/resources/log4j.properties | 2 +
.../src/config/checkstyle-suppressions.xml | 2 +
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 146 ++-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 85 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 165 ++-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 47 +
.../fs/azurebfs/constants/AbfsServiceType.java | 29 +-
.../fs/azurebfs/constants/AuthConfigurations.java | 4 +
.../fs/azurebfs/constants/ConfigurationKeys.java | 42 +-
.../fs/azurebfs/constants/FSOperationType.java | 3 +-
.../constants/FileSystemConfigurations.java | 10 +
.../azurebfs/constants/FileSystemUriSchemes.java | 5 +-
.../fs/azurebfs/constants/HttpOperationType.java | 11 +-
.../AbfsApacheHttpExpect100Exception.java | 15 +-
...ueException.java => HttpResponseException.java} | 23 +-
.../InvalidConfigurationValueException.java | 4 +
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 52 +-
.../oauth2/WorkloadIdentityTokenProvider.java | 142 +++
.../fs/azurebfs/services/AbfsAHCHttpOperation.java | 394 ++++++
.../fs/azurebfs/services/AbfsApacheHttpClient.java | 144 +++
.../hadoop/fs/azurebfs/services/AbfsClient.java | 1308 +++++++-------------
.../fs/azurebfs/services/AbfsClientHandler.java | 127 ++
.../services/AbfsClientThrottlingIntercept.java | 3 +-
.../azurebfs/services/AbfsConnectionManager.java | 207 ++++
.../hadoop/fs/azurebfs/services/AbfsDfsClient.java | 1302 +++++++++++++++++++
.../services/AbfsHttpClientConnectionFactory.java | 45 +
.../fs/azurebfs/services/AbfsHttpOperation.java | 536 ++++----
.../fs/azurebfs/services/AbfsJdkHttpOperation.java | 300 +++++
.../services/AbfsManagedApacheHttpConnection.java | 240 ++++
.../services/AbfsManagedHttpClientContext.java | 76 ++
.../services/AbfsManagedHttpRequestExecutor.java | 109 ++
.../services/AbfsNoOpThrottlingIntercept.java | 8 +-
.../fs/azurebfs/services/AbfsRestOperation.java | 142 ++-
.../azurebfs/services/AbfsThrottlingIntercept.java | 5 +-
.../fs/azurebfs/services/KeepAliveCache.java | 306 +++++
.../fs/azurebfs/services/SharedKeyCredentials.java | 17 +-
.../hadoop/fs/azurebfs/utils/NamespaceUtil.java | 88 --
.../hadoop/fs/azurebfs/utils/TracingContext.java | 1 +
.../apache/hadoop/fs/azurebfs/utils/UriUtils.java | 36 +
.../hadoop-azure/src/site/markdown/abfs.md | 94 +-
.../hadoop-azure/src/site/markdown/fns_blob.md | 82 ++
.../hadoop-azure/src/site/markdown/index.md | 2 +
.../src/site/markdown/testing_azure.md | 36 +
.../hadoop-azure/src/site/markdown/wasb.md | 97 ++
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 14 +-
.../fs/azurebfs/ITestAbfsCustomEncryption.java | 91 +-
.../fs/azurebfs/ITestAbfsReadWriteAndSeek.java | 64 +-
.../ITestAzureBlobFileSystemAuthorization.java | 3 +-
.../ITestAzureBlobFileSystemCheckAccess.java | 9 +-
.../azurebfs/ITestAzureBlobFileSystemCreate.java | 9 +-
.../ITestAzureBlobFileSystemDelegationSAS.java | 2 +-
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 4 +-
.../fs/azurebfs/ITestAzureBlobFileSystemE2E.java | 17 +-
.../ITestAzureBlobFileSystemInitAndCreate.java | 87 ++
.../fs/azurebfs/ITestAzureBlobFileSystemLease.java | 25 +-
.../fs/azurebfs/ITestGetNameSpaceEnabled.java | 62 +-
.../fs/azurebfs/TestAccountConfiguration.java | 122 +-
.../hadoop/fs/azurebfs/TestTracingContext.java | 14 +-
.../fs/azurebfs/contract/ITestAbfsWrappedIO.java | 53 +
.../extensions/MockDelegationSASTokenProvider.java | 9 +-
.../oauth2/TestWorkloadIdentityTokenProvider.java | 144 +++
.../hadoop/fs/azurebfs/oauth2/package-info.java | 17 +-
.../fs/azurebfs/services/AbfsClientTestUtil.java | 8 +-
.../fs/azurebfs/services/AbfsClientUtils.java | 3 -
.../fs/azurebfs/services/ITestAbfsClient.java | 140 ++-
.../ITestAbfsHttpClientRequestExecutor.java | 406 ++++++
.../azurebfs/services/ITestAbfsOutputStream.java | 46 +-
.../azurebfs/services/ITestAbfsRestOperation.java | 138 ++-
.../services/ITestApacheClientConnectionPool.java | 63 +
.../fs/azurebfs/services/TestAbfsPerfTracker.java | 36 +-
.../services/TestAbfsRenameRetryRecovery.java | 27 +-
.../azurebfs/services/TestAbfsRestOperation.java | 7 +-
.../TestAbfsRestOperationMockFailures.java | 15 +-
.../services/TestApacheClientConnectionPool.java | 290 +++++
.../services/TestApacheHttpClientFallback.java | 226 ++++
.../fs/azurebfs/utils/TracingHeaderValidator.java | 2 +-
.../src/test/resources/workload-identity-pod.yaml | 32 +
.../apache/hadoop/contrib/utils/join/JobBase.java | 2 +-
.../java/org/apache/hadoop/tools/DistCpSync.java | 2 +-
.../org/apache/hadoop/tools/util/DistCpUtils.java | 2 +-
.../org/apache/hadoop/mapred/gridmix/FilePool.java | 2 +-
.../hadoop/tools/rumen/datatypes/NodeName.java | 2 +-
.../org/apache/hadoop/streaming/PipeMapRed.java | 2 +-
.../org/apache/hadoop/streaming/StreamJob.java | 2 +-
.../hadoop/streaming/StreamXmlRecordReader.java | 2 -
.../hadoop/streaming/io/IdentifierResolver.java | 2 +-
.../java/org/apache/hadoop/streaming/package.html | 2 +-
.../java/org/apache/hadoop/typedbytes/package.html | 8 +-
.../hadoop/streaming/TestMultipleArchiveFiles.java | 2 +-
.../java/org/apache/hadoop/streaming/UtilTest.java | 2 +-
.../hadoop-yarn/dev-support/findbugs-exclude.xml | 9 +
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 6 +-
.../protocolrecords/SignalContainerRequest.java | 2 +-
.../records/timeline/TimelineEntityGroupId.java | 2 +-
.../yarn/api/resource/PlacementConstraint.java | 10 +-
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 19 +-
.../hadoop/yarn/service/ServiceScheduler.java | 106 +-
.../timelineservice/ServiceMetricsSink.java | 2 +-
.../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 2 +-
.../hadoop/yarn/security/AdminACLsManager.java | 4 +-
.../apache/hadoop/yarn/util/BoundedAppender.java | 2 +-
.../hadoop/yarn/util/ProcfsBasedProcessTree.java | 2 +-
.../apache/hadoop/yarn/webapp/view/JQueryUI.java | 4 +-
.../src/main/resources/yarn-default.xml | 6 +
.../logaggregation/TestAggregatedLogFormat.java | 2 +-
.../federation/cache/FederationCaffeineCache.java | 4 +-
.../federation/cache/FederationGuavaCache.java | 4 +-
.../server/federation/cache/FederationJCache.java | 4 +-
.../store/sql/FederationQueryRunner.java | 2 +-
.../hadoop/yarn/server/utils/LeveldbIterator.java | 2 +-
.../server/nodemanager/DirectoryCollection.java | 158 ++-
.../deletion/task/DockerContainerDeletionTask.java | 2 +-
.../privileged/PrivilegedOperationExecutor.java | 2 +-
.../resources/NetworkPacketTaggingHandlerImpl.java | 2 +-
.../TrafficControlBandwidthHandlerImpl.java | 4 +-
.../linux/resources/TrafficController.java | 2 +-
.../linux/runtime/docker/DockerCommand.java | 2 +-
.../localizer/LocalCacheDirectoryManager.java | 2 +-
.../com/nvidia/NvidiaGPUPluginForRuntimeV2.java | 2 +-
.../nodemanager/recovery/NMStateStoreService.java | 2 +-
.../nodemanager/TestDirectoryCollection.java | 36 +-
.../TestPrivilegedOperationExecutor.java | 2 +-
.../TestNetworkPacketTaggingHandlerImpl.java | 2 +-
.../TestTrafficControlBandwidthHandlerImpl.java | 2 +-
.../linux/resources/TestTrafficController.java | 2 +-
.../linux/runtime/TestDockerContainerRuntime.java | 2 +-
.../runtime/TestHdfsManifestToResourcesPlugin.java | 2 +-
.../runtime/TestImageTagToManifestPlugin.java | 2 +-
.../linux/runtime/TestRuncContainerRuntime.java | 2 +-
.../runtime/docker/TestDockerCommandExecutor.java | 4 +-
.../localizer/TestLocalCacheDirectoryManager.java | 2 +-
.../deviceframework/TestDevicePluginAdapter.java | 2 +-
.../scheduler/capacity/PlanQueue.java | 2 +-
.../resourcemanager/webapp/RMWebServices.java | 2 +-
.../webapp/dao/AllocationTagsInfo.java | 2 +-
.../constraint/TestPlacementProcessor.java | 2 +-
.../constraint/algorithm/TestCircularIterator.java | 8 +-
.../scheduler/fair/TestQueuePlacementPolicy.java | 62 +-
.../router/webapp/FederationInterceptorREST.java | 8 +-
.../timelineservice/storage/common/BaseTable.java | 2 +-
.../src/main/webapp/app/models/yarn-app-attempt.js | 2 +-
.../src/main/webapp/app/models/yarn-container.js | 2 +-
.../webapp/app/models/yarn-timeline-appattempt.js | 2 +-
.../webapp/app/models/yarn-timeline-container.js | 2 +-
start-build-env.sh | 2 +-
525 files changed, 19319 insertions(+), 3500 deletions(-)
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java
=>
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ClosedIOException.java
(64%)
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FlagSet.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java
=>
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/FileSystemStatisticNames.java
(66%)
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/wrappedio/WrappedStatistics.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/wrappedio/impl/DynamicWrappedIO.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/wrappedio/impl/DynamicWrappedStatistics.java
copy
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/BaseTable.java
=>
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/wrappedio/impl/package-info.java
(73%)
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java
=>
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/wrappedio/package-info.java
(61%)
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ConfigurationHelper.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/dynamic/BindingUtils.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/dynamic/DynConstructors.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/dynamic/DynMethods.java
copy
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/{functional/CallableRaisingIOE.java
=> dynamic/package-info.java} (63%)
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FunctionalIO.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/LazyAtomicReference.java
create mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/LazyAutoCloseableReference.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/impl/TestFlagSet.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestErasureCodingEncodeAndDecode.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/wrappedio/impl/TestWrappedIO.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/wrappedio/impl/TestWrappedStatistics.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestConfigurationHelper.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/dynamic/Concatenator.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/dynamic/TestDynConstructors.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/dynamic/TestDynMethods.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/functional/TestFunctionalIO.java
create mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/functional/TestLazyReferences.java
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsNoOpThrottlingIntercept.java
=>
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/libhdfs_getjni_test.cc
(52%)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/order/LeaderFollowerResolver.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/resolver/order/TestLeaderFollowerResolver.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterHttpServerXFrame.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/CustomizedCallbackHandler.java
copy
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FunctionRaisingIOE.java
=>
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/contract/hdfs/TestDFSWrappedIO.java
(53%)
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/TestCustomizedCallbackHandler.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSDirEncryptionZoneOp.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshotParallelAndCompress.java
copy
hadoop-tools/{hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java
=>
hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/api/PerformanceFlagEnum.java}
(62%)
copy
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RemoteIterator.java
=>
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ClientManager.java
(51%)
create mode 100644
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ClientManagerImpl.java
create mode 100644
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/S3AEncryption.java
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractMkdirWithCreatePerf.java
copy
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FunctionRaisingIOE.java
=>
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AWrappedIO.java
(65%)
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSEKMSWithEncryptionContext.java
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AEndpointParsing.java
copy
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/fileContext/{ITestS3AFileContextCreateMkdir.java
=> ITestS3AFileContextCreateMkdirCreatePerf.java} (54%)
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestClientManager.java
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestS3AEncryption.java
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/test/StubS3ClientFactory.java
create mode 100644
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/sdk/TestAWSV2SDK.java
copy
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FunctionRaisingIOE.java
=>
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsServiceType.java
(57%)
copy
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/BaseTable.java
=>
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpOperationType.java
(82%)
copy
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/BaseTable.java
=>
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsApacheHttpExpect100Exception.java
(59%)
copy
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/{InvalidConfigurationValueException.java
=> HttpResponseException.java} (59%)
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/WorkloadIdentityTokenProvider.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsAHCHttpOperation.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsApacheHttpClient.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientHandler.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsConnectionManager.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsDfsClient.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpClientConnectionFactory.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsJdkHttpOperation.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsManagedApacheHttpConnection.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsManagedHttpClientContext.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsManagedHttpRequestExecutor.java
create mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/KeepAliveCache.java
delete mode 100644
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/NamespaceUtil.java
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/fns_blob.md
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/wasb.md
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsWrappedIO.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/TestWorkloadIdentityTokenProvider.java
copy
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/BaseTable.java
=>
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/package-info.java
(78%)
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsHttpClientRequestExecutor.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestApacheClientConnectionPool.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestApacheClientConnectionPool.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestApacheHttpClientFallback.java
create mode 100644
hadoop-tools/hadoop-azure/src/test/resources/workload-identity-pod.yaml
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
