This is an automated email from the ASF dual-hosted git repository.
cnauroth pushed a change to branch HADOOP-19343
in repository https://gitbox.apache.org/repos/asf/hadoop.git
from 4235dc62687 HDFS-17772. Fix JournaledEditsCache int overflow while the
maximum capacity to be Integer MAX_VALUE. (#7617). Contributed by Guo Wei.
add e73722eef08 Revert "YARN-11765. Refactor: Move Clock Class from
hadoop-mapreduce-project to hadoop-common-project for Reusability (#7352)
Contributed by Jiandan Yang." (#7599) Contributed by Cheng Pan.
add a47df85360b HDFS-17226. Building native libraries fails on Fedora 38.
(#6191)
add 5770647dc73 HDFS-12431. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-hdfs Part3. (#7626)
add 08f4068080e HADOOP-19416. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-kms. (#7637)
add 1293a0ec28c HADOOP-19411. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-cos. (#7640)
add 751468d6fad HADOOP-19412. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-huaweicloud. (#7639)
add e08bb0b28ea HADOOP-19418. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-nfs. (#7635)
add 91923fb6399 HADOOP-19430. Upgrade JUnit from 4 to 5 in
hadoop-datajoin. (#7618)
add 809b1b59738 HADOOP-19427. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-compat-bench. (#7619)
add b6682d376a1 HADOOP-19419. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-registry. (#7624)
add 676027176de HADOOP-19423. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-archives. (#7620)
add 10cbe27d565 HADOOP-19406. ABFS: [FNSOverBlob] Support User Delegation
SAS for FNS Blob (#7523)
add 709d10e40f6 HADOOP-19532. Update commons-lang3 to 3.17.0 (#7591)
add ed7e7dabcc8 HADOOP-19542. S3A: Close AAL factory on service stop.
(#7616)
add 189c8b65e2e YARN-11813. Fix the fallback ordering between cgroup v2
and v1. (#7631)
add 96c380d1522 HADOOP-19410. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-client-integration-tests. (#7650)
add 72939fe79d6 HADOOP-19417. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-minikdc. (#7636)
add 7c7adefb65f YARN-11801: NPE in FifoCandidatesSelector.selectCandidates
when preempting resources for an auto-created queue without child queues (#7607)
add f9f7b292eb8 HADOOP-19548: [ABFS] Fix Logging in FSDataInputStream to
Mention Correct Buffersize (#7642)
add cfde3a3f645 HADOOP-19545. Update to ApacheDS 2.0.0.AM27 and ldap-api
2.1.7 (#7628) Contributed by Istvan Toth.
add db81ea603cf HADOOP-19555. Fix testRenameFileWithFullQualifiedPath on
Windows (#7654)
add b949ca64a15 HADOOP-19557 S3A: S3ABlockOutputStream to never log/reject
hflush() calls (#7662)
add a4130c8818a HADOOP-19414. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-auth. (#7638)
add e442a32a8ba YARN-11815: Fix NodeQueueLoadMonitor scheduler running on
standby RMs (#7665)
add 26a0fdaa755 HADOOP-19421. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-aliyun. (#7634)
add e57face3fc0 HADOOP-19417. [Addendum] [JDK17] Upgrade JUnit from 4 to 5
in hadoop-minikdc. (#7661)
add e10314226da HADOOP-19560. Update build instructions for Windows (#7673)
add b0a71903ae6 HADOOP-19558. Skip testRenameFileBeingAppended on Windows
(#7666)
add 2ae4de8b688 YARN-11818. Fix "submitted by user jenkins to unknown
queue: default" error in hadoop-yarn-client tests. (#7675) Contributed by
Shilun Fan.
add 5279af9b2fb HADOOP-19540. [JDK17] Add ubuntu:noble as a build platform
with JDK-17 as default (#7608)
add 7ea3f51287c HADOOP-19551. Fix compilation error of native libraries on
newer GCC. (#7644)
add 90a6f3b2021 HADOOP-19426. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-azure-datalake Part1. (#7652)
add 4272a80dabf HADOOP-19477: Update GitHub Actions to latest versions
add 2661feb6db2 Revert "HADOOP-19477: Update GitHub Actions to latest
versions"
add 1c0c3004718 HADOOP-19384. S3A: Add support for
ProfileCredentialsProvider (#7284)
add 47ad1c0cdf0 YARN-11793: Replace Grizzly with Jetty in Jersey Test
Framework (#7497)
add 4769febaccc HADOOP-19554. LocalDirAllocator still doesn't always
recover from directory deletion (#7651)
add cd68d134989 Revert " HADOOP-19554. LocalDirAllocator still doesn't
always recover from directory deletion (#7651)"
add 81e4848a7e7 HADOOP-19554. LocalDirAllocator still doesn't always
recover from directory deletion (#7683)
add 356fd6aea33 HADOOP-19550. Migrate ViewFileSystemBaseTest to Junit 5
add eec726ebe01 HDFS-17767. Skip tests that depend on custom
SecurityManager when Java doesn't support it
add 9c05d9667bc HADOOP-19526. Skip tests in Hadoop common that depend on
SecurityManager if the JVM does not support it
add b04db8283c0 HADOOP-19563. Upgrade libopenssl to 3.1.2 on Windows
(#7680)
add f34c4be413d HADOOP-19433. [JDK17] Upgrade JUnit from 4 to 5 in
hadoop-extras. (#7586)
add 912c26035e0 HADOOP-19562. Fix TestTextCommand on Windows (#7679)
add b44ce0ae032 HADOOP-19570. Upgrade libxxhash to 0.8.3 in Windows 10
(#7689)
add 20f40119aa3 HADOOP-19530. Add --enable-native-access=ALL-UNNAMED JVM
option
add d491f0baf68 HADOOP-19573. S3A:
ITestS3AConfiguration.testDirectoryAllocatorDefval() failing (#7699)
add 66adc68b620 HADOOP-19572. [ABFS][BugFix] Empty Page Issue on
Subsequent ListBlob call with NextMarker (#7698)
add 949292eac6d HADOOP-19571. Improve PrometheusMetricsSink#normalizeName
performance (#7692) Contributed by Ivan Andika.
add f347b64fb7d HADOOP-19567. S3A: error stack traces printed on analytics
stream factory close (#7701)
add 43b5183dbbf YARN-11816. Fix flaky test:
TestCapacitySchedulerMultiNodes#testCheckRequestOnceForUnsatisfiedRequest.
(#7659) Contributed by Tao Yang.
add 58fbe4a9554 HDFS-17788. [ARR] getFileInfo not handle exception rightly
which may cause FileNotFoundException in DistributedFileSystem. (#7703).
Contributed by hfutatzhanghb.
add 31e24808e6b HADOOP-19343: GCS / Add implementation for create() API
No new revisions were added by this update.
Summary of changes:
BUILDING.txt | 24 +-
LICENSE-binary | 4 +-
.../docker/{Dockerfile => Dockerfile_ubuntu_24} | 19 +-
dev-support/docker/Dockerfile_windows_10 | 12 +-
dev-support/docker/pkg-resolver/packages.json | 66 +-
dev-support/docker/pkg-resolver/platforms.json | 1 +
.../hadoop-client-integration-tests/pom.xml | 20 +
.../apache/hadoop/example/ITUseHadoopCodecs.java | 10 +-
.../apache/hadoop/example/ITUseMiniCluster.java | 15 +-
.../hadoop-client-minicluster/pom.xml | 2 +-
.../hadoop-client-runtime/pom.xml | 7 +
.../apache/hadoop/fs/cosn/TestCosCredentials.java | 8 +-
.../apache/hadoop/fs/cosn/TestCosNInputStream.java | 36 +-
.../hadoop/fs/cosn/TestCosNOutputStream.java | 17 +-
.../hadoop/fs/obs/TestOBSFSMainOperations.java | 21 +-
.../fs/obs/TestOBSFileContextCreateMkdir.java | 11 +-
.../fs/obs/TestOBSFileContextMainOperations.java | 15 +-
.../hadoop/fs/obs/TestOBSFileContextURI.java | 13 +-
.../hadoop/fs/obs/TestOBSFileContextUtil.java | 9 +-
.../hadoop/fs/obs/TestOBSFileSystemContract.java | 13 +-
hadoop-common-project/hadoop-auth/pom.xml | 5 +
.../client/AuthenticatorTestCase.java | 25 +-
.../client/TestAuthenticatedURL.java | 71 +-
.../client/TestKerberosAuthenticator.java | 106 +--
.../client/TestPseudoAuthenticator.java | 16 +-
.../TestAltKerberosAuthenticationHandler.java | 56 +-
.../server/TestAuthenticationFilter.java | 556 +++++++-------
.../server/TestAuthenticationToken.java | 19 +-
.../TestJWTRedirectAuthenticationHandler.java | 162 +++--
.../server/TestKerberosAuthenticationHandler.java | 83 +--
.../server/TestLdapAuthenticationHandler.java | 95 +--
.../TestMultiSchemeAuthenticationHandler.java | 91 ++-
.../server/TestPseudoAuthenticationHandler.java | 40 +-
.../authentication/util/TestAuthToken.java | 58 +-
.../authentication/util/TestCertificateUtil.java | 10 +-
.../util/TestFileSignerSecretProvider.java | 23 +-
.../authentication/util/TestJaasConfiguration.java | 30 +-
.../authentication/util/TestKerberosName.java | 46 +-
.../authentication/util/TestKerberosUtil.java | 72 +-
.../util/TestRandomSignerSecretProvider.java | 30 +-
.../util/TestRolloverSignerSecretProvider.java | 31 +-
.../security/authentication/util/TestSigner.java | 43 +-
.../util/TestStringSignerSecretProvider.java | 12 +-
.../util/TestZKSignerSecretProvider.java | 137 ++--
.../hadoop-common/HadoopCommon.cmake | 3 +
.../hadoop-common/src/main/bin/hadoop-functions.sh | 1 +
.../org/apache/hadoop/fs/LocalDirAllocator.java | 133 +++-
.../hadoop/fs/statistics/StreamStatisticNames.java | 7 +
.../metrics2/sink/PrometheusMetricsSink.java | 52 +-
.../main/java/org/apache/hadoop/util/Clock.java | 32 -
.../org/apache/hadoop/util/MonotonicClock.java | 45 --
.../java/org/apache/hadoop/util/SystemClock.java | 48 --
.../main/java/org/apache/hadoop/util/UTCClock.java | 40 -
.../src/main/resources/core-default.xml | 1 +
.../apache/hadoop/fs/TestLocalDirAllocator.java | 43 +-
.../rawlocal/TestRawlocalContractAppend.java | 11 +
.../apache/hadoop/fs/ftp/TestFTPFileSystem.java | 8 +-
.../apache/hadoop/fs/shell/TestTextCommand.java | 2 +-
.../viewfs/TestViewFileSystemLocalFileSystem.java | 28 +-
...ViewFileSystemWithAuthorityLocalFileSystem.java | 34 +-
.../hadoop/fs/viewfs/ViewFileSystemBaseTest.java | 807 +++++++++++----------
.../java/org/apache/hadoop/log/TestLogLevel.java | 54 +-
.../security/TestFixKerberosTicketOrder.java | 49 +-
.../hadoop/security/TestRaceWhenRelogin.java | 10 +-
.../hadoop/crypto/key/kms/server/TestKMS.java | 489 +++++++------
.../hadoop/crypto/key/kms/server/TestKMSACLs.java | 71 +-
.../hadoop/crypto/key/kms/server/TestKMSAudit.java | 39 +-
.../kms/server/TestKMSAuthenticationFilter.java | 7 +-
.../crypto/key/kms/server/TestKMSMDCFilter.java | 32 +-
.../crypto/key/kms/server/TestKMSWithZK.java | 11 +-
.../server/TestKeyAuthorizationKeyProvider.java | 109 +--
hadoop-common-project/hadoop-minikdc/pom.xml | 30 +
.../hadoop/minikdc/KerberosSecurityTestcase.java | 8 +-
.../org/apache/hadoop/minikdc/TestMiniKdc.java | 22 +-
hadoop-common-project/hadoop-registry/pom.xml | 26 +
.../hadoop/registry/AbstractRegistryTest.java | 4 +-
.../hadoop/registry/AbstractZKRegistryTest.java | 23 +-
.../apache/hadoop/registry/RegistryTestHelper.java | 41 +-
.../hadoop/registry/cli/TestRegistryCli.java | 10 +-
.../registry/client/binding/TestMarshalling.java | 84 ++-
.../client/binding/TestRegistryOperationUtils.java | 7 +-
.../client/binding/TestRegistryPathUtils.java | 18 +-
.../registry/client/impl/TestCuratorService.java | 34 +-
.../impl/TestFSRegistryOperationsService.java | 116 +--
.../client/impl/TestMicroZookeeperService.java | 20 +-
.../integration/TestYarnPolicySelector.java | 4 +-
.../operations/TestRegistryOperations.java | 46 +-
.../secure/AbstractSecureRegistryTest.java | 38 +-
.../secure/TestRegistrySecurityHelper.java | 32 +-
.../hadoop/registry/secure/TestSecureLogins.java | 23 +-
.../hadoop/registry/secure/TestSecureRegistry.java | 10 +-
.../registry/server/dns/TestRegistryDNS.java | 160 ++--
.../registry/server/dns/TestReverseZoneUtils.java | 27 +-
.../libhdfspp/include/hdfspp/content_summary.h | 1 +
.../main/native/libhdfspp/include/hdfspp/fsinfo.h | 1 +
.../native/libhdfspp/include/hdfspp/statinfo.h | 1 +
.../src/main/native/libhdfspp/include/hdfspp/uri.h | 1 +
hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 30 +
.../org/apache/hadoop/hdfs/nfs/TestMountd.java | 4 +-
.../hdfs/nfs/nfs3/TestClientAccessPrivilege.java | 32 +-
.../hadoop/hdfs/nfs/nfs3/TestDFSClientCache.java | 37 +-
.../hadoop/hdfs/nfs/nfs3/TestExportsTable.java | 41 +-
.../hadoop/hdfs/nfs/nfs3/TestNfs3HttpServer.java | 18 +-
.../apache/hadoop/hdfs/nfs/nfs3/TestNfs3Utils.java | 39 +-
.../hadoop/hdfs/nfs/nfs3/TestOffsetRange.java | 31 +-
.../hadoop/hdfs/nfs/nfs3/TestOpenFileCtxCache.java | 21 +-
.../apache/hadoop/hdfs/nfs/nfs3/TestReaddir.java | 23 +-
.../hadoop/hdfs/nfs/nfs3/TestRpcProgramNfs3.java | 281 +++----
.../hadoop/hdfs/nfs/nfs3/TestViewfsWithNfs3.java | 72 +-
.../apache/hadoop/hdfs/nfs/nfs3/TestWrites.java | 166 ++---
.../router/async/RouterAsyncClientProtocol.java | 2 +
.../TestRouterAsyncRpcWhenNamenodeFailover.java | 81 +++
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 20 +
.../hadoop/hdfs/TestBlockTokenWrappingQOP.java | 44 +-
.../java/org/apache/hadoop/hdfs/TestDFSShell.java | 63 +-
.../hadoop/hdfs/TestDatanodeRegistration.java | 9 +-
.../apache/hadoop/hdfs/TestMultipleNNPortQOP.java | 17 +-
.../hdfs/client/impl/BlockReaderTestUtil.java | 4 +-
.../hdfs/client/impl/TestBlockReaderFactory.java | 109 +--
.../client/impl/TestBlockReaderIoProvider.java | 6 +-
.../hdfs/client/impl/TestBlockReaderLocal.java | 117 +--
.../client/impl/TestBlockReaderLocalLegacy.java | 32 +-
.../client/impl/TestBlockReaderLocalMetrics.java | 26 +-
.../hdfs/client/impl/TestBlockReaderRemote.java | 16 +-
.../client/impl/TestClientBlockVerification.java | 10 +-
.../hadoop/hdfs/net/TestDFSNetworkTopology.java | 27 +-
.../net/TestDFSNetworkTopologyPerformance.java | 18 +-
.../hadoop/hdfs/protocol/TestAnnotations.java | 12 +-
.../hadoop/hdfs/protocol/TestBlockListAsLongs.java | 22 +-
.../hadoop/hdfs/protocol/TestLayoutVersion.java | 68 +-
.../hadoop/hdfs/protocol/TestLocatedBlock.java | 8 +-
.../protocol/datatransfer/TestPacketReceiver.java | 6 +-
.../sasl/SaslDataTransferTestCase.java | 10 +-
.../TestBlackListBasedTrustedChannelResolver.java | 16 +-
.../sasl/TestCustomizedCallbackHandler.java | 12 +-
.../datatransfer/sasl/TestSaslDataTransfer.java | 58 +-
.../TestSaslDataTransferExpiredBlockToken.java | 26 +-
.../hadoop/hdfs/protocolPB/TestPBHelper.java | 103 ++-
.../apache/hadoop/hdfs/qjournal/QJMTestUtil.java | 21 +-
.../hdfs/qjournal/TestMiniJournalCluster.java | 10 +-
.../apache/hadoop/hdfs/qjournal/TestNNWithQJM.java | 26 +-
.../hadoop/hdfs/qjournal/TestSecureNNWithQJM.java | 33 +-
.../hdfs/qjournal/client/TestEpochsAreUnique.java | 11 +-
.../hdfs/qjournal/client/TestIPCLoggerChannel.java | 15 +-
.../hdfs/qjournal/client/TestQJMWithFaults.java | 17 +-
.../hdfs/qjournal/client/TestQuorumCall.java | 21 +-
.../client/TestQuorumJournalManagerUnit.java | 23 +-
.../client/TestSegmentRecoveryComparator.java | 5 +-
.../qjournal/server/TestGetJournalEditServlet.java | 6 +-
.../hadoop/hdfs/qjournal/server/TestJournal.java | 73 +-
.../server/TestJournalNodeHttpServerXFrame.java | 15 +-
.../qjournal/server/TestJournalNodeMXBean.java | 18 +-
.../TestJournalNodeRespectsBindHostKeys.java | 52 +-
.../qjournal/server/TestJournaledEditsCache.java | 33 +-
.../hadoop/hdfs/security/TestDelegationToken.java | 177 ++---
.../security/TestDelegationTokenForProxyUser.java | 30 +-
.../hdfs/security/token/block/TestBlockToken.java | 38 +-
.../token/block/TestUpdateDataNodeCurrentKey.java | 24 +-
.../balancer/TestBalancerWithSaslDataTransfer.java | 2 +-
.../hdfs/server/datanode/TestDataNodeMXBean.java | 86 ++-
.../hdfs/server/namenode/TestSecureNameNode.java | 70 +-
.../hdfs/shortcircuit/TestShortCircuitCache.java | 200 ++---
.../shortcircuit/TestShortCircuitLocalRead.java | 91 +--
.../hadoop-mapreduce-client-app/pom.xml | 2 +-
.../apache/hadoop/mapred/MapTaskAttemptImpl.java | 2 +-
.../hadoop/mapred/ReduceTaskAttemptImpl.java | 2 +-
.../apache/hadoop/mapreduce/v2/app/AppContext.java | 2 +-
.../hadoop/mapreduce/v2/app/MRAppMaster.java | 4 +-
.../v2/app/TaskAttemptFinishingMonitor.java | 2 +-
.../mapreduce/v2/app/TaskHeartbeatHandler.java | 2 +-
.../hadoop/mapreduce/v2/app/job/impl/JobImpl.java | 2 +-
.../mapreduce/v2/app/job/impl/MapTaskImpl.java | 2 +-
.../mapreduce/v2/app/job/impl/ReduceTaskImpl.java | 2 +-
.../mapreduce/v2/app/job/impl/TaskAttemptImpl.java | 2 +-
.../hadoop/mapreduce/v2/app/job/impl/TaskImpl.java | 2 +-
.../mapreduce/v2/app/rm/RMContainerAllocator.java | 2 +-
.../v2/app/speculate/DefaultSpeculator.java | 2 +-
.../mapred/TestTaskAttemptFinishingMonitor.java | 2 +-
.../hadoop/mapred/TestTaskAttemptListenerImpl.java | 2 +-
.../org/apache/hadoop/mapreduce/v2/app/MRApp.java | 4 +-
.../hadoop/mapreduce/v2/app/MockAppContext.java | 2 +-
.../hadoop/mapreduce/v2/app/TestRecovery.java | 4 +-
.../mapreduce/v2/app/TestRuntimeEstimators.java | 4 +-
.../mapreduce/v2/app/TestTaskHeartbeatHandler.java | 4 +-
.../v2/app/commit/TestCommitterEventHandler.java | 4 +-
.../mapreduce/v2/app/job/impl/TestJobImpl.java | 2 +-
.../v2/app/job/impl/TestShuffleProvider.java | 2 +-
.../mapreduce/v2/app/job/impl/TestTaskAttempt.java | 4 +-
.../job/impl/TestTaskAttemptContainerRequest.java | 2 +-
.../mapreduce/v2/app/job/impl/TestTaskImpl.java | 4 +-
.../mapreduce/v2/app/rm/TestRMCommunicator.java | 2 +-
.../v2/app/rm/TestRMContainerAllocator.java | 4 +-
.../main/java/org/apache/hadoop/mapred/Clock.java | 10 +-
.../jobhistory/MapAttemptFinishedEvent.java | 2 +-
.../jobhistory/ReduceAttemptFinishedEvent.java | 2 +-
.../jobhistory/TaskAttemptFinishedEvent.java | 2 +-
.../TaskAttemptUnsuccessfulCompletionEvent.java | 2 +-
.../mapreduce/jobhistory/TaskFailedEvent.java | 2 +-
.../mapreduce/jobhistory/TaskFinishedEvent.java | 2 +-
.../java/org/apache/hadoop/mapred/TestClock.java | 21 +-
.../hadoop/mapreduce/v2/hs/HistoryFileManager.java | 4 +-
.../apache/hadoop/mapreduce/v2/hs/JobHistory.java | 2 +-
.../mapreduce/v2/hs/TestHistoryFileManager.java | 4 +-
.../org/apache/hadoop/mapred/MiniMRCluster.java | 4 +-
.../org/apache/hadoop/mapred/UtilsForTests.java | 5 +-
.../hadoop-mapreduce-client/pom.xml | 2 +-
.../hadoop-mapreduce-examples/pom.xml | 2 +-
hadoop-project/pom.xml | 26 +-
.../fs/aliyun/oss/ITAliyunOSSSignatureV4.java | 15 +-
.../fs/aliyun/oss/TestAliyunCredentials.java | 2 +-
.../aliyun/oss/TestAliyunOSSBlockOutputStream.java | 89 ++-
.../oss/TestAliyunOSSFileSystemContract.java | 61 +-
.../aliyun/oss/TestAliyunOSSFileSystemStore.java | 36 +-
.../fs/aliyun/oss/TestAliyunOSSInputStream.java | 54 +-
.../oss/fileContext/TestOSSFileContextUtil.java | 4 +-
.../apache/hadoop/fs/aliyun/oss/yarn/TestOSS.java | 32 +-
.../apache/hadoop/tools/TestHadoopArchives.java | 45 +-
.../apache/hadoop/fs/s3a/S3ABlockOutputStream.java | 5 +-
.../java/org/apache/hadoop/fs/s3a/Statistic.java | 4 +
.../fs/s3a/auth/ProfileAWSCredentialsProvider.java | 106 +++
.../apache/hadoop/fs/s3a/impl/S3AStoreImpl.java | 16 +-
.../s3a/impl/streams/AnalyticsStreamFactory.java | 16 +
.../s3a/impl/streams/ObjectInputStreamFactory.java | 3 +
.../markdown/tools/hadoop-aws/authentication.md | 23 +
.../ITestS3AAnalyticsAcceleratorStreamReading.java | 3 +
.../hadoop/fs/s3a/ITestS3AConfiguration.java | 43 +-
.../fs/s3a/TestS3AAWSCredentialsProvider.java | 35 +-
.../hadoop/fs/s3a/TestS3ABlockOutputStream.java | 6 +-
.../fs/s3a/impl/streams/TestStreamFactories.java | 6 +
.../hadoop/fs/adl/TestAzureADTokenProvider.java | 41 +-
.../hadoop/fs/adl/TestRelativePathFormation.java | 16 +-
.../hadoop/fs/adl/TestValidateConfiguration.java | 4 +-
.../adl/live/TestAdlDifferentSizeWritesLive.java | 40 +-
.../live/TestAdlInternalCreateNonRecursive.java | 36 +-
.../fs/adl/live/TestAdlSdkConfiguration.java | 23 +-
.../fs/adl/live/TestAdlSupportedCharsetInPath.java | 103 ++-
.../apache/hadoop/fs/adl/live/TestMetadata.java | 48 +-
.../dev-support/testrun-scripts/configsupport.sh | 2 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 13 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 11 +-
.../fs/azurebfs/extensions/SASTokenProvider.java | 6 +-
.../fs/azurebfs/services/AbfsBlobClient.java | 130 ++--
.../hadoop/fs/azurebfs/services/AbfsClient.java | 70 +-
.../hadoop/fs/azurebfs/services/AbfsDfsClient.java | 16 +
.../hadoop/fs/azurebfs/services/AbfsErrors.java | 3 +-
.../hadoop-azure/src/site/markdown/index.md | 17 +-
.../ITestAzureBlobFileSystemDelegationSAS.java | 165 ++++-
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 2 +-
.../ITestAzureBlobFileSystemFileStatus.java | 2 +-
.../ITestAzureBlobFileSystemListStatus.java | 100 +++
.../fs/azurebfs/ITestFileSystemProperties.java | 29 +
.../azurebfs/contract/TestBlobListXmlParser.java | 42 +-
.../fs/azurebfs/services/ITestAbfsClient.java | 81 +++
.../fs/azurebfs/utils/DelegationSASGenerator.java | 14 +-
.../fs/azurebfs/utils/DirectoryStateHelper.java | 2 +-
hadoop-tools/hadoop-compat-bench/pom.xml | 25 +
.../compat/common/TestHdfsCompatDefaultSuites.java | 9 +-
.../fs/compat/common/TestHdfsCompatFsCommand.java | 16 +-
.../common/TestHdfsCompatInterfaceCoverage.java | 12 +-
.../compat/common/TestHdfsCompatShellCommand.java | 20 +-
.../hadoop/contrib/utils/join/TestDataJoin.java | 13 +-
.../main/java/org/apache/hadoop/tools/DistCp.java | 3 +-
.../org/apache/hadoop/tools/TestExternalCall.java | 45 +-
.../apache/hadoop/mapred/tools/TestGetGroups.java | 55 --
.../java/org/apache/hadoop/tools/TestDistCh.java | 21 +-
.../dev-support/findbugs-exclude.xml | 14 +-
hadoop-tools/hadoop-gcp/pom.xml | 503 +++++++++++++
.../src/config/checkstyle-suppressions.xml} | 25 +-
.../java/org/apache/hadoop/fs/gs/Constants.java | 17 +-
.../org/apache/hadoop/fs/gs/CreateOptions.java | 121 +++
.../apache/hadoop/fs/gs/ErrorTypeExtractor.java | 60 ++
.../java/org/apache/hadoop/fs/gs/FileInfo.java | 203 ++++++
.../apache/hadoop/fs/gs/GoogleCloudStorage.java | 261 +++++++
.../gs/GoogleCloudStorageClientWriteChannel.java | 116 +++
.../hadoop/fs/gs/GoogleCloudStorageFileSystem.java | 89 +++
.../hadoop/fs/gs/GoogleCloudStorageItemInfo.java | 423 +++++++++++
.../hadoop/fs/gs/GoogleHadoopFileSystem.java | 505 +++++++++++++
.../fs/gs/GoogleHadoopFileSystemConfiguration.java | 75 ++
.../hadoop/fs/gs/GoogleHadoopOutputStream.java | 124 ++++
.../hadoop/fs/gs/HadoopConfigurationProperty.java | 90 +++
.../org/apache/hadoop/fs/gs/StorageResourceId.java | 328 +++++++++
.../java/org/apache/hadoop/fs/gs/StringPaths.java | 169 +++++
.../java/org/apache/hadoop/fs/gs/UriPaths.java | 113 +++
.../hadoop/fs/gs/VerificationAttributes.java | 68 ++
.../org/apache/hadoop/fs/gs}/package-info.java | 5 +-
.../apache/hadoop/fs/gs/TestStorageResourceId.java | 285 ++++++++
.../org/apache/hadoop/fs/gs/TestStringPaths.java | 164 +++++
.../java/org/apache/hadoop/fs/gs/TestUriPaths.java | 150 ++++
.../org/apache/hadoop/fs/gs}/package-info.java | 4 +-
.../mapred/gridmix/TestGridmixSubmission.java | 3 -
hadoop-tools/hadoop-resourceestimator/pom.xml | 2 +-
.../java/org/apache/hadoop/yarn/sls/AMRunner.java | 2 +-
hadoop-tools/pom.xml | 1 +
.../distributedshell/ApplicationMaster.java | 2 +-
.../hadoop/yarn/service/ServiceScheduler.java | 4 +-
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 2 +-
.../api/impl/TestYarnClientWithReservation.java | 4 +-
.../hadoop/yarn/client/cli/TestSchedConfCLI.java | 4 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 2 +-
.../apache/hadoop/yarn/event/AsyncDispatcher.java | 4 +-
.../apache/hadoop/yarn/event/EventDispatcher.java | 4 +-
.../ifile/LogAggregationIndexedFileController.java | 4 +-
.../yarn/util/AbstractLivelinessMonitor.java | 2 -
.../java/org/apache/hadoop/yarn/util/Clock.java | 4 -
.../apache/hadoop/yarn/util/MonotonicClock.java | 4 -
.../hadoop/yarn/util/ProcfsBasedProcessTree.java | 2 -
.../org/apache/hadoop/yarn/util/SystemClock.java | 4 -
.../java/org/apache/hadoop/yarn/util/UTCClock.java | 5 +-
.../hadoop/yarn/util/WindowsBasedProcessTree.java | 2 -
.../TestLogAggregationIndexedFileController.java | 2 +-
.../apache/hadoop/yarn/util/ControlledClock.java | 2 -
.../yarn/util/TestProcfsBasedProcessTree.java | 2 -
.../yarn/util/TestWindowsBasedProcessTree.java | 1 -
.../pom.xml | 2 +-
.../store/impl/MemoryFederationStateStore.java | 2 +-
.../store/impl/SQLFederationStateStore.java | 4 +-
.../store/impl/ZookeeperFederationStateStore.java | 4 +-
.../store/impl/FederationStateStoreBaseTest.java | 3 +-
.../utils/FederationStateStoreTestUtil.java | 2 +-
.../pom.xml | 2 +-
.../hadoop-yarn-server-nodemanager/pom.xml | 2 +-
.../nodemanager/amrmproxy/AMRMProxyService.java | 4 +-
.../amrmproxy/FederationInterceptor.java | 2 +-
.../containermanager/ContainerManagerImpl.java | 2 +-
.../containermanager/container/ContainerImpl.java | 4 +-
.../container/SlidingWindowRetryPolicy.java | 2 +-
.../linux/resources/AbstractCGroupsHandler.java | 4 +-
.../AbstractCGroupsResourceCalculator.java | 4 +-
.../resources/CGroupElasticMemoryController.java | 4 +-
.../linux/resources/CGroupsV2HandlerImpl.java | 18 +-
.../linux/resources/ResourceHandlerModule.java | 13 +-
.../util/CgroupsLCEResourcesHandler.java | 4 +-
.../hadoop-yarn-server-resourcemanager/pom.xml | 2 +-
.../server/resourcemanager/ClientRMService.java | 4 +-
.../DecommissioningNodesWatcher.java | 2 +-
.../server/resourcemanager/NodesListManager.java | 4 +-
.../OpportunisticContainerAllocatorAMService.java | 8 +
.../resourcemanager/RMActiveServiceContext.java | 4 +-
.../yarn/server/resourcemanager/RMContextImpl.java | 2 +-
.../yarn/server/resourcemanager/RMServerUtils.java | 4 +-
.../federation/FederationClientMethod.java | 2 +-
.../federation/FederationStateStoreService.java | 4 +-
.../ProportionalCapacityPreemptionPolicy.java | 21 +-
.../invariants/ReservationInvariantsChecker.java | 2 +-
.../resourcemanager/recovery/ZKRMStateStore.java | 4 +-
.../reservation/AbstractReservationSystem.java | 4 +-
.../reservation/AbstractSchedulerPlanFollower.java | 2 +-
.../reservation/CapacitySchedulerPlanFollower.java | 2 +-
.../reservation/FairSchedulerPlanFollower.java | 2 +-
.../resourcemanager/reservation/InMemoryPlan.java | 4 +-
.../resourcemanager/reservation/PlanFollower.java | 2 +-
.../reservation/ReservationInputValidator.java | 2 +-
.../planning/SimpleCapacityReplanner.java | 4 +-
.../server/resourcemanager/rmapp/RMAppImpl.java | 4 +-
.../rmapp/attempt/AMLivelinessMonitor.java | 2 +-
.../rmapp/monitor/RMAppLifetimeMonitor.java | 2 +-
.../scheduler/AbstractYarnScheduler.java | 4 +-
.../scheduler/SchedulerApplicationAttempt.java | 2 +-
.../scheduler/activities/ActivitiesManager.java | 2 +-
.../scheduler/capacity/AbstractLeafQueue.java | 2 +-
.../scheduler/capacity/AbstractParentQueue.java | 9 +
.../capacity/AutoCreatedQueueDeletionPolicy.java | 2 +-
.../capacity/CapacitySchedulerContext.java | 2 +-
.../QueueConfigurationAutoRefreshPolicy.java | 4 +-
.../capacity/QueueManagementDynamicEditPolicy.java | 4 +-
.../GuaranteedOrZeroCapacityOverTimePolicy.java | 4 +-
.../distributed/NodeQueueLoadMonitor.java | 10 +-
.../fair/AllocationFileLoaderService.java | 4 +-
.../webapp/dao/AppActivitiesInfo.java | 2 +-
.../hadoop/yarn/server/resourcemanager/MockRM.java | 12 +-
.../resourcemanager/TestClientRMService.java | 4 +-
.../TestReservationSystemWithRMHA.java | 4 +-
.../TestProportionalCapacityPreemptionPolicy.java | 96 ++-
...ionalCapacityPreemptionPolicyMockFramework.java | 2 +-
.../TestCapacitySchedulerPlanFollower.java | 2 +-
.../reservation/TestFairSchedulerPlanFollower.java | 2 +-
.../reservation/TestInMemoryPlan.java | 4 +-
.../reservation/TestReservationInputValidator.java | 2 +-
.../reservation/TestSchedulerPlanFollowerBase.java | 2 +-
.../planning/TestSimpleCapacityReplanner.java | 2 +-
.../capacity/TestCapacitySchedulerMultiNodes.java | 50 +-
.../fair/TestAllocationFileLoaderService.java | 2 +-
.../scheduler/fair/TestFSParentQueue.java | 2 +-
.../fair/TestFairSchedulerPreemption.java | 2 +-
.../scheduler/fair/TestQueueManager.java | 2 +-
.../scheduler/fair/TestQueuePlacementPolicy.java | 2 +-
.../scheduler/policy/MockSchedulableEntity.java | 2 +-
.../webapp/TestRMWebServicesReservation.java | 4 +-
.../webapp/dao/TestFairSchedulerQueueInfo.java | 2 +-
.../clientrm/FederationClientInterceptor.java | 4 +-
.../rmadmin/FederationRMAdminInterceptor.java | 4 +-
.../router/webapp/FederationInterceptorREST.java | 4 +-
.../router/clientrm/BaseRouterClientRMTest.java | 4 +-
.../webapp/MockDefaultRequestInterceptorREST.java | 2 +-
.../webapp/TestFederationInterceptorREST.java | 2 +-
.../pom.xml | 2 +-
.../hadoop-yarn-server-web-proxy/pom.xml | 2 +-
start-build-env.sh | 26 +-
398 files changed, 9754 insertions(+), 4636 deletions(-)
copy dev-support/docker/{Dockerfile => Dockerfile_ubuntu_24} (87%)
delete mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Clock.java
delete mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MonotonicClock.java
delete mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SystemClock.java
delete mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/UTCClock.java
create mode 100644
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/async/TestRouterAsyncRpcWhenNamenodeFailover.java
copy
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/constants/package-info.java
=>
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Clock.java
(83%)
copy
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestPureJavaCrc32C.java
=>
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestClock.java
(73%)
create mode 100644
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/ProfileAWSCredentialsProvider.java
delete mode 100644
hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/mapred/tools/TestGetGroups.java
copy hadoop-tools/{hadoop-datajoin =>
hadoop-gcp}/dev-support/findbugs-exclude.xml (68%)
create mode 100644 hadoop-tools/hadoop-gcp/pom.xml
copy
hadoop-tools/{hadoop-azure/src/test/resources/azure-auth-keys.xml.template =>
hadoop-gcp/src/config/checkstyle-suppressions.xml} (55%)
copy
hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/conf/ArgumentKey.java
=>
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/Constants.java
(73%)
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/CreateOptions.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/ErrorTypeExtractor.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/FileInfo.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageClientWriteChannel.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageItemInfo.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopFileSystem.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopFileSystemConfiguration.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopOutputStream.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/HadoopConfigurationProperty.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/StorageResourceId.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/StringPaths.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/UriPaths.java
create mode 100644
hadoop-tools/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/VerificationAttributes.java
copy hadoop-tools/{hadoop-extras/src/main/java/org/apache/hadoop/tools =>
hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs}/package-info.java (92%)
create mode 100644
hadoop-tools/hadoop-gcp/src/test/java/org/apache/hadoop/fs/gs/TestStorageResourceId.java
create mode 100644
hadoop-tools/hadoop-gcp/src/test/java/org/apache/hadoop/fs/gs/TestStringPaths.java
create mode 100644
hadoop-tools/hadoop-gcp/src/test/java/org/apache/hadoop/fs/gs/TestUriPaths.java
copy hadoop-tools/{hadoop-extras/src/main/java/org/apache/hadoop/tools =>
hadoop-gcp/src/test/java/org/apache/hadoop/fs/gs}/package-info.java (91%)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
