This is an automated email from the ASF dual-hosted git repository.
jhung pushed a change to branch YARN-8200
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
discard a611077 YARN-7033. Add support for NM Recovery of assigned resources
(e.g. GPU's, NUMA, FPGA's) to container. (Devaraj K and Wangda Tan)
discard e255686 YARN-7223. Document GPU isolation feature. Contributed by
Wangda Tan.
discard 8f2f900 YARN-6852. Native code changes to support isolate GPU devices
by using CGroups. (wangda)
discard 91c7ac2 YARN-6620. Add support in NodeManager to isolate GPU devices
by using CGroups. Contributed by Wangda Tan.
discard 8ca8d66 YARN-7137. [YARN-3926] Move newly added APIs to unstable in
YARN-3926 branch. Contributed by Wangda Tan.
discard 00a5b0a YARN-7136. Additional Performance Improvement for Resource
Profile Feature (Contributed by Wangda Tan via Daniel Templeton)
discard 9424124 YARN-7056. [Partial backport] Document Resource Profiles
feature. (Sunil G via wangda)
discard 56eeff3 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check
is duplicated. Contributed by Manikandan R.
discard 9fd1019 YARN-7093. Improve log message in ResourceUtils. (Sunil G via
wangda)
discard 4ff1d15 YARN-7039. Fix javac and javadoc errors in YARN-3926 branch.
(Sunil G via wangda)
discard 1ddebbe YARN-7067. [YARN-3926] Optimize ResourceType information
display in UI. Contributed by Wangda Tan.
discard 7dbf56d YARN-7043. [Partial backport] Cleanup ResourceProfileManager.
(wangda)
discard 4052a7b YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap
takes an unnecessary Map parameter. Contributed by Yu-Tang Lin.
discard 5ef4a86 YARN-6789. Add Client API to get all supported resource types
from RM. (Sunil G via wangda)
discard 0f22be74 YARN-7042. Clean up unit tests after YARN-6610. (Daniel
Templeton via wangda)
discard ca704d3 YARN-7030. [YARN-3926] Performance optimizations in Resource
and ResourceUtils class. Contributed by Wangda Tan.
discard ae51c3f YARN-6610. [YARN-3926]
DominantResourceCalculator#getResourceAsValue dominant param is updated to
handle multiple resources. Contributed by Daniel Templeton.
discard 94b8612 YARN-6892. [YARN-3926] Improve API implementation in
Resources and DominantResourceCalculator class. Contributed by Sunil G.
discard fcd0863 YARN-6994. [YARN-3926] Remove last uses of Long from resource
types code. (Daniel Templeton via Yufei Gu)
discard 9d04a74 YARN-6788. [YARN-3926] Improve performance of resource
profile branch (Contributed by Sunil Govindan via Daniel Templeton)
discard 40499f7 YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup.
Contributed by Yeliang Cang.
discard a63683b YARN-6761. Fix build for YARN-3926 branch. Contributed by
Varun Vasudev.
discard 12c68b7 YARN-6445. [YARN-3926] Performance improvements in resource
profile branch with respect to SLS. Contributed by Varun Vasudev.
discard 37dab13 Addendum to YARN-6232 to change tracking url state store call
to use resource map
discard 0d277ce YARN-6232. Update resource usage and preempted resource
calculations to take into account all resource types. Contributed by Varun
Vasudev.
discard faa64ad YARN-5588. [Partial backport] [YARN-3926] Add support for
resource profiles in distributed shell. Contributed by Varun Vasudev.
discard 8a45ff7 YARN-5587. [Partial backport] Add support for resource
profiles. (vvasudev via asuresh)
discard 6c3b233 YARN-5586. Update the Resources class to consider all
resource types. Contributed by Varun Vasudev.
discard 703bc98 YARN-5242. Update DominantResourceCalculator to consider all
resource types in calculations. Contributed by Varun Vasudev.
discard 11824c5 YARN-4830. Add support for resource types in the nodemanager.
Contributed by Varun Vasudev.
discard e037499 YARN-4829. Add support for binary units in Resource
class.(vvasudev via asuresh)
discard f0b0dc9 YARN-4715. Add support to read resource types from a config
file. Contributed by Varun Vasudev.
discard 0d5ae90 YARN-4172. Extend DominantResourceCalculator to account for
all resources. (Varun Vasudev via wangda)
discard 8d4244f YARN-4081. Add support for multiple resource types in the
Resource class. (Varun Vasudev via wangda)
omit 37e3661 HADOOP-15323. AliyunOSS: Improve copy file performance for
AliyunOSSFileSystemStore. Contributed wujinhu.
omit f804d7d HADOOP-16025. Update the year to 2019.
omit c157ab4 HADOOP-16009. Replace the url of the repository in Apache
Hadoop source code.
omit 10cc02f HDFS-14171. Performance improvement in Tailing EditLog.
Contributed by Kenneth Yang.
omit 15286d9 YARN 9108. Fix FederationIntercepter merge home and secondary
allocate response typo. Contributed by Abhishek Modi.
omit d62bfaf HADOOP-15973. Configuration: Included properties are not
cached if resource is a stream. Contributed by Eric Payne
omit 500b2a0 HADOOP-16008. Fix typo in CommandsManual.md
omit eb8b1ea YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics
omit 5cef413 HADOOP-15950. Failover for LdapGroupsMapping. Contributed by
Lukas Majercak.
omit a29eb49 Revert "HADOOP-15950. Failover for LdapGroupsMapping.
Contributed by Lukas Majercak."
omit b98ffbe HADOOP-15950. Failover for LdapGroupsMapping. Contributed by
Lukas Majercak.
omit 74e4098 YARN-7794. SLSRunner is not loading timeline service jars,
causing failure.
omit 7456da9 YARN-9036. Escape newlines in health report in YARN UI.
Contributed by Keqiu Hu
omit a3470c6 HADOOP-15959. Revert "HADOOP-12751. While using kerberos
Hadoop incorrectly assumes names with '@' to be non-simple"
omit 50e11dc HDFS-13870. WebHDFS: Document ALLOWSNAPSHOT and
DISALLOWSNAPSHOT API doc. Contributed by Siyao Meng.
omit 408f60e HADOOP-15943. AliyunOSS: add missing owner & group attributes
for oss FileStatus. Contributed by wujinhu.
omit d892d82 Update jdiff.stable.api from 2.9.1 to 2.9.2.
omit 6cae964 Add 2.9.2 release notes and changes documents.
omit 311be88 HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed
by wujinhu.
omit c3f0681 HDFS-14056. Fix error messages in HDFS-12716. Contributed by
Ayush Saxena.
omit bc4098e HDFS-13963. NN UI is broken with IE11. Contributed by Ayush
Saxena.
omit c39e1ee HADOOP-15925. The config and log of gpg-agent are removed in
create-release script. Contributed by Dinesh Chitlangia.
omit 4199086 HADOOP-15930. Exclude MD5 checksum files from release
artifact.
omit 3aac324 HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps
in statistics. Contributed by Jinhu Wu.
omit a86b665 HADOOP-15923. create-release script should set max-cache-ttl
as well as default-cache-ttl for gpg-agent.
omit b7ca20e YARN-8933. [AMRMProxy] Fix potential empty fields in
allocation response, move SubClusterTimeout to FederationInterceptor.
Contributed by Botong Huang.
omit ee1cb3b YARN-8233. NPE in CapacityScheduler#tryCommit when handling
allocate/reserve proposal whose allocatedOrReservedContainer is null.
Contributed by Tao Yang.
omit 5e433e5 MAPREDUCE-7156. NullPointerException when reaching max
shuffle connections. Contributed by Peter Bacsko
omit d9b3b58 HDFS-14048. DFSOutputStream close() throws exception on
subsequent call after DataNode restart. Contributed by Erik Krogen.
omit 98075d9 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by
Lukas Majercak.
omit 939827d HADOOP-15900. Update JSch versions in LICENSE.txt.
omit 72ee45a HADOOP-15899. Update AWS Java SDK versions in NOTICE.txt.
omit 731b90d YARN-8905. [Router] Add JvmMetricsInfo and pause monitor.
Contributed by Bilwa S T.
omit 81da8b2 YARN-8893. [AMRMProxy] Fix thread leak in AMRMClientRelayer
and UAM client. Contributed by Botong Huang.
omit 62ec800 YARN-7225. Add queue and partition info to RM audit log.
Contributed by Eric Payne
omit c19baf1 Addendum: Fix potential FSImage corruption. Contributed by
Xiao Chen.
omit f23f339 HADOOP-15866. Renamed
HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility.
Contributed by Wei-Chiu Chuang.
omit a0aa610 HADOOP-15868. AliyunOSS: update document for properties of
multiple part download, multiple part upload and directory copy. Contributed by
Jinhu Wu.
omit f2739f3 HADOOP-15850. CopyCommitter#concatFileChunks should check
that the blocks per chunk is not 0. Contributed by Ted Yu.
omit ca69977 HADOOP-15822. zstd compressor can fail with a small output
buffer. Contributed by Jason Lowe.
omit dbf36db YARN-8826. Fix lingering timeline collector after serviceStop
in TimelineCollectorManager. Contributed by Prabha Manepalli.
omit 56a2728 YARN-7502. Nodemanager restart docs should describe
nodemanager supervised property. Contributed by Suma Shivaprasad.
omit e412d8f HADOOP-15859. ZStandardDecompressor.c mistakes a class for an
instance. Contributed by Jason Lowe
omit e212d7d8 Fix potential FSImage corruption. Contributed by Daryn Sharp.
omit 0ba6210 HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed
by Fei Hui.
omit 437bfa9 HADOOP-15837. DynamoDB table Update can fail S3A FS init.
Contributed by Steve Loughran.
omit 7ed627a YARN-3879 [Storage implementation] Create HDFS backing
storage implementation for ATS reads. Contributed by Abhishek Modi.
omit 91f18a5 YARN-7274. Ability to disable elasticity at leaf queue level.
(Zian Chen via wangda)
omit 5da4963 HADOOP-15679. ShutdownHookManager shutdown time needs to be
configurable & extended.
omit 7a5d27d YARN-6989 Ensure timeline service v2 codebase gets UGI from
HttpServletRequest in a consistent way. Contributed by Abhishek Modi
omit cc1bf7f YARN-8858. CapacityScheduler should respect maximum node
resource when per-queue maximum-allocation is being used. Contributed by Wangda
Tan.
omit d412785 YARN-8720. CapacityScheduler does not enforce max resource
allocation check at queue level. Contributed by Tarun Parimi.
omit b0900ad YARN-7652. Handle AM register requests asynchronously in
FederationInterceptor. Contributed by Botong Huang.
omit fa4a111 HADOOP-15818. Fix deprecated maven-surefire-plugin
configuration in hadoop-kms module
omit c632c6e HDFS-13768. Adding replicas to volume map makes DataNode
start slowly. Contributed by Surendra Singh Lilhore.
omit 5539dd9 HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with
No Active Namenode under nameservice. Contributed by Ayush Saxena.
omit d7e9ea3 HADOOP-15820. ZStandardDecompressor native code sets an
integer field as a long. Contributed by Jason Lowe
omit 7b88a57 HADOOP-15817. Reuse Object Mapper in KMSJSONReader.
Contributed by Jonathan Eagles.
omit 65aaa10 YARN-8840. Add missing cleanupSSLConfig() call for
TestTimelineClient test. Contributed by Aki Tanaka.
omit 1505058 MAPREDUCE-6896. Document wrong spelling in usage of
MapredTestDriver tools. (Contributed by LiXin Ge via Daniel Templeton)
omit 57caab6 YARN-8760. [AMRMProxy] Fix concurrent re-register due to
YarnRM failover in AMRMClientRelayer. Contributed by Botong Huang.
omit d7d0e55 YARN-8774. Memory leak when CapacityScheduler allocates from
reserved container with non-default label. Contributed by Tao Yang.
omit 1b0a110 YARN-8804. resourceLimits may be wrongly calculated when
leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang
omit 3d67080 HDFS-13790. RBF: Move ClientProtocol APIs to its own module
Contributed by Chao Sun.
omit 4d69741 YARN-7974. Allow updating application tracking url after
registration. (Jonathan Hung via asuresh)
omit c617dba HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS.
Contributed by Jinhu Wu.
omit 6937925 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home
sub-cluster heartbeat async. Contributed by Botong Huang.
omit 6056597 YARN-8658. [AMRMProxy] Metrics for AMRMClientRelayer inside
FederationInterceptor. Contributed by Young Chen.
omit 3a6ad9c HADOOP-15726. Create utility to limit frequency of log
statements. Contributed by Erik Krogen.
omit 5b77991 HADOOP-15748. S3 listing inconsistency can raise NPE in
globber. Contributed by Steve Loughran.
omit 0dd6861 HADOOP-15772. Remove the 'Path ... should be specified as a
URI' warnings on startup. Contributed by Ayush Saxena.
omit 3e476dc HDFS-13908. TestDataNodeMultipleRegistrations is flaky.
Contributed by Ayush Saxena.
omit 726aee3 MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark
should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko
omit f5cc9f7 MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with
NullPointerException. Contributed by Oleksandr Shevchenko
omit 88e2ca4 HADOOP-15755. StringUtils#createStartupShutdownMessage throws
NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia
omit 97a13eb MAPREDUCE-3801:
org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator
fails intermittently. Contributed by Jason Lowe
omit f6ab07e HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0.
Contributed by Jinhu Wu.
omit 9bd2616 HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js.
Contributed by yanghuafeng.
omit 1e61f38 MAPREDUCE-7133. History Server task attempts REST API returns
invalid data. Contributed by Oleksandr Shevchenko
omit bbd0987 HDFS-13914. Fix DN UI logs link broken when https is enabled
after HDFS-13902. Contributed by Jianfei Jiang.
omit 9befa5e HDFS-13902. Add JMX, conf and stacks menus to the datanode
page. Contributed by fengchuang.
omit a7b9e7d HDFS-13051. Fix dead lock during async editlog rolling if
edit queue is full. Contributed by Daryn Sharp.
omit 121cefd YARN-8709: CS preemption monitor always fails since one
under-served queue was deleted. Contributed by Tao Yang.
omit e56f183 HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to
support minimum number of volumes to be available. Contributed by Ranith Sardar
omit 6c8fc8f HADOOP-15731. TestDistributedShell fails on Windows.
Contributed by Botong Huang.
omit 1b12925 HDFS-13836. RBF: Handle mount table znode with null value.
Contributed by yanghuafeng.
omit 3eb70d4 MAPREDUCE-7131. Job History Server has race condition where
it moves files from intermediate to finished but thinks file is in
intermediate. Contributed by Anthony Hsu
omit f6e4b13 HADOOP-15657 Registering MutableQuantiles via Metric
annotation. Contributed by Sushil Ks
omit 0468b6e HDFS-13812. Fix the inconsistent default refresh interval on
Caching documentation. Contributed by Hrishikesh Gadre.
omit 2e744bd HDFS-13857. RBF: Choose to enable the default nameservice to
read/write files. Contributed by yanghuafeng.
omit 5902c06 HADOOP-10219. ipc.Client.setupIOstreams() needs to check for
ClientCache.stopClient requested shutdowns. Contributed by Kihwal Lee and Lukas
Majercak.
omit 4e342f6 HADOOP-15547/ WASB: improve listStatus performance.
Contributed by Thomas Marquardt.
omit 21aa7f1 YARN-8051: TestRMEmbeddedElector#testCallbackSynchronization
is flakey. Contributed by Robert Kanter and Jason Lowe.
omit 8137b47 HDFS-13634. RBF: Configurable value in xml for async
connection request queue size. Contributed by CR Hota.
omit 2a8f4ae HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime
should display by JMX with ms unit. Contributed by yanghuafeng.
omit bb0e75f YARN-8697. LocalityMulticastAMRMProxyPolicy should fallback
to random sub-cluster when cannot resolve resource. Contributed by Botong Huang.
omit d4a3be9 HADOOP-15699. Fix some of testContainerManager failures in
Windows. Contributed by Botong Huang.
omit 548a595 YARN-8705. Refactor the UAM heartbeat thread in preparation
for YARN-8696. Contributed by Botong Huang.
omit 361d0e4 HDFS-13831. Make block increment deletion number
configurable. Contributed by Ryan Wu.
omit 64681a3 HDFS-13848. Refactor NameNode failover proxy providers.
Contributed by Konstantin Shvachko.
omit c4e3df2 YARN-8649. NPE in localizer hearbeat processing if a
container is killed while localizing. Contributed by lujie
omit 947b9a2 HADOOP-14314. The OpenSolaris taxonomy link is dead in
InterfaceClassification.md. Contributed by Rui Gao.
omit ac7eef4 HADOOP-15689. Add "*.patch" into .gitignore file of branch-2.
Contributed by Rui Gao.
omit 8e09ff9 HDFS-13821. RBF: Add
dfs.federation.router.mount-table.cache.enable so that users can disable cache.
Contributed by Fei Hui.
omit 8e6807e YARN-8673. [AMRMProxy] More robust responseId resync after an
YarnRM master slave switch. Contributed by Botong Huang.
omit 89da0e9 YARN-8581. [AMRMProxy] Add sub-cluster timeout in
LocalityMulticastAMRMProxyPolicy. Contributed by Botong Huang.
omit 1a53aab YARN-8129. Improve error message for invalid value in fields
attribute. Contributed by Abhishek Modi.
omit 18ebe18 HDFS-13750. RBF: Router ID in RouterRpcClient is always null.
Contributed by Takanobu Asanuma.
omit c68d1d4 YARN-8679. [ATSv2] If HBase cluster is down for long time,
high chances that NM ContainerManager dispatcher get blocked. Contributed by
Wangda Tan.
omit 8056472 HADOOP-15674. Test failure
TestSSLHttpServer.testExcludedCiphers with
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard
Nemeth.
omit e2210a5 YARN-8612. Fix NM Collector Service Port issue in
YarnConfiguration. Contributed by Prabha Manepalli.
omit 04c8b5d HDFS-13758. DatanodeManager should throw exception if it has
BlockRecoveryCommand but the block is not under construction. Contributed by
chencan.
omit a44e53a YARN-8640. Restore previous state in container-executor after
failure. Contributed by Jim Brennan
omit ad5c330 HDFS-13813. Exit NameNode if dangling child inode is detected
when saving FsImage. Contributed by Siyao Meng.
new 1d8b76f YARN-4081. Add support for multiple resource types in the
Resource class. (Varun Vasudev via wangda)
new 74c0a11 YARN-4172. Extend DominantResourceCalculator to account for
all resources. (Varun Vasudev via wangda)
new e4e3d0c YARN-4715. Add support to read resource types from a config
file. Contributed by Varun Vasudev.
new 797cc50 YARN-4829. Add support for binary units in Resource
class.(vvasudev via asuresh)
new 3b4c095 YARN-4830. Add support for resource types in the nodemanager.
Contributed by Varun Vasudev.
new c7a26fc0 YARN-5242. Update DominantResourceCalculator to consider all
resource types in calculations. Contributed by Varun Vasudev.
new d6756cd YARN-5586. Update the Resources class to consider all
resource types. Contributed by Varun Vasudev.
new 7091aa1 YARN-5587. [Partial backport] Add support for resource
profiles. (vvasudev via asuresh)
new ee2c227 YARN-5588. [Partial backport] [YARN-3926] Add support for
resource profiles in distributed shell. Contributed by Varun Vasudev.
new aae2333 YARN-6232. Update resource usage and preempted resource
calculations to take into account all resource types. Contributed by Varun
Vasudev.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a611077)
\
N -- N -- N refs/heads/YARN-8200 (aae2333)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 -
LICENSE.txt | 2 +-
NOTICE.txt | 2 +-
dev-support/bin/create-release | 19 +-
.../src/main/resources/META-INF/LICENSE.txt | 454 +-----
.../src/main/resources/META-INF/NOTICE.txt | 176 +--
.../security/authentication/util/KerberosName.java | 9 +-
.../server/TestKerberosAuthenticationHandler.java | 7 +-
.../authentication/util/TestKerberosName.java | 17 +-
.../java/org/apache/hadoop/conf/Configuration.java | 6 +-
.../hadoop/fs/CommonConfigurationKeysPublic.java | 28 +-
.../main/java/org/apache/hadoop/fs/Globber.java | 13 +-
.../io/compress/zstd/ZStandardCompressor.java | 2 +-
.../io/compress/zstd/ZStandardDecompressor.java | 4 +-
.../hadoop/io/retry/FailoverProxyProvider.java | 15 +-
.../main/java/org/apache/hadoop/ipc/Client.java | 14 -
.../org/apache/hadoop/log/LogThrottlingHelper.java | 358 -----
.../apache/hadoop/metrics2/annotation/Metric.java | 5 -
.../hadoop/metrics2/lib/MutableMetricsFactory.java | 4 -
.../java/org/apache/hadoop/security/KDiag.java | 46 +-
.../apache/hadoop/security/LdapGroupsMapping.java | 163 +-
.../apache/hadoop/util/ShutdownHookManager.java | 169 +-
.../java/org/apache/hadoop/util/StringUtils.java | 2 +-
.../hadoop/util/curator/ZKCuratorManager.java | 10 +-
.../hadoop/io/compress/zstd/ZStandardCompressor.c | 21 +-
.../io/compress/zstd/ZStandardDecompressor.c | 12 +-
.../src/main/resources/core-default.xml | 44 +-
.../src/site/markdown/CommandsManual.md | 2 +-
.../src/site/markdown/GroupsMapping.md | 54 +-
.../src/site/markdown/InterfaceClassification.md | 7 +-
.../hadoop-common/src/site/markdown/SecureMode.md | 6 +
.../site/markdown/release/2.9.2/CHANGES.2.9.2.md | 274 ----
.../markdown/release/2.9.2/RELEASENOTES.2.9.2.md | 31 -
.../org/apache/hadoop/conf/TestConfiguration.java | 99 --
.../org/apache/hadoop/http/TestSSLHttpServer.java | 54 +-
.../zstd/TestZStandardCompressorDecompressor.java | 10 +-
.../test/java/org/apache/hadoop/ipc/TestIPC.java | 45 -
.../apache/hadoop/log/TestLogThrottlingHelper.java | 172 --
.../java/org/apache/hadoop/security/TestKDiag.java | 16 +
.../hadoop/security/TestLdapGroupsMapping.java | 80 +-
.../hadoop/security/TestLdapGroupsMappingBase.java | 76 +-
.../TestLdapGroupsMappingWithFailover.java | 142 --
.../TestLdapGroupsMappingWithOneQuery.java | 16 +-
.../TestLdapGroupsMappingWithPosixGroup.java | 10 +-
.../hadoop/security/TestUserGroupInformation.java | 27 +-
.../hadoop/util/TestShutdownHookManager.java | 328 +---
.../org/apache/hadoop/util/TestStringUtils.java | 9 -
.../hadoop/util/curator/TestZKCuratorManager.java | 23 -
hadoop-common-project/hadoop-kms/pom.xml | 3 +-
.../crypto/key/kms/server/KMSJSONReader.java | 4 +-
.../java/org/apache/hadoop/hdfs/DFSUtilClient.java | 2 +-
.../java/org/apache/hadoop/hdfs/DataStreamer.java | 5 +-
.../ha/AbstractNNFailoverProxyProvider.java | 152 +-
.../ha/ConfiguredFailoverProxyProvider.java | 141 +-
.../namenode/ha/IPFailoverProxyProvider.java | 51 +-
.../org/apache/hadoop/hdfs/util/LongBitFormat.java | 8 -
.../federation/metrics/FederationRPCMetrics.java | 13 +-
.../metrics/FederationRPCPerformanceMonitor.java | 15 +-
.../federation/resolver/MountTableResolver.java | 76 +-
.../federation/router/ConnectionManager.java | 18 +-
.../server/federation/router/RBFConfigKeys.java | 13 -
.../federation/router/RouterClientProtocol.java | 1656 --------------------
.../server/federation/router/RouterRpcClient.java | 18 +-
.../server/federation/router/RouterRpcServer.java | 1267 +++++++++++++--
.../src/main/resources/hdfs-rbf-default.xml | 25 -
.../src/main/webapps/router/federationhealth.html | 3 +-
.../contract/router/web/RouterWebHDFSContract.java | 1 -
.../hdfs/server/federation/MockResolver.java | 3 -
.../resolver/TestMountTableResolver.java | 57 -
.../hdfs/server/federation/router/TestRouter.java | 22 -
.../federation/store/driver/TestStateStoreZK.java | 53 -
.../dev-support/jdiff/Apache_Hadoop_HDFS_2.9.2.xml | 312 ----
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 14 -
.../blockmanagement/BlockManagerSafeMode.java | 7 +-
.../server/blockmanagement/DatanodeManager.java | 7 +-
.../org/apache/hadoop/hdfs/server/common/Util.java | 3 +-
.../hadoop/hdfs/server/datanode/DataNode.java | 11 +-
.../datanode/checker/DatasetVolumeChecker.java | 6 +-
.../datanode/checker/StorageLocationChecker.java | 30 +-
.../datanode/fsdataset/impl/BlockPoolSlice.java | 182 +--
.../datanode/fsdataset/impl/FsDatasetImpl.java | 51 +-
.../datanode/fsdataset/impl/FsDatasetUtil.java | 30 +-
.../datanode/fsdataset/impl/FsVolumeList.java | 5 +-
.../server/datanode/fsdataset/impl/ReplicaMap.java | 24 -
.../hdfs/server/namenode/AclEntryStatusFormat.java | 109 +-
.../hadoop/hdfs/server/namenode/FSDirectory.java | 2 -
.../hdfs/server/namenode/FSEditLogAsync.java | 61 +-
.../hdfs/server/namenode/FSImageFormatPBINode.java | 127 +-
.../server/namenode/FSImageFormatProtobuf.java | 35 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 11 +-
.../hdfs/server/namenode/FSNamesystemLock.java | 42 +-
.../server/namenode/INodeWithAdditionalFields.java | 36 +-
.../hdfs/server/namenode/SerialNumberManager.java | 200 +--
.../hdfs/server/namenode/SerialNumberMap.java | 43 +-
.../hadoop/hdfs/server/namenode/XAttrFormat.java | 95 +-
.../hadoop/hdfs/server/namenode/XAttrStorage.java | 11 +
.../tools/offlineImageViewer/FSImageLoader.java | 18 +-
.../offlineImageViewer/PBImageTextWriter.java | 3 +-
.../tools/offlineImageViewer/PBImageXmlWriter.java | 22 +-
.../hadoop/hdfs/util/PersistentLongFile.java | 2 -
.../hadoop-hdfs/src/main/proto/fsimage.proto | 1 -
.../src/main/resources/hdfs-default.xml | 21 -
.../src/main/webapps/datanode/datanode.html | 5 +-
.../src/main/webapps/hdfs/dfshealth.html | 2 +-
.../src/main/webapps/hdfs/explorer.html | 2 +-
.../src/main/webapps/journal/index.html | 2 +-
.../src/main/webapps/secondary/status.html | 2 +-
.../src/main/webapps/static/dfs-dust.js | 4 +-
.../site/markdown/CentralizedCacheManagement.md | 2 +-
.../hadoop-hdfs/src/site/markdown/WebHDFS.md | 24 -
.../TestClientProtocolForPipelineRecovery.java | 2 -
.../TestDataNodeMultipleRegistrations.java | 4 +-
.../TestDataNodeVolumeFailureToleration.java | 6 +-
.../fsdataset/impl/FsDatasetImplTestUtils.java | 14 +-
.../datanode/fsdataset/impl/TestFsVolumeList.java | 62 -
.../namenode/TestCommitBlockSynchronization.java | 6 +-
.../hdfs/server/namenode/TestEditLogRace.java | 158 +-
.../server/namenode/TestLargeDirectoryDelete.java | 2 +-
.../hdfs/server/namenode/TestSaveNamespace.java | 56 -
.../apache/hadoop/tools/TestHdfsConfigFields.java | 2 -
.../v2/app/speculate/DefaultSpeculator.java | 4 +-
.../v2/app/webapp/dao/TaskAttemptsInfo.java | 8 +-
.../hadoop/mapreduce/v2/app/MRAppBenchmark.java | 20 +-
.../v2/app/webapp/TestAMWebServicesAttempts.java | 3 -
.../src/site/markdown/MapredAppMasterRest.md | 2 +-
.../hadoop/mapreduce/v2/hs/HistoryFileManager.java | 12 +-
.../mapreduce/v2/hs/TestHistoryFileManager.java | 52 -
.../apache/hadoop/mapred/ResourceMgrDelegate.java | 7 -
.../apache/hadoop/mapred/TestClientRedirect.java | 9 -
.../org/apache/hadoop/test/MapredTestDriver.java | 2 +-
.../org/apache/hadoop/mapred/ShuffleHandler.java | 4 +-
hadoop-project-dist/pom.xml | 2 +-
hadoop-project/pom.xml | 4 +-
hadoop-project/src/site/site.xml | 3 +-
hadoop-tools/hadoop-aliyun/pom.xml | 6 -
.../fs/aliyun/oss/AliyunOSSBlockOutputStream.java | 5 +-
.../fs/aliyun/oss/AliyunOSSCopyFileTask.java | 7 +-
.../hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java | 46 +-
.../fs/aliyun/oss/AliyunOSSFileSystemStore.java | 63 +-
.../hadoop/fs/aliyun/oss/AliyunOSSUtils.java | 8 +-
.../aliyun/oss/AssumedRoleCredentialProvider.java | 115 --
.../org/apache/hadoop/fs/aliyun/oss/Constants.java | 24 -
.../java/org/apache/hadoop/fs/aliyun/oss/OSS.java | 43 -
.../apache/hadoop/fs/aliyun/oss/OSSFileStatus.java | 38 -
.../src/site/markdown/tools/hadoop-aliyun/index.md | 105 +-
.../hadoop/fs/aliyun/oss/AliyunOSSTestUtils.java | 19 +-
.../fs/aliyun/oss/TestAliyunCredentials.java | 55 +-
.../aliyun/oss/TestAliyunOSSBlockOutputStream.java | 74 +-
.../oss/TestAliyunOSSFileSystemContract.java | 27 +-
.../aliyun/oss/TestAliyunOSSFileSystemStore.java | 10 +-
.../fs/aliyun/oss/contract/AliyunOSSContract.java | 9 -
.../oss/contract/TestAliyunOSSContractDistCp.java | 1 +
.../aliyun/oss/fileContext/TestOSSFileContext.java | 28 -
.../fileContext/TestOSSFileContextCreateMkdir.java | 40 -
.../TestOSSFileContextMainOperations.java | 71 -
.../fileContext/TestOSSFileContextStatistics.java | 69 -
.../oss/fileContext/TestOSSFileContextURI.java | 51 -
.../oss/fileContext/TestOSSFileContextUtil.java | 40 -
.../apache/hadoop/fs/aliyun/oss/yarn/TestOSS.java | 83 -
.../fs/aliyun/oss/yarn/TestOSSMiniYarnCluster.java | 160 --
.../fs/s3a/s3guard/DynamoDBMetadataStore.java | 5 +-
.../hadoop-azure/dev-support/findbugs-exclude.xml | 10 -
hadoop-tools/hadoop-azure/pom.xml | 12 -
.../fs/azure/AzureNativeFileSystemStore.java | 182 ++-
.../org/apache/hadoop/fs/azure/FileMetadata.java | 77 +-
.../hadoop/fs/azure/NativeAzureFileSystem.java | 376 +++--
.../hadoop/fs/azure/NativeFileSystemStore.java | 15 +-
.../apache/hadoop/fs/azure}/PartialListing.java | 21 +-
.../hadoop/fs/azure/ITestListPerformance.java | 196 ---
.../ITestNativeAzureFileSystemConcurrencyLive.java | 4 +-
.../fs/azure/NativeAzureFileSystemBaseTest.java | 2 +-
.../apache/hadoop/tools/mapred/CopyCommitter.java | 8 +-
.../hadoop/tools/mapred/TestCopyCommitter.java | 4 +
hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh | 3 +-
.../hadoop-yarn/dev-support/findbugs-exclude.xml | 18 -
.../ams/ApplicationMasterServiceProcessor.java | 8 +-
.../hadoop/yarn/api/ApplicationClientProtocol.java | 17 -
.../yarn/api/protocolrecords/AllocateRequest.java | 47 +-
.../GetAllResourceTypeInfoRequest.java | 35 -
.../GetAllResourceTypeInfoResponse.java | 60 -
.../records/ApplicationResourceUsageReport.java | 1 +
.../apache/hadoop/yarn/api/records/Resource.java | 349 ++---
.../yarn/api/records/ResourceInformation.java | 89 +-
.../hadoop/yarn/api/records/ResourceRequest.java | 1 -
.../hadoop/yarn/api/records/ResourceTypeInfo.java | 197 ---
.../yarn/api/records/impl/LightWeightResource.java | 163 --
.../hadoop/yarn/api/records/impl/package-info.java | 26 -
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 48 +-
.../exceptions/YARNFeatureNotEnabledException.java | 45 -
.../hadoop/yarn/util/UnitsConversionUtil.java | 43 +-
.../hadoop/yarn/util/resource/package-info.java | 22 -
.../main/proto/applicationclient_protocol.proto | 1 -
.../src/main/proto/yarn_protos.proto | 6 -
.../src/main/proto/yarn_service_protos.proto | 8 -
.../hadoop/yarn/conf/TestResourceInformation.java | 7 +-
.../yarn/conf/TestYarnConfigurationFields.java | 4 -
.../hadoop/yarn/util/TestUnitsConversionUtil.java | 68 +-
.../yarn/applications/distributedshell/Client.java | 5 +-
.../distributedshell/TestDistributedShell.java | 12 +-
.../apache/hadoop/yarn/client/api/AMRMClient.java | 11 -
.../apache/hadoop/yarn/client/api/YarnClient.java | 16 -
.../yarn/client/api/async/AMRMClientAsync.java | 11 -
.../client/api/async/impl/AMRMClientAsyncImpl.java | 5 -
.../yarn/client/api/impl/AMRMClientImpl.java | 11 -
.../yarn/client/api/impl/YarnClientImpl.java | 13 +-
.../yarn/client/api/impl/TestAMRMClient.java | 77 -
.../ApplicationClientProtocolPBClientImpl.java | 18 -
.../ApplicationClientProtocolPBServiceImpl.java | 21 -
.../impl/pb/AllocateRequestPBImpl.java | 27 +-
.../pb/GetAllResourceTypeInfoRequestPBImpl.java | 71 -
.../pb/GetAllResourceTypeInfoResponsePBImpl.java | 184 ---
.../pb/ApplicationResourceUsageReportPBImpl.java | 1 +
.../yarn/api/records/impl/pb/ProtoUtils.java | 5 +-
.../yarn/api/records/impl/pb/ResourcePBImpl.java | 126 +-
.../records/impl/pb/ResourceTypeInfoPBImpl.java | 152 --
.../apache/hadoop/yarn/client/AMRMClientUtils.java | 55 -
.../util/resource/DominantResourceCalculator.java | 583 +++----
.../hadoop/yarn/util/resource/ResourceUtils.java | 352 ++---
.../hadoop/yarn/util/resource/Resources.java | 250 ++-
.../src/main/resources/yarn-default.xml | 39 -
.../apache/hadoop/yarn/api/TestPBImplRecords.java | 26 +-
.../apache/hadoop/yarn/api/TestResourcePBImpl.java | 61 -
.../yarn/client/api/impl/TestTimelineClient.java | 33 +-
.../yarn/util/resource/TestResourceCalculator.java | 227 +--
.../yarn/util/resource/TestResourceUtils.java | 89 +-
.../hadoop/yarn/util/resource/TestResources.java | 19 +-
.../resource-types/resource-types-error-2.xml | 4 -
.../resource-types/resource-types-error-3.xml | 2 +-
.../yarn/server/AMHeartbeatRequestHandler.java | 238 ---
.../hadoop/yarn/server/AMRMClientRelayer.java | 488 ++----
.../amrmproxy/BroadcastAMRMProxyPolicy.java | 4 +-
.../amrmproxy/FederationAMRMProxyPolicy.java | 8 +-
.../LocalityMulticastAMRMProxyPolicy.java | 147 +-
.../policies/amrmproxy/RejectAMRMProxyPolicy.java | 4 +-
.../federation/utils/FederationRegistryClient.java | 10 +-
.../utils/FederationStateStoreFacade.java | 9 -
.../server/metrics/AMRMClientRelayerMetrics.java | 368 -----
.../hadoop/yarn/server/metrics/package-info.java | 18 -
.../yarn/server/uam/UnmanagedAMPoolManager.java | 73 +-
.../server/uam/UnmanagedApplicationManager.java | 218 ++-
.../yarn/server/MockResourceManagerFacade.java | 108 +-
.../hadoop/yarn/server/TestAMRMClientRelayer.java | 88 +-
.../policies/BaseFederationPoliciesTest.java | 5 +-
.../TestBroadcastAMRMProxyFederationPolicy.java | 10 +-
.../TestLocalityMulticastAMRMProxyPolicy.java | 173 +-
.../amrmproxy/TestRejectAMRMProxyPolicy.java | 5 +-
.../utils/FederationPoliciesTestUtil.java | 7 +-
.../metrics/TestAMRMClientRelayerMetrics.java | 507 ------
.../uam/TestUnmanagedApplicationManager.java | 78 +-
.../src/CMakeLists.txt | 8 +-
.../yarn/server/nodemanager/ContainerExecutor.java | 3 +-
.../hadoop/yarn/server/nodemanager/Context.java | 3 -
.../nodemanager/DefaultContainerExecutor.java | 2 +-
.../server/nodemanager/LinuxContainerExecutor.java | 10 +-
.../yarn/server/nodemanager/NodeManager.java | 92 +-
.../server/nodemanager/NodeStatusUpdaterImpl.java | 46 +-
.../amrmproxy/FederationInterceptor.java | 949 +++++------
.../containermanager/container/Container.java | 7 -
.../containermanager/container/ContainerImpl.java | 13 -
.../container/ResourceMappings.java | 124 --
.../linux/privileged/PrivilegedOperation.java | 1 -
.../linux/resources/ResourceHandlerChain.java | 4 +-
.../linux/resources/ResourceHandlerModule.java | 42 +-
.../linux/resources/gpu/GpuResourceAllocator.java | 242 ---
.../resources/gpu/GpuResourceHandlerImpl.java | 153 --
.../localizer/LocalResourcesTrackerImpl.java | 5 -
.../localizer/ResourceLocalizationService.java | 12 +-
.../resourceplugin/NodeResourceUpdaterPlugin.java | 52 -
.../resourceplugin/ResourcePlugin.java | 83 -
.../resourceplugin/ResourcePluginManager.java | 106 --
.../resourceplugin/gpu/GpuDiscoverer.java | 254 ---
.../gpu/GpuNodeResourceUpdateHandler.java | 66 -
.../resourceplugin/gpu/GpuResourcePlugin.java | 61 -
.../recovery/NMLeveldbStateStoreService.java | 42 -
.../recovery/NMNullStateStoreService.java | 7 -
.../nodemanager/recovery/NMStateStoreService.java | 23 -
.../nodemanager/util/NodeManagerHardwareUtils.java | 36 +-
.../webapp/dao/gpu/GpuDeviceInformation.java | 72 -
.../webapp/dao/gpu/GpuDeviceInformationParser.java | 87 -
.../webapp/dao/gpu/PerGpuDeviceInformation.java | 165 --
.../webapp/dao/gpu/PerGpuMemoryUsage.java | 58 -
.../webapp/dao/gpu/PerGpuTemperature.java | 80 -
.../webapp/dao/gpu/PerGpuUtilizations.java | 50 -
.../container-executor/impl/container-executor.c | 71 +-
.../container-executor/impl/container-executor.h | 2 -
.../src/main/native/container-executor/impl/main.c | 11 -
.../impl/modules/cgroups/cgroups-operations.c | 161 --
.../impl/modules/cgroups/cgroups-operations.h | 55 -
.../impl/modules/gpu/gpu-module.c | 229 ---
.../impl/modules/gpu/gpu-module.h | 45 -
.../test/modules/cgroups/test-cgroups-module.cc | 121 --
.../test/modules/gpu/test-gpu-module.cc | 203 ---
.../test/test-container-executor.c | 1 +
.../server/nodemanager/NodeManagerTestBase.java | 164 --
.../nodemanager/TestDefaultContainerExecutor.java | 4 +-
.../nodemanager/TestLinuxContainerExecutor.java | 2 +-
.../TestLinuxContainerExecutorWithMocks.java | 2 +-
.../yarn/server/nodemanager/TestNodeManager.java | 2 +-
.../server/nodemanager/TestNodeStatusUpdater.java | 100 +-
.../nodemanager/amrmproxy/BaseAMRMProxyTest.java | 46 +-
.../amrmproxy/TestAMRMProxyService.java | 1 +
.../amrmproxy/TestFederationInterceptor.java | 432 ++---
.../amrmproxy/TestableFederationInterceptor.java | 126 +-
.../containermanager/TestContainerManager.java | 18 +-
.../TestContainerManagerRecovery.java | 161 +-
.../linux/resources/TestResourceHandlerModule.java | 8 +-
.../resources/gpu/TestGpuResourceHandler.java | 382 -----
.../localizer/TestResourceLocalizationService.java | 12 +-
.../TestContainersMonitorResourceChange.java | 2 +-
.../resourceplugin/TestResourcePluginManager.java | 261 ---
.../resourceplugin/gpu/TestGpuDiscoverer.java | 123 --
.../recovery/NMMemoryStateStoreService.java | 14 -
.../recovery/TestNMLeveldbStateStoreService.java | 121 +-
.../util/TestNodeManagerHardwareUtils.java | 2 +-
.../server/nodemanager/webapp/MockContainer.java | 6 -
.../dao/gpu/TestGpuDeviceInformationParser.java | 50 -
.../test/resources/nvidia-smi-sample-xml-output | 547 -------
.../server/resourcemanager/AMSProcessingChain.java | 2 +-
.../resourcemanager/ApplicationMasterService.java | 24 +-
.../server/resourcemanager/ClientRMService.java | 23 +-
.../resourcemanager/DefaultAMSProcessor.java | 8 +-
.../OpportunisticContainerAllocatorAMService.java | 3 +-
.../yarn/server/resourcemanager/RMAppManager.java | 5 +-
.../yarn/server/resourcemanager/RMAuditLogger.java | 81 +-
.../ProportionalCapacityPreemptionPolicy.java | 3 -
.../rmapp/attempt/RMAppAttemptImpl.java | 21 -
.../rmapp/attempt/RMAppAttemptMetrics.java | 11 +-
.../event/RMAppAttemptStatusupdateEvent.java | 11 -
.../scheduler/AbstractYarnScheduler.java | 21 +-
.../scheduler/ClusterNodeTracker.java | 11 -
.../resourcemanager/scheduler/ResourceLimits.java | 24 -
.../scheduler/SchedulerApplicationAttempt.java | 9 +-
.../scheduler/capacity/CSQueueMetrics.java | 39 -
.../scheduler/capacity/CSQueueUtils.java | 33 +-
.../scheduler/capacity/CapacityScheduler.java | 84 +-
.../scheduler/capacity/LeafQueue.java | 4 -
.../scheduler/capacity/ParentQueue.java | 22 +-
.../allocator/RegularContainerAllocator.java | 18 +-
.../scheduler/common/fica/FiCaSchedulerApp.java | 19 +-
.../scheduler/fair/FSAppAttempt.java | 5 +-
.../scheduler/fifo/FifoAppAttempt.java | 10 +-
.../webapp/MetricsOverviewTable.java | 11 +-
.../server/resourcemanager/webapp/NodesPage.java | 4 +-
.../server/resourcemanager/webapp/dao/AppInfo.java | 2 +-
.../resourcemanager/webapp/dao/SchedulerInfo.java | 3 +-
.../hadoop/yarn/server/resourcemanager/MockRM.java | 6 +-
.../server/resourcemanager/TestAppManager.java | 4 -
.../TestApplicationMasterService.java | 45 +-
.../resourcemanager/TestClientRMService.java | 48 -
.../server/resourcemanager/TestRMAuditLogger.java | 23 +-
.../resourcemanager/TestRMEmbeddedElector.java | 112 +-
.../yarn/server/resourcemanager/TestRMRestart.java | 45 -
.../applicationsmanager/TestAMLaunchFailure.java | 2 +-
.../TestSchedulerNegotiator.java | 2 +-
...ionalCapacityPreemptionPolicyMockFramework.java | 5 -
...ortionalCapacityPreemptionPolicyIntraQueue.java | 90 +-
.../scheduler/capacity/TestCapacityScheduler.java | 181 ++-
.../TestCapacitySchedulerAsyncScheduling.java | 94 --
.../capacity/TestCapacitySchedulerPerf.java | 265 ----
.../capacity/TestContainerAllocation.java | 132 --
.../capacity/TestNodeLabelContainerAllocation.java | 225 ---
.../scheduler/capacity/TestQueueParsing.java | 168 +-
.../webapp/TestRMWebServicesApps.java | 2 +-
.../apache/hadoop/yarn/server/router/Router.java | 8 -
.../clientrm/DefaultClientRequestInterceptor.java | 8 -
.../clientrm/FederationClientInterceptor.java | 7 -
.../router/clientrm/RouterClientRMService.java | 9 -
.../hadoop/yarn/server/router/TestRouter.java | 38 -
.../PassThroughClientRequestInterceptor.java | 8 -
.../apache/hadoop/yarn/server/MiniYARNCluster.java | 7 +-
.../TestTimelineServiceClientIntegration.java | 3 +-
.../security/TestTimelineAuthFilterForV2.java | 2 +-
.../PerNodeTimelineCollectorsAuxService.java | 13 +-
.../collector/TimelineCollectorManager.java | 2 +-
.../reader/TimelineReaderWebServicesUtils.java | 27 +-
...TimelineReaderWhitelistAuthorizationFilter.java | 2 +-
.../storage/FileSystemTimelineReaderImpl.java | 87 +-
.../src/site/markdown/NodeManager.md | 10 +-
.../src/site/markdown/ResourceProfiles.md | 79 -
.../src/site/markdown/UsingGpus.md | 230 ---
380 files changed, 5209 insertions(+), 20879 deletions(-)
copy LICENSE.txt => hadoop-build-tools/src/main/resources/META-INF/LICENSE.txt
(83%)
copy NOTICE.txt => hadoop-build-tools/src/main/resources/META-INF/NOTICE.txt
(65%)
delete mode 100644
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/log/LogThrottlingHelper.java
delete mode 100644
hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.2/CHANGES.2.9.2.md
delete mode 100644
hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.2/RELEASENOTES.2.9.2.md
delete mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/log/TestLogThrottlingHelper.java
delete mode 100644
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithFailover.java
delete mode 100644
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java
delete mode 100644
hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_2.9.2.xml
delete mode 100644
hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AssumedRoleCredentialProvider.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/OSS.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/OSSFileStatus.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContext.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextCreateMkdir.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextMainOperations.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextStatistics.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextURI.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextUtil.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/yarn/TestOSS.java
delete mode 100644
hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/yarn/TestOSSMiniYarnCluster.java
copy hadoop-tools/{hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native =>
hadoop-azure/src/main/java/org/apache/hadoop/fs/azure}/PartialListing.java (80%)
delete mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestListPerformance.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetAllResourceTypeInfoRequest.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetAllResourceTypeInfoResponse.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceTypeInfo.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/LightWeightResource.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/package-info.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/exceptions/YARNFeatureNotEnabledException.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/package-info.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAllResourceTypeInfoRequestPBImpl.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAllResourceTypeInfoResponsePBImpl.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourceTypeInfoPBImpl.java
rename hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-api =>
hadoop-yarn-common}/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceUtils.java
(59%)
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestResourcePBImpl.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/AMHeartbeatRequestHandler.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/AMRMClientRelayerMetrics.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/package-info.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/metrics/TestAMRMClientRelayerMetrics.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ResourceMappings.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/GpuResourceAllocator.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/GpuResourceHandlerImpl.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/NodeResourceUpdaterPlugin.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/ResourcePlugin.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/ResourcePluginManager.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuDiscoverer.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuNodeResourceUpdateHandler.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuResourcePlugin.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/GpuDeviceInformation.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/GpuDeviceInformationParser.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuDeviceInformation.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuMemoryUsage.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuTemperature.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuUtilizations.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/cgroups/cgroups-operations.c
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/cgroups/cgroups-operations.h
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.c
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.h
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/modules/cgroups/test-cgroups-module.cc
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/modules/gpu/test-gpu-module.cc
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/NodeManagerTestBase.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceHandler.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/TestResourcePluginManager.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/TestGpuDiscoverer.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/TestGpuDeviceInformationParser.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-xml-output
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerPerf.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/TestRouter.java
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceProfiles.md
delete mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/UsingGpus.md
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
