This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch HDDS-2067
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
omit e4d4fca package-info fix + removing FWIO
omit 5b359af remove RpcController
omit adee693 Update
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/OzoneProtocolMessageDispatcher.java
omit 7a44cdc Update
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/OzoneProtocolMessageDispatcher.java
omit 7256c69 HDDS-2067. Create generic service facade with
tracing/metrics/logging support
add aa664d7 HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container
raises NPE (#1302). Contributed by Gabor Bota.
add 2b5fc95 HADOOP-16591 Fix S3A ITest*MRjob failures.
add c30e495 HDFS-14853. NPE in
DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not
present. Contributed by Ranith Sardar.
add 3d78b12 YARN-9762. Add submission context label to audit logs.
Contributed by Manoj Kumar
add 3fd3d74 HDDS-2161. Create RepeatedKeyInfo structure to be saved in
deletedTable
add 6cbe5d3 HDDS-2160. Add acceptance test for ozonesecure-mr compose.
Contributed by Xiaoyu Yao. (#1490)
add 0a716bd HDDS-2159. Fix Race condition in ProfileServlet#pid.
add bfe1dac HADOOP-16560. [YARN] use protobuf-maven-plugin to generate
protobuf classes (#1496)
add e8e7d7b HADOOP-16561. [MAPREDUCE] use protobuf-maven-plugin to
generate protobuf classes (#1500)
add 8f1a135 HDDS-2081. Fix
TestRatisPipelineProvider#testCreatePipelinesDnExclude. Contributed by
Aravindan Vijayan. (#1506)
add 51c64b3 HDFS-13660. DistCp job fails when new data is appended in the
file while the DistCp copy job is running
add 91f50b9 HDDS-2167. Hadoop31-mr acceptance test is failing due to the
shading
add 43203b4 HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by
Jinglun.
add 816d3cb HDFS-14837. Review of Block.java. Contributed by David
Mollitor.
add afa1006 HDFS-14843. Double Synchronization in
BlockReportLeaseManager. Contributed by David Mollitor.
add f16cf87 HDDS-2170. Add Object IDs and Update ID to Volume Object
(#1510)
add eb96a30 HDFS-14655. [SBN Read] Namenode crashes if one of The JN is
down. Contributed by Ayush Saxena.
add 66400c1 HDFS-14808. EC: Improper size values for corrupt ec block in
LOG. Contributed by Ayush Saxena.
add c2731d4 YARN-9730. Support forcing configured partitions to be
exclusive based on app node label
add 6917754 HDDS-2172.Ozone shell should remove description about REST
protocol support. Contributed by Siddharth Wagle.
add a346381 HDDS-2168. TestOzoneManagerDoubleBufferWithOMResponse
sometimes fails with out of memory error (#1509)
add 3f89084 HDFS-14845. Ignore AuthenticationFilterInitializer for
HttpFSServerWebServer and honor hadoop.http.authentication configs.
add bec0864 YARN-9808. Zero length files in container log output haven't
got a header. Contributed by Adam Antal
add c724577 YARN-6715. Fix documentation about NodeHealthScriptRunner.
Contributed by Peter Bacsko
add 5d472d1 HDDS-2067. Create generic service facade with
tracing/metrics/logging support
add e20836e Update
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/OzoneProtocolMessageDispatcher.java
add 89d5e22 Update
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/OzoneProtocolMessageDispatcher.java
add c17eb3e remove RpcController
add ed6ac73 package-info fix + removing FWIO
add 63c730a fix checkstyle issue
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e4d4fca)
\
N -- N -- N refs/heads/HDDS-2067 (63c730a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../java/org/apache/hadoop/fs/shell/Mkdir.java | 11 +-
.../apache/hadoop/util/NodeHealthScriptRunner.java | 1 +
.../src/site/markdown/DeprecatedProperties.md | 4 +
.../hadoop/util/TestNodeHealthScriptRunner.java | 9 +
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 +
.../server/OzoneProtocolMessageDispatcher.java | 1 -
.../apache/hadoop/hdds/server/ProfileServlet.java | 10 +-
.../org/apache/hadoop/hdfs/protocol/Block.java | 141 ++++++++----
.../fs/http/server/HttpFSAuthenticationFilter.java | 16 +-
.../fs/http/server/HttpFSServerWebServer.java | 22 ++
.../src/main/resources/httpfs-default.xml | 20 +-
.../server/federation/router/TestRouterQuota.java | 2 +-
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 5 +-
.../apache/hadoop/hdfs/net/DFSNetworkTopology.java | 3 +
.../hdfs/qjournal/client/IPCLoggerChannel.java | 14 +-
.../hadoop/hdfs/qjournal/client/QuorumCall.java | 18 ++
.../hdfs/qjournal/client/QuorumJournalManager.java | 2 +
.../hdfs/server/blockmanagement/BlockManager.java | 13 +-
.../blockmanagement/BlockReportLeaseManager.java | 8 +-
.../src/main/resources/hdfs-default.xml | 8 +
.../hadoop/hdfs/net/TestDFSNetworkTopology.java | 16 ++
.../hadoop/hdfs/qjournal/MiniJournalCluster.java | 6 +-
.../qjournal/client/TestQuorumJournalManager.java | 34 ++-
.../hadoop-mapreduce-client-common/pom.xml | 34 +--
.../src/main/proto/HSAdminRefreshProtocol.proto | 3 +-
.../src/main/proto/MRClientProtocol.proto | 1 +
.../src/main/proto/mr_protos.proto | 1 +
.../src/main/proto/mr_service_protos.proto | 1 +
.../hadoop-mapreduce-client-shuffle/pom.xml | 27 +--
.../src/main/proto/ShuffleHandlerRecovery.proto | 1 +
.../apache/hadoop/ozone/om/OMMetadataManager.java | 6 +-
.../ozone/om/codec/RepeatedOmKeyInfoCodec.java | 52 +++++
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 100 +++++++--
.../hadoop/ozone/om/helpers/RepeatedOmKeyInfo.java | 91 ++++++++
.../src/main/proto/OzoneManagerProtocol.proto | 14 +-
.../{ozone-mr/hadoop32 => ozonesecure-mr}/test.sh | 12 +-
.../smoketest/{kinit.robot => kinit-hadoop.robot} | 2 +-
hadoop-ozone/dist/src/main/smoketest/kinit.robot | 5 +-
.../dist/src/main/smoketest/mapreduce.robot | 2 +-
.../scm/pipeline/TestRatisPipelineProvider.java | 26 ++-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 47 +++-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 90 ++++----
.../ozone/om/request/key/OMKeyDeleteRequest.java | 3 +-
.../multipart/S3MultipartUploadAbortRequest.java | 5 +-
.../S3MultipartUploadCommitPartRequest.java | 4 +-
.../om/request/volume/OMVolumeCreateRequest.java | 5 +
.../ozone/om/response/key/OMKeyDeleteResponse.java | 27 ++-
.../multipart/S3MultipartUploadAbortResponse.java | 21 +-
.../S3MultipartUploadCommitPartResponse.java | 34 ++-
.../org/apache/hadoop/ozone/web/ozShell/Shell.java | 3 +-
...TestOzoneManagerDoubleBufferWithOMResponse.java | 21 +-
.../ozone/om/request/TestOMRequestUtils.java | 14 +-
.../request/volume/TestOMVolumeCreateRequest.java | 10 +-
.../om/response/key/TestOMKeyDeleteResponse.java | 20 +-
.../s3/multipart/TestS3MultipartResponse.java | 3 +-
.../TestS3MultipartUploadAbortResponse.java | 19 +-
hadoop-ozone/ozonefs/pom.xml | 4 +
.../fs/s3a/commit/AbstractYarnClusterITest.java | 5 +-
.../fs/azurebfs/ITestAzureBlobFileSystemCLI.java | 65 ++++++
.../org/apache/hadoop/tools/DistCpConstants.java | 6 +
.../apache/hadoop/tools/mapred/CopyCommitter.java | 10 +-
.../org/apache/hadoop/tools/mapred/CopyMapper.java | 3 +-
.../tools/mapred/RetriableFileCopyCommand.java | 34 ++-
.../org/apache/hadoop/tools/util/DistCpUtils.java | 26 ++-
.../hadoop/tools/mapred/TestCopyCommitter.java | 5 +-
.../apache/hadoop/tools/mapred/TestCopyMapper.java | 56 +++++
.../tools/mapred/TestRetriableFileCopyCommand.java | 25 ++-
.../apache/hadoop/tools/util/TestDistCpUtils.java | 45 ++--
.../tools/util/TestDistCpUtilsWithCombineMode.java | 4 +-
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 41 +---
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 6 +
.../src/main/proto/YarnCsiAdaptor.proto | 1 +
.../main/proto/application_history_client.proto | 1 +
.../main/proto/applicationclient_protocol.proto | 1 +
.../main/proto/applicationmaster_protocol.proto | 1 +
.../src/main/proto/client_SCM_protocol.proto | 1 +
.../main/proto/containermanagement_protocol.proto | 1 +
.../src/main/proto/server/SCM_Admin_protocol.proto | 1 +
.../proto/server/application_history_server.proto | 1 +
.../resourcemanager_administration_protocol.proto | 1 +
...arn_server_resourcemanager_service_protos.proto | 1 +
.../src/main/proto/yarn_csi_adaptor.proto | 4 +-
.../src/main/proto/yarn_protos.proto | 1 +
.../src/main/proto/yarn_service_protos.proto | 1 +
.../hadoop-yarn-services-core/pom.xml | 22 +-
.../src/main/proto/ClientAMProtocol.proto | 1 +
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 31 +--
.../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 59 ++++-
.../src/test/proto/test_amrm_token.proto | 1 +
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 41 ++--
.../hadoop/yarn/logaggregation/LogToolUtils.java | 60 +++--
.../src/main/proto/yarn_security_token.proto | 1 +
.../src/main/resources/yarn-default.xml | 9 +
.../logaggregation/TestAggregatedLogFormat.java | 42 +++-
.../TestLogAggregationIndexedFileController.java | 31 ++-
.../pom.xml | 39 ++--
.../yarn_server_timelineserver_recovery.proto | 1 +
.../hadoop-yarn-server-common/pom.xml | 39 +---
.../src/main/proto/ResourceTracker.proto | 1 +
.../src/main/proto/SCMUploader.proto | 1 +
.../main/proto/collectornodemanager_protocol.proto | 1 +
.../proto/distributed_scheduling_am_protocol.proto | 2 +-
.../src/main/proto/yarn_server_common_protos.proto | 1 +
.../proto/yarn_server_common_service_protos.proto | 1 +
.../main/proto/yarn_server_federation_protos.proto | 1 +
.../hadoop-yarn-server-nodemanager/pom.xml | 34 +--
.../src/main/proto/LocalizationProtocol.proto | 1 +
.../proto/yarn_server_nodemanager_recovery.proto | 1 +
.../yarn_server_nodemanager_service_protos.proto | 1 +
.../logaggregation/TestLogAggregationService.java | 106 +++++----
.../nodemanager/webapp/TestNMWebServices.java | 43 ++--
.../hadoop-yarn-server-resourcemanager/pom.xml | 69 +++---
.../server/resourcemanager/ClientRMService.java | 6 +-
.../resourcemanager/DefaultAMSProcessor.java | 6 +
.../yarn/server/resourcemanager/RMAppManager.java | 6 +
.../yarn/server/resourcemanager/RMAuditLogger.java | 53 ++++-
.../yarn/server/resourcemanager/RMContext.java | 4 +
.../yarn/server/resourcemanager/RMContextImpl.java | 15 ++
.../scheduler/SchedulerApplicationAttempt.java | 9 +
.../resourcemanager/scheduler/SchedulerUtils.java | 23 ++
.../capacity/CapacitySchedulerConfiguration.java | 7 +
.../scheduler/capacity/LeafQueue.java | 8 +-
.../policy/AbstractComparatorOrderingPolicy.java | 8 +-
.../FifoOrderingPolicyWithExclusivePartitions.java | 144 ++++++++++++
.../IteratorSelector.java} | 32 +--
.../scheduler/policy/OrderingPolicy.java | 3 +-
.../scheduler/policy/SchedulableEntity.java | 5 +
.../yarn_server_resourcemanager_recovery.proto | 1 +
.../server/resourcemanager/TestRMAuditLogger.java | 21 +-
.../scheduler/TestSchedulerUtils.java | 142 ++++++++++++
.../scheduler/capacity/TestCapacityScheduler.java | 11 +-
.../scheduler/capacity/TestLeafQueue.java | 145 +++++++++++-
.../scheduler/policy/MockSchedulableEntity.java | 15 +-
.../scheduler/policy/TestFairOrderingPolicy.java | 12 +-
.../scheduler/policy/TestFifoOrderingPolicy.java | 2 +-
.../TestFifoOrderingPolicyForPendingApps.java | 5 +-
...tFifoOrderingPolicyWithExclusivePartitions.java | 244 +++++++++++++++++++++
.../src/test/proto/test_client_tokens.proto | 1 +
.../hadoop-yarn-server-tests/pom.xml | 31 +--
.../src/test/proto/test_token.proto | 1 +
.../src/site/markdown/NodeManager.md | 8 +-
141 files changed, 2306 insertions(+), 723 deletions(-)
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/RepeatedOmKeyInfoCodec.java
create mode 100644
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/RepeatedOmKeyInfo.java
copy hadoop-ozone/dist/src/main/compose/{ozone-mr/hadoop32 =>
ozonesecure-mr}/test.sh (83%)
copy hadoop-ozone/dist/src/main/smoketest/{kinit.robot => kinit-hadoop.robot}
(94%)
create mode 100644
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCLI.java
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FifoOrderingPolicyWithExclusivePartitions.java
copy
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/{event/ReleaseContainerEvent.java
=> policy/IteratorSelector.java} (62%)
create mode 100644
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFifoOrderingPolicyWithExclusivePartitions.java
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
