RE: [Result] [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby
My late +1. Really it's useful feature.. Great work. -Original Message- From: Konstantin Shvachko [mailto:shv.had...@gmail.com] Sent: Saturday, December 22, 2018 6:48 AM To: Hadoop Common ; hdfs-dev Cc: mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: [Result] [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby Obviously +1 from me. With four binding +1s, two non-binding +1s, and no -1s this vote passes. Thank you folks for working on the feature and for voting. Will do the merge in bit. Thanks, --Konst On Fri, Dec 14, 2018 at 6:16 PM Konstantin Shvachko wrote: > Hi Hadoop developers, > > I would like to propose to merge to trunk the feature branch > HDFS-12943 for Consistent Reads from Standby Node. The feature is > intended to scale read RPC workloads. On large clusters reads comprise > 95% of all RPCs to the NameNode. We should be able to accommodate > higher overall RPC workloads (up to 4x by some estimates) by adding multiple > ObserverNodes. > > The main functionality has been implemented see sub-tasks of HDFS-12943. > We followed up with the test plan. Testing was done on two independent > clusters (see HDFS-14058 and HDFS-14059) with security enabled. > We ran standard HDFS commands, MR jobs, admin commands including > manual failover. > We know of one cluster running this feature in production. > > Since the previous vote we addressed Daryn's concern (see HDFS-13873), > added documentation for the new feature, and fixed a few other jiras. > > I attached a unified patch to the umbrella jira for the review. > Please vote on this thread. The vote will run for 7 days until Wed Dec 21. > > Thanks, > --Konstantin >
[Result] [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby
Obviously +1 from me. With four binding +1s, two non-binding +1s, and no -1s this vote passes. Thank you folks for working on the feature and for voting. Will do the merge in bit. Thanks, --Konst On Fri, Dec 14, 2018 at 6:16 PM Konstantin Shvachko wrote: > Hi Hadoop developers, > > I would like to propose to merge to trunk the feature branch HDFS-12943 > for Consistent Reads from Standby Node. The feature is intended to scale > read RPC workloads. On large clusters reads comprise 95% of all RPCs to the > NameNode. We should be able to accommodate higher overall RPC workloads (up > to 4x by some estimates) by adding multiple ObserverNodes. > > The main functionality has been implemented see sub-tasks of HDFS-12943. > We followed up with the test plan. Testing was done on two independent > clusters (see HDFS-14058 and HDFS-14059) with security enabled. > We ran standard HDFS commands, MR jobs, admin commands including manual > failover. > We know of one cluster running this feature in production. > > Since the previous vote we addressed Daryn's concern (see HDFS-13873), > added documentation for the new feature, and fixed a few other jiras. > > I attached a unified patch to the umbrella jira for the review. > Please vote on this thread. The vote will run for 7 days until Wed Dec 21. > > Thanks, > --Konstantin >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/ [Dec 20, 2018 2:58:00 AM] (inigoiri) YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed [Dec 20, 2018 5:17:32 AM] (aajisaka) HADOOP-15935. [JDK 11] Update maven.plugin-tools.version to 3.6.0. [Dec 20, 2018 5:43:15 AM] (nanda) HDDS-934. Freon run hung on non-functional pipeline. Contributed by [Dec 20, 2018 3:29:48 PM] (jlowe) HADOOP-15973. Configuration: Included properties are not cached if [Dec 20, 2018 5:12:42 PM] (bharat) HDDS-881.009. Encapsulate all client to OM requests into one request [Dec 20, 2018 6:00:05 PM] (stevel) HADOOP-15965. Upgrade to ADLS SDK which has major performance [Dec 20, 2018 6:09:01 PM] (stevel) HADOOP-16015. Add bouncycastle jars to hadoop-aws as test dependencies. [Dec 20, 2018 11:45:49 PM] (eyang) YARN-9131. Updated document usage for dynamic auxiliary service. [Dec 21, 2018 12:21:55 AM] (eyang) YARN-9152. Improved AuxServices REST API output.Contributed -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.security.ssl.TestSSLFactory hadoop.util.TestReadWriteDiskValidator hadoop.util.TestBasicDiskValidator hadoop.util.TestDiskCheckerWithDiskIo hadoop.registry.secure.TestSecureLogins hadoop.hdfs.server.balancer.TestBalancer hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor hadoop.yarn.service.TestYarnNativeServices cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-compile-javac-root.txt [336K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/whitespace-eol.txt [9.3M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/whitespace-tabs.txt [1.1M] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-hdds_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_common.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/994/artifact/out/branch-findbugs-hadoop-ozone_tools.txt [8.0K] javadoc:
[jira] [Created] (HADOOP-16019) ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration
luhuachao created HADOOP-16019: -- Summary: ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration Key: HADOOP-16019 URL: https://issues.apache.org/jira/browse/HADOOP-16019 Project: Hadoop Common Issue Type: Improvement Components: common Affects Versions: 3.1.0 Reporter: luhuachao when the config ZK_DTSM_ZK_KERBEROS_KEYTAB or ZK_DTSM_ZK_KERBEROS_PRINCIPAL are not set, the IllegalArgumentException message cannot be logged. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
