Re: [E] [VOTE] Release Apache Hadoop 2.10.1 (RC0)
Thanks for your work on this Masatake! I am +1 (non-binding) on this 2.10.1 release. I built from source and ran hdfs, resourcemanager, and nodemanager unit tests. I set up a one-node-cluster and ran some example jobs (pi, sleep). I tested NM/RM recovery by killing NM/RM during jobs and verifying they completed successfully after restarting them. On Mon, Sep 14, 2020 at 12:59 PM Masatake Iwasaki < iwasak...@oss.nttdata.co.jp> wrote: > Hi folks, > > This is the first release candidate for the second release of Apache > Hadoop 2.10. > It contains 218 fixes/improvements since 2.10.0 [1]. > > The RC0 artifacts are at: > > https://urldefense.com/v3/__http://home.apache.org/*iwasakims/hadoop-2.10.1-RC0/__;fg!!Op6eflyXZCqGR5I!SffKK-HwJYiYKe2rYhl5ajTjL20w4JKLTHjt3MeLb7WjTV7hbhiFFlECw5DPzXq4v4K_eE4$ > > RC tag is release-2.10.1-RC0: > > https://urldefense.com/v3/__https://github.com/apache/hadoop/tree/release-2.10.1-RC0__;!!Op6eflyXZCqGR5I!SffKK-HwJYiYKe2rYhl5ajTjL20w4JKLTHjt3MeLb7WjTV7hbhiFFlECw5DPzXq4x2Dtalw$ > > The maven artifacts are hosted here: > > https://urldefense.com/v3/__https://repository.apache.org/content/repositories/orgapachehadoop-1279/__;!!Op6eflyXZCqGR5I!SffKK-HwJYiYKe2rYhl5ajTjL20w4JKLTHjt3MeLb7WjTV7hbhiFFlECw5DPzXq4i6I2aus$ > > My public key is available here: > > https://urldefense.com/v3/__https://dist.apache.org/repos/dist/release/hadoop/common/KEYS__;!!Op6eflyXZCqGR5I!SffKK-HwJYiYKe2rYhl5ajTjL20w4JKLTHjt3MeLb7WjTV7hbhiFFlECw5DPzXq4cgWpFYE$ > > The vote will run for 5 days, until Saturday, September 19 at 10:00 am PDT. > > [1] > https://urldefense.com/v3/__https://issues.apache.org/jira/issues/?jql=project*20in*20(HDFS*2C*20YARN*2C*20HADOOP*2C*20MAPREDUCE)*20AND*20resolution*20*3D*20Fixed*20AND*20fixVersion*20*3D*202.10.1__;JSUlJSUlJSUlJSUlJSUlJSUl!!Op6eflyXZCqGR5I!SffKK-HwJYiYKe2rYhl5ajTjL20w4JKLTHjt3MeLb7WjTV7hbhiFFlECw5DPzXq4Q9b1yE4$ > > Thanks, > Masatake Iwasaki > > - > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/ [Sep 15, 2020 2:12:00 AM] (noreply) HADOOP-17262. Switch to Yetus main branch. (#2304) [Sep 15, 2020 4:05:04 AM] (Xiaoqiao He) HDFS-15559. Complement initialize member variables in TestHdfsConfigFields#initializeMemberVariables. Contributed by Lisheng Sun. [Sep 15, 2020 4:40:21 AM] (Hemanth Boyina) HDFS-15574. Remove unnecessary sort of block list in DirectoryScanner. Contributed by Stephen O'Donnell. [Sep 15, 2020 4:36:46 PM] (Takanobu Asanuma) HDFS-15576. Erasure Coding: Add rs and rs-legacy codec test for addPolicies. Contributed by Fei Hui. -1 overall The following subsystems voted -1: asflicense pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.hdfs.TestFileChecksum hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes hadoop.hdfs.server.datanode.TestDataNodeLifeline hadoop.hdfs.TestFileChecksumCompositeCrc hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination hadoop.yarn.server.federation.policies.amrmproxy.TestLocalityMulticastAMRMProxyPolicy hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption hadoop.yarn.applications.distributedshell.TestDistributedShell cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-compile-cc-root.txt [48K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-compile-javac-root.txt [568K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-patch-shelldocs.txt [44K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/whitespace-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/whitespace-tabs.txt [1.9M] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/diff-javadoc-javadoc-root.txt [1.3M] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [540K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [472K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt [36K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [104K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/267/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt [16K]
Re: Sep. Y2020 Hadoop Meetup in China
This is great news! Can you also share this with the user@ and user-zh@ mailing lists? We need to get the message out. On Tue, Sep 15, 2020 at 10:36 PM 俊平堵 wrote: > Hi, > Last year Aug, we had a very successful meetup in Beijing ( > https://blogs.apache.org/hadoop) which help to promote hadoop user > adoption > and development locally. This year, we would like to do it again in > Shanghai on 9/26. > So if you have topic to share on hadoop development or some relevant > interesting case. Please nominate it by replying this letter or contact me > directly. Slot is limited, first come first serve. Thanks! > BTW, this meetup is plan to be offline. If many people want to share > but cannot come to Shanghai by then, we can plan something online later. > > Thanks, > > Junping >
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/ No changes -1 overall The following subsystems voted -1: asflicense hadolint jshint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml Failed junit tests : hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.namenode.ha.TestBootstrapStandby hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.resourceestimator.solver.impl.TestLpSolver hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.yarn.server.resourcemanager.TestClientRMService jshint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-patch-jshint.txt [208K] cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-compile-javac-root.txt [436K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-patch-shellcheck.txt [56K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/whitespace-tabs.txt [1.3M] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/xml.txt [4.0K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/diff-javadoc-javadoc-root.txt [20K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [276K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [36K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [12K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [96K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [104K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/58/artifact/out/patch-unit-hadoop-tools_hadoop-resourceestimator.txt [16K] https://ci-hadoo
Re: Sep. Y2020 Hadoop Meetup in China
hi This is good news. The hadoop meetup held in Beijing last year was very successful and attracted many participants. Looking forward to it very much !!! On Wed, Sep 16, 2020 at 1:36 PM 俊平堵 wrote: > Hi, > Last year Aug, we had a very successful meetup in Beijing ( > https://blogs.apache.org/hadoop) which help to promote hadoop user > adoption > and development locally. This year, we would like to do it again in > Shanghai on 9/26. > So if you have topic to share on hadoop development or some relevant > interesting case. Please nominate it by replying this letter or contact me > directly. Slot is limited, first come first serve. Thanks! > BTW, this meetup is plan to be offline. If many people want to share > but cannot come to Shanghai by then, we can plan something online later. > > Thanks, > > Junping >