Re: About 2.6.5 and 2.7.4 release
> Chiwan Park reported Hadoop incompatibility by the change of HADOOP-11252. > Now, HADOOP-13579 is committed to branch-2.6 and branch-2.7. complement: HADOOP-13579 fixes the problem of incompatibility by HADOOP-11252. Thanks, - Tsuyoshi On Fri, Sep 9, 2016 at 2:42 PM, Tsuyoshi Ozawa wrote: > Hi developers, > > Chiwan Park reported Hadoop incompatibility by the change of HADOOP-11252. > Now, HADOOP-13579 is committed to branch-2.6 and branch-2.7. > > Should we release 2.6.5 and 2.7.4 soon? > > Thanks, > - Tsuyoshi - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
About 2.6.5 and 2.7.4 release
Hi developers, Chiwan Park reported Hadoop incompatibility by the change of HADOOP-11252. Now, HADOOP-13579 is committed to branch-2.6 and branch-2.7. Should we release 2.6.5 and 2.7.4 soon? Thanks, - Tsuyoshi - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5629) Persist collector discovery information to support RM HA
Li Lu created YARN-5629: --- Summary: Persist collector discovery information to support RM HA Key: YARN-5629 URL: https://issues.apache.org/jira/browse/YARN-5629 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Li Lu Assignee: Li Lu As discussed in YARN-3039, we deliberately delayed the work to persist collector discovery information. However, this feature becomes a blocker if we want to run timeline v2 on a HAed cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5628) Remove package line length checkstyle rule
Shane Kumpf created YARN-5628:
-
Summary: Remove package line length checkstyle rule
Key: YARN-5628
URL: https://issues.apache.org/jira/browse/YARN-5628
Project: Hadoop YARN
Issue Type: Sub-task
Components: yarn
Reporter: Shane Kumpf
Assignee: Shane Kumpf
Priority: Trivial
The packages related to the DockerLinuxContainerRuntime all exceed the 80 char
line length limit enforced by checkstyle. This causes every build to fail with
a -1. I would like to exclude this rule from causing a failure.
{code}
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerCommandExecutor.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker;:
Line is longer than 80 characters (found 88).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerContainerStatusHandler.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker;:
Line is longer than 80 characters (found 88).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/package-info.java:23:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker;:
Line is longer than 80 characters (found 88).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/MockPrivilegedOperationCaptor.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged;:
Line is longer than 80 characters (found 84).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerRuntimeTestingUtils.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime;:
Line is longer than 80 characters (found 81).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/MockDockerContainerStatusHandler.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker;:
Line is longer than 80 characters (found 88).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/TestDockerCommandExecutor.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker;:
Line is longer than 80 characters (found 88).
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/TestDockerContainerStatusHandler.java:17:package
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.docker;:
Line is longer than 80 characters (found 88).
{code}
Alternatively, we could look to restructure the packages here, but I question
what value this check really provides.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Merge HADOOP-13341
> On Sep 8, 2016, at 2:50 AM, Steve Loughran wrote: > > I'm trying to do the review effort here even though I don't know detailed > bash, as I expect I don't know any less than others, and what better way to > learn than reviewing code written by people that do know bash? Just a heads up that I'm using bash variable references. While not exactly rare, they are uncommon. [We use them in lots of places in the shell code already, so no new ground being broken.] > Could you submit a PR of that HADOOP-13341 branch, so I can review it there. Sure. https://github.com/apache/hadoop/pull/126 has been opened. Thanks! - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/ [Sep 7, 2016 11:25:23 AM] (stevel) HADOOP-13541 explicitly declare the Joda time version S3A depends on. [Sep 7, 2016 6:09:41 PM] (aengineer) HADOOP-13388. Clean up TestLocalFileSystemPermission. Contributed by [Sep 7, 2016 9:50:17 PM] (kai.zheng) Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for [Sep 8, 2016 6:13:43 AM] (aajisaka) HDFS-10778. Add -format option to make the output of FileDistribution -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_test_libhdfs_zerocopy_hdfs_static Failed junit tests : hadoop.contrib.bkjournal.TestBootstrapStandbyWithBKJM hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.contrib.bkjournal.TestBootstrapStandbyWithBKJM Timed out junit tests : org.apache.hadoop.http.TestHttpServerLifecycle cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-compile-javac-root.txt [168K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-patch-pylint.txt [16K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] CTEST: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt [24K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [120K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [188K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [268K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [124K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [8.0K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/158/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Merge HADOOP-13341
> On 7 Sep 2016, at 14:44, Allen Wittenauer wrote: > > > I’d like to call for a vote to run for 5 days (ending Mon 12, 2016 at > 7AM PT) to merge the HADOOP-13341 feature branch into trunk. This branch was > developed exclusively by me. As usual with large shell script changes, it's > been broken up into several smaller commits to make it easier to read. The > core of the functionality is almost entirely in hadoop-functions.sh with the > majority of the rest of the new additions either being documentation or test > code. In addition, large swaths of code is removed from the hadoop, hdfs, > mapred, and yarn executables. > > Here's a quick summary: > > * makes the rules around _OPTS consistent across all the projects > * makes it possible to provide custom _OPTS for every hadoop, hdfs, mapred, > and yarn subcommand > * with the exception of deprecations, removes all of the custom daemon _OPTS > handling sprinkled around the hadoop, hdfs, mapred, and yarn subcommands > * removes the custom handling handling of HADOOP_CLIENT_OPTS and makes it > consistent for non-daemon subcommands > * makes the _USER blocker consistent with _OPTS as well as providing better > documentation around this feature's existence. Note that this is an > incompatible change against -alpha1. > * by consolidating all of this code, makes it possible to finally fix a good > chunk of the "directory name containing spaces blows up the bash code" > problems that's been around since the beginning of the project > > Thanks! > I'm trying to do the review effort here even though I don't know detailed bash, as I expect I don't know any less than others, and what better way to learn than reviewing code written by people that do know bash? I can see the branch at https://github.com/apache/hadoop/tree/HADOOP-13341 And the patch at https://github.com/apache/hadoop/blob/HADOOP-13341/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh But...the latter only does the _USER stuff, I don't see it ouching the _OPTS stuff. Could you submit a PR of that HADOOP-13341 branch, so I can review it there. Like you said, one thing that will change is that env var going from HADOOP_namenode_USER to HDFS_NAMENODE_USER . Like you say, incompatible with alpha-1 welll, we've a track record of breaking compatibility from alphas, so I'm OK with that, -Steve - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-5627) [Atsv2] Support streaming reader API to fetch entities
Rohith Sharma K S created YARN-5627: --- Summary: [Atsv2] Support streaming reader API to fetch entities Key: YARN-5627 URL: https://issues.apache.org/jira/browse/YARN-5627 Project: Hadoop YARN Issue Type: Sub-task Components: timelinereader Reporter: Rohith Sharma K S Assignee: Rohith Sharma K S There is no limit for size of TimelineEntitie object. It can be varied from KB's to MB. While reading entities list, it would be an potential issue that TimeLineReder would go into OOM situation based on the entity size and limit. Proposal is to support an streaming API to read entities list. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
