How to run distributed-shell with node label expression??

Hi All, I am using Hadoop 2.7.1. I have 4 vms(dev, sut-1, sut-2, sut3), but I cannot submit my app to YARN. very appreciate for any help:) what I did is: Add this to yarn-site.xml file: yarn.node-labels.enabled true yarn.node-labels.fs-store.root-dir file:///tmp/ Add this

Build failed in Jenkins: Hadoop-Yarn-trunk #1526

See Changes: [shv] Update CHANGES.txt to move HDFS-9516 to 2.7.3 section. -- [...truncated 10313 lines...] Generating

Hadoop-Yarn-trunk - Build # 1526 - Still Failing

See https://builds.apache.org/job/Hadoop-Yarn-trunk/1526/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 10506 lines...] Building index for all classes...

does yarn 2.7.1 really support label expr like && or || ?

Hi All, Well, my distributedshell works very well with "--shell_command ls -node_label_expression aaa" and I added a new label named "bbb" and tried this: bin/hadoop jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.1.jar

[RESULT] [VOTE] Release Apache Hadoop 2.6.3 RC0

Hi all in community, I give my binding +1 to close the vote for 2.6.3 RC0. With 18 +1s (10 binding), one +0 (non-binding) and no -1s, the vote passes. Thanks for everyone who tried the release candidate and voted. Thanks Vinod, Sangjin and Steve to provide me advices on release process.

Re:How to run distributed-shell with node label expression??

Response to myself: well, this JIRA really helped me: YARN-2502 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java what surprise me is that it set

Re: label based scheduler for fair scheduler?

any JIRAs related to this? Thanks. At 2015-12-17 05:34:53, "Karthik Kambatla" wrote: >I am actively designing the node labels feature for FairScheduler. Keeping >in mind the long-term debuggability/supportability of the scheduler, I >believe it needs a little bit of

Build failed in Jenkins: Hadoop-Yarn-trunk-Java8 #806

See Changes: [kihwal] HDFS-9569. Log the name of the fsimage being loaded for better [aw] HDFS-9570. Minor typos, grammar, and case sensitivity cleanup in -- [...truncated 4418

Hadoop-Yarn-trunk-Java8 - Build # 806 - Failure

See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/806/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 4611 lines...] Tests run: 19, Failures: 0, Errors:

Hadoop-Yarn-trunk - Build # 1527 - Still Failing

See https://builds.apache.org/job/Hadoop-Yarn-trunk/1527/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 9374 lines...] Running

Build failed in Jenkins: Hadoop-Yarn-trunk #1527

See Changes: [kihwal] HDFS-9569. Log the name of the fsimage being loaded for better [aw] HDFS-9570. Minor typos, grammar, and case sensitivity cleanup in -- [...truncated 9181 lines...]

Re: [Important] Lots of 2.8.0 commits are in branch-2 only

Tx for your comments. We decided to cut the branch mid last month with an intention of locking down feature set and have realistic progress towards getting out an RC. We had elaborate discussions on the feature set for 2.8.0 last couple of months. As much as is possible, I’ll avoid resetting

Re: [Important] Lots of 2.8.0 commits are in branch-2 only

In fact, 3 out of those 4 bugs are part of a bigger umbrella refactor effort. So I don’t see any problem as of now. +Vinod > On Dec 17, 2015, at 11:29 AM, Vinod Kumar Vavilapalli > wrote: > > Wangda, > > I see just 13 tickets *only in 2.9.0* >

[jira] [Created] (YARN-4468) Document the general ReservationSystem functionality, and the REST API

Carlo Curino created YARN-4468: -- Summary: Document the general ReservationSystem functionality, and the REST API Key: YARN-4468 URL: https://issues.apache.org/jira/browse/YARN-4468 Project: Hadoop YARN

Jenkins build is back to normal : Hadoop-Yarn-trunk #1528

See

Build failed in Jenkins: Hadoop-Yarn-trunk-Java8 #807

See Changes: [arp] HDFS-9565. [cnauroth] HDFS-9552. Document types of permission checks performed for HDFS [cnauroth] HDFS-9570. Update CHANGES.txt to list under 2.8.0. [vinodkv] YARN-1856. Added cgroups based memory

Hadoop-Yarn-trunk-Java8 - Build # 807 - Still Failing

See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/807/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5363 lines...] Tests run: 4, Failures: 0, Errors:

Hadoop-Yarn-trunk-Java8 - Build # 811 - Still Failing

See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/811/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5405 lines...] Results : Failed tests:

Build failed in Jenkins: Hadoop-Yarn-trunk-Java8 #811

See Changes: [aajisaka] HADOOP-12588. Fix intermittent test failure of TestGangliaMetrics. -- [...truncated 5212 lines...] Running

Hadoop-Yarn-trunk - Build # 1532 - Still Failing

See https://builds.apache.org/job/Hadoop-Yarn-trunk/1532/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 10784 lines...] at

Build failed in Jenkins: Hadoop-Yarn-trunk #1532

See Changes: [aajisaka] HADOOP-12588. Fix intermittent test failure of TestGangliaMetrics. -- [...truncated 10591 lines...] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:

RE: label based scheduler for fair scheduler?

FYI ... YARN-2497 From: Allen Zhang [allenzhang...@126.com] Sent: Thursday, December 17, 2015 20:09 To: yarn-dev@hadoop.apache.org Subject: Re: label based scheduler for fair scheduler? any JIRAs related to this? Thanks. At 2015-12-17 05:34:53,

[jira] [Created] (YARN-4479) Retrospect app-priority in pendingOrderingPolicy during recovering applications

Rohith Sharma K S created YARN-4479: --- Summary: Retrospect app-priority in pendingOrderingPolicy during recovering applications Key: YARN-4479 URL: https://issues.apache.org/jira/browse/YARN-4479

Build failed in Jenkins: Hadoop-Yarn-trunk-Java8 #812

See Changes: [aajisaka] HADOOP-12657. Add a option to skip newline on empty files with getMerge -- [...truncated 5255 lines...] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time

Hadoop-Yarn-trunk-Java8 - Build # 812 - Still Failing

See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/812/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5448 lines...] Tests run: 1, Failures: 0, Errors:

RE: Re:How to run distributed-shell with node label expression??

Hi Allen, I am planning to capture them as part of YARN-4415. (either update the documentation or change in the behavior to consider max capacity as 100) Basically behavior of capacity configuration for queue in default partition is capacity =0 and max capacity =100 But for labels current

Jenkins build is back to normal : Hadoop-Yarn-trunk-Java8 #808

See

[jira] [Created] (YARN-4469) yarn application -status should not show a stack trace for an unknown application ID

Daniel Templeton created YARN-4469: -- Summary: yarn application -status should not show a stack trace for an unknown application ID Key: YARN-4469 URL: https://issues.apache.org/jira/browse/YARN-4469

Build failed in Jenkins: Hadoop-Yarn-trunk #1531

See Changes: [cnauroth] HDFS-9571. Fix ASF Licence warnings in Jenkins reports. Contributed by -- [...truncated 10593 lines...] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:

Hadoop-Yarn-trunk - Build # 1531 - Failure

See https://builds.apache.org/job/Hadoop-Yarn-trunk/1531/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 10786 lines...] at

Hadoop-Yarn-trunk-Java8 - Build # 810 - Failure

See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/810/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5352 lines...] Tests run: 1, Failures: 0, Errors:

Build failed in Jenkins: Hadoop-Yarn-trunk-Java8 #810

See Changes: [cnauroth] HDFS-9571. Fix ASF Licence warnings in Jenkins reports. Contributed by -- [...truncated 5159 lines...] Running

[jira] [Created] (YARN-4476) Matcher for complex node label expresions

Chris Douglas created YARN-4476: --- Summary: Matcher for complex node label expresions Key: YARN-4476 URL: https://issues.apache.org/jira/browse/YARN-4476 Project: Hadoop YARN Issue Type:

Re: [Important] Lots of 2.8.0 commits are in branch-2 only

Wangda, I see just 13 tickets *only in 2.9.0* (https://issues.apache.org/jira/issues/?jql=project%20in%20%28YARN%2C%20MAPREDUCE%2C%20HDFS%2C%20HADOOP%29%20AND%20resolution%20%3D%20Fixed%20and%20fixVersion%20%3D%202.9.0%20ORDER%20BY%20priority%20DESC). If you leave the new features /

[jira] [Created] (YARN-4470) Application Master in-place upgrade

Giovanni Matteo Fumarola created YARN-4470: -- Summary: Application Master in-place upgrade Key: YARN-4470 URL: https://issues.apache.org/jira/browse/YARN-4470 Project: Hadoop YARN

[jira] [Created] (YARN-4471) Add a public API to request an application upgrade

Giovanni Matteo Fumarola created YARN-4471: -- Summary: Add a public API to request an application upgrade Key: YARN-4471 URL: https://issues.apache.org/jira/browse/YARN-4471 Project: Hadoop

[jira] [Created] (YARN-4472) Introduce additional states in the app and app attempt state machines to keep track of the upgrade process

Giovanni Matteo Fumarola created YARN-4472: -- Summary: Introduce additional states in the app and app attempt state machines to keep track of the upgrade process Key: YARN-4472 URL:

[jira] [Created] (YARN-4473) Add version information for the application and the application attempts

Giovanni Matteo Fumarola created YARN-4473: -- Summary: Add version information for the application and the application attempts Key: YARN-4473 URL: https://issues.apache.org/jira/browse/YARN-4473

[jira] [Created] (YARN-4475) Add the possibility to recover the state of an ongoing upgrade in case of RM restart

Giovanni Matteo Fumarola created YARN-4475: -- Summary: Add the possibility to recover the state of an ongoing upgrade in case of RM restart Key: YARN-4475 URL:

[jira] [Created] (YARN-4474) Implementation of the application master upgrade logic

Giovanni Matteo Fumarola created YARN-4474: -- Summary: Implementation of the application master upgrade logic Key: YARN-4474 URL: https://issues.apache.org/jira/browse/YARN-4474 Project:

Re: [RESULT] [VOTE] Release Apache Hadoop 2.6.3 RC0

My late +1 (binding) - Downloaded source tarball - Verified signatures and checksums - Compiled and built a single node cluster - Confirmed commons-collections is upgraded to 3.2.2 - Compiled Hive 1.2.1 and Tez 0.7.0 using Hadoop 2.6.3 pom successfully - Ran some Hive on Tez/MRv2 queries

[jira] [Created] (YARN-4477) FairScheduler: encounter infinite loop in attemptScheduling

Tao Jie created YARN-4477: - Summary: FairScheduler: encounter infinite loop in attemptScheduling Key: YARN-4477 URL: https://issues.apache.org/jira/browse/YARN-4477 Project: Hadoop YARN Issue Type:

[jira] [Created] (YARN-4478) [Umbrella] : Track all the Test failures in YARN

Rohith Sharma K S created YARN-4478: --- Summary: [Umbrella] : Track all the Test failures in YARN Key: YARN-4478 URL: https://issues.apache.org/jira/browse/YARN-4478 Project: Hadoop YARN