RE: [VOTE] merge feature branch YARN-2928

+1 (Binding), May be we need to further look into the points mentioned by Allen ! Regards, + Naga From: Allen Wittenauer [a...@apache.org] Sent: Tuesday, July 05, 2016 11:40 To: yarn-dev@hadoop.apache.org Subject: Re: [VOTE] merge feature branch YARN-2928

RE: Sending an event from YARN to applications running on the cluster

Hi Charles, Ok but should it not be the reverse way : polling the other way round to get these stats from the REST (preferable) /RPC and do this calculation in the application or the manager side ? May be if there is any short fall in information from REST for an app then we need to expose

RE: Sending an event from YARN to applications running on the cluster

Hi Charles, What YARN event needs to be sent ? may be some info on it will be useful to help you. Regards, + Naga From: Charlie Wright [charliewri...@live.ca] Sent: Thursday, January 14, 2016 08:01 To: yarn-dev@hadoop.apache.org Subject: Sending an event

RE: can anyone give me a hint on "lhs" and "rhs"

LHS : left hand side RHS : Right hand side From: Allen Zhang [allenzhang...@126.com] Sent: Tuesday, December 22, 2015 19:30 To: yarn-dev@hadoop.apache.org Subject: can anyone give me a hint on "lhs" and "rhs" Hello all, can anyone give me a hint on "lhs"

RE: [VOTE] Release Apache Hadoop 2.7.2 RC1

+1 (Non binding) * Release notes and CHANGES.txt looked fine * Verified md5 checksums for both source and binary tar balls * Ran `mvn apache-rat:check` on source tarball, passed * Installed pseudo cluster * Did some basic filesystem operations * Ran a grep, sleepjob, word count successfully *

RE: label based scheduler for fair scheduler?

simha G R (Naga)" <garlanaganarasi...@huawei.com> wrote: >FYI ... YARN-2497 > >From: Allen Zhang [allenzhang...@126.com] >Sent: Thursday, December 17, 2015 20:09 >To: yarn-dev@hadoop.apache.org >Subject: Re: label based schedule

RE: does yarn 2.7.1 really support label expr like && or || ?

Hi Allen, Node Labels are of 2 types Partitions and Constraints, With Partition labels you will be able get the resource gurantees and it almost works like a cluster within a cluster. node_label_expression for a resource request can mention only one partition ! and Constraints are like

RE: label based scheduler for fair scheduler?

FYI ... YARN-2497 From: Allen Zhang [allenzhang...@126.com] Sent: Thursday, December 17, 2015 20:09 To: yarn-dev@hadoop.apache.org Subject: Re: label based scheduler for fair scheduler? any JIRAs related to this? Thanks. At 2015-12-17 05:34:53,

RE: Re:How to run distributed-shell with node label expression??

Hi Allen, I am planning to capture them as part of YARN-4415. (either update the documentation or change in the behavior to consider max capacity as 100) Basically behavior of capacity configuration for queue in default partition is capacity =0 and max capacity =100 But for labels current

RE: Meet A java null pointer when use unmanaged AM and yarn.resourcemanager.system-metrics-publisher.enabled is enable

, Naganarasimha G R (Naga) < garlanaganarasi...@huawei.com> wrote: > Hi Wen Lin, > > This seems to be a bug in the code, which version are you using, seems > like it can happen in any version but just to confirm > Would you like to raise and handle the issue or shall i ?

RE: [Timeline V2 branch] Latest timeline v2 and SMP problem

vent/entity ? was it in RM Recover mode ? Regards, + Naga From: Varun Saxena [vsaxena.va...@gmail.com] Sent: Tuesday, December 15, 2015 10:48 To: Li Lu Cc: yarn-dev@hadoop.apache.org; Sangjin Lee; Junping Du; Vrushali Channapattan; Joep Rottinghuis; Naganarasimha

RE: Meet A java null pointer when use unmanaged AM and yarn.resourcemanager.system-metrics-publisher.enabled is enable

Hi Wen Lin, This seems to be a bug in the code, which version are you using, seems like it can happen in any version but just to confirm Would you like to raise and handle the issue or shall i ? Regards, + Naga From: Wen Lin [w...@pivotal.io] Sent:

RE: [YARN-2928] first drop on trunk

Hi Sangjin, Seems like the list is fine, can we have YARN-4343 too in the list ? + Naga From: Varun Saxena [vsaxena.va...@gmail.com] Sent: Saturday, November 14, 2015 01:46 To: Sangjin Lee Cc: Joep Rottinghuis; Karthik Kambatla;

RE: [VOTE] Release Apache Hadoop 2.7.2 RC0

Hi Vinod, Can you also look at YARN-2859, if its important then we might have to revert that one and fix it for 2.7.2. + Naga From: Vinod Kumar Vavilapalli [vino...@apache.org] Sent: Friday, November 13, 2015 23:56 To: Jason Lowe Cc:

RE: [VOTE] Release Apache Hadoop 2.7.2 RC0

+1 (Non Binding) * Release notes and CHANGES.txt looked fine * Verified md5 checksums for both the tar balls * Ran `mvn apache-rat:check` on source tarball, passed * Built with "-Pdist" from source tarball * Installed pseudo cluster * Did some basic filesystem operations * Ran a grep, sleepjob,

RE: [YARN-2928] working on rebase with trunk

Hi Sangjin, Yes it would be fine you can go ahead, Issue 1: i would take a look at it as it might be impacted by earlier Jira YARN-4129. Once the merge is finished will take look into it. Issue 2: "reconciling with YARN-4183": yes you are right it requires more discussion and also some other

RE: 2.7.2 release plan

Thanks Wangda, for looking into the list, If the changes are riskier then hoping to get 2.8 earlier :) + Naga From: Wangda Tan [wheele...@gmail.com] Sent: Monday, November 02, 2015 13:54 To: Naganarasimha G R (Naga) Cc: yarn-dev@hadoop.apache.org; Tsuyoshi

RE: 2.7.2 release plan

reduce the stability going from 2.7.2 to 2.7.3? If yes, can we just leave them for 2.8.0? On Wed, Oct 28, 2015 at 12:07 PM, Naganarasimha G R (Naga) < garlanaganarasi...@huawei.com> wrote: > Yes Vinod & Tsuyoshi, > > Within a week merging them would be difficult. I can start backp

RE: 2.7.2 release plan

, Naganarasimha G R (Naga) <garlanaganarasi...@huawei.com> wrote: > Hi Vinod, & Wangda > > I think it would be good to backport, following jira's related to NodeLabels > as it will improve debug ability and usability of NodeLabels > > Key

RE: 2.7.2 release plan

Sent: Wednesday, October 28, 2015 23:13 To: Vinod Vavilapalli Cc: yarn-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; common-...@hadoop.apache.org; Vinod Kumar Vavilapalli; Wangda Tan; Tsuyoshi Ozawa; Naganarasimha G R (Naga) Subject: Re: 2.7.2 release plan Vinod, Thank you for taking

RE: [VOTE] Release Apache Hadoop 2.6.2

+1 (Non binding) *. dowloaded src verified signatures and successfully compiled *. Downloaded tar.gz verified signatures and installed a pseudo cluster. *. Did some put and get from HDFS and it was fine. *. Ran PI MR application and verified in ATS V1 Web ui , All were working fine Regards, +

RE: 2.7.2 release plan

Hi Vinod, & Wangda I think it would be good to backport, following jira's related to NodeLabels as it will improve debug ability and usability of NodeLabels Key Summary YARN-4215 YARN-2492 RMNodeLabels

RE: YARN-2928 branch reset

issues which you want to raise together with it ? +Naga From: Li Lu [l...@hortonworks.com] Sent: Friday, August 14, 2015 03:32 To: Vinod Kumar Vavilapalli Cc: yarn-dev@hadoop.apache.org; Zhijie Shen; Junping Du; Sangjin Lee; Vrushali Channapattan; Naganarasimha G R

RE: YARN-2928 branch reset

From: Vinod Kumar Vavilapalli [vino...@hortonworks.com] Sent: Friday, August 14, 2015 02:51 To: yarn-dev@hadoop.apache.org Cc: Li Lu; Zhijie Shen; Junping Du; Sangjin Lee; Vrushali Channapattan; Naganarasimha G R (Naga) Subject: Re: YARN-2928 branch reset I had quite a bit of trouble with some

Is it valid to use userLimit in calculating maxApplicationsPerUser ?

Hi Folks , Came across one scenario where in maxApplications @ cluster level(2 node) was set to a low value like 10 and based on capacity configuration for a particular queue it was coming to 2 as value, but further while calculating maxApplicationsPerUser formula used is :

One query related to cgroupsin 2.4.1

Hi All, In 2.4.1 When there is multiple components like Hbase and yarn runnning on the same node can we restrict the cpu usage of of all YARN containers in that NM through cgroups Assume a node has 10 cores and cpu vcores for this NM is configured to 7. so suppose 7 containers each requesting

Some information required related to Hadoop documentation developement

Hi All, I wanted to know the following related to Hadoop Documentation 1. what and how are these *.vm files(format) used in Hadoop [Hdfs and Yarn] documentation ? 2. Does vm files stand for velocity macro? 3. Do we use some kind of editor to create/update these files ? 4. Are there

RE: Regarding interface to modify the YARN acls?

, Sep 16, 2014 at 7:35 PM, Naganarasimha G R (Naga) garlanaganarasi...@huawei.com wrote: Hi All, Wanted to know, is there any interface (like java interface, rest api etc... ) to modify the ACLS of the YARN queues (CS or FS). Regards, Naga Huawei Technologies Co., Ltd. Phone: Fax

Regarding interface to modify the YARN acls?

Hi All, Wanted to know, is there any interface (like java interface, rest api etc... ) to modify the ACLS of the YARN queues (CS or FS). Regards, Naga Huawei Technologies Co., Ltd. Phone: Fax: Mobile: +91 9980040283 Email: naganarasimh...@huawei.commailto:naganarasimh...@huawei.com Huawei

Regarding a scenario where applications are being hung

Hi , I have one sceanrio which makes the applications to get hung , so wanted to validate whether its a bug (if so will raise a jira). Consider a cluster setup which has 2 NMS of each 8GB resource, And 2 applications are launched in the default queue where in each AM is taking 2 GB

RE: Regarding a scenario where applications are being hung

questions. Thanks, Wangda On Mon, Sep 1, 2014 at 10:46 PM, Naganarasimha G R (Naga) garlanaganarasi...@huawei.com wrote: Hi , I have one sceanrio which makes the applications to get hung , so wanted to validate whether its a bug (if so will raise a jira). Consider a cluster setup which has

RE: Regarding a scenario where applications are being hung

time. Preemption is another story, preemption is used to reclaim resource for a queue under-satisfied from a queue over-satisfied. It's not your case too. Hope this helps, Wangda On Mon, Sep 1, 2014 at 11:11 PM, Naganarasimha G R (Naga) garlanaganarasi...@huawei.com wrote: Hi Wangda

Can anyone please add me as a yarn contributor,

Hi, Could anyone please add me as a yarn contributor, I would like contribute to YARN project. Regards, Naga Huawei Technologies Co., Ltd. Phone: Fax: Mobile: +91 9980040283 Email: naganarasimh...@huawei.commailto:naganarasimh...@huawei.com Huawei Technologies Co., Ltd. Bantian,

RE: Need help in understanding hadoop yarn source code

Hi Anurag, After having the basic understanding of the modules and responsibility you can start with this link http://ercoppa.github.io/HadoopInternals/. I will be quite helpful in understanding the FSM's Regards, Naga Huawei Technologies Co., Ltd. Phone: Fax: Mobile: +91