Success: YARN-378 PreCommit Build #503

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-378 Build: https://builds.apache.org/job/PreCommit-YARN-Build/503/ ### ## LAST 60 LINES OF THE CONSOLE ###

Failed: YARN-468 PreCommit Build #504

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-468 Build: https://builds.apache.org/job/PreCommit-YARN-Build/504/ ### ## LAST 60 LINES OF THE CONSOLE ###

Failed: YARN-18 PreCommit Build #505

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-18 Build: https://builds.apache.org/job/PreCommit-YARN-Build/505/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

Failed: YARN-460 PreCommit Build #507

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-460 Build: https://builds.apache.org/job/PreCommit-YARN-Build/507/ ### ## LAST 60 LINES OF THE CONSOLE ###

Failed: YARN-382 PreCommit Build #508

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-382 Build: https://builds.apache.org/job/PreCommit-YARN-Build/508/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Created] (YARN-469) Make scheduling mode in FS pluggable

2013-03-12 Thread Karthik Kambatla (JIRA)
Karthik Kambatla created YARN-469: - Summary: Make scheduling mode in FS pluggable Key: YARN-469 URL: https://issues.apache.org/jira/browse/YARN-469 Project: Hadoop YARN Issue Type: Sub-task

Success: YARN-71 PreCommit Build #509

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-71 Build: https://builds.apache.org/job/PreCommit-YARN-Build/509/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated

Re: Mapping ResourceRequest to returned ListContainer

2013-03-12 Thread Arun C Murthy
YARN doesn't provide this mapping. If you really care, you could ask for containers with different Priority and the returned allocation will tell you what Priority was the origin for the container. Arun On Mar 12, 2013, at 11:11 AM, David Capwell wrote: I had a question on working with

Re: Resource Manager Queue builds up

2013-03-12 Thread Arun C Murthy
What version are you running? On Mar 11, 2013, at 11:28 PM, Muntasir Raihan Rahman wrote: Hi, I am trying to do some experiments with hadoop yarn. I am submitting a large number of jobs to a queue, but after some time the resource manager freezes, and I get the following yarn log message:

Re: Resource Manager Queue builds up

2013-03-12 Thread Muntasir Raihan Rahman
I am using yarn 0.23. Muntasir. On Tue, Mar 12, 2013 at 6:30 PM, Arun C Murthy a...@hortonworks.com wrote: What version are you running? On Mar 11, 2013, at 11:28 PM, Muntasir Raihan Rahman wrote: Hi, I am trying to do some experiments with hadoop yarn. I am submitting a large

[jira] [Created] (YARN-470) Support a way to disable resource monitoring on the NodeManager

2013-03-12 Thread Hitesh Shah (JIRA)
Hitesh Shah created YARN-470: Summary: Support a way to disable resource monitoring on the NodeManager Key: YARN-470 URL: https://issues.apache.org/jira/browse/YARN-470 Project: Hadoop YARN

[jira] [Created] (YARN-471) RM does not validate the resource capability of an NM when the NM registers with the RM

2013-03-12 Thread Hitesh Shah (JIRA)
Hitesh Shah created YARN-471: Summary: RM does not validate the resource capability of an NM when the NM registers with the RM Key: YARN-471 URL: https://issues.apache.org/jira/browse/YARN-471 Project:

Re: Resource Manager Queue builds up

2013-03-12 Thread David Capwell
2.0.3 On Mar 12, 2013 4:31 PM, Arun C Murthy a...@hortonworks.com wrote: What version are you running? On Mar 11, 2013, at 11:28 PM, Muntasir Raihan Rahman wrote: Hi, I am trying to do some experiments with hadoop yarn. I am submitting a large number of jobs to a queue, but after some

[jira] [Created] (YARN-472) Fix MR Job falied if RM restarted when the job is running

2013-03-12 Thread jian he (JIRA)
jian he created YARN-472: Summary: Fix MR Job falied if RM restarted when the job is running Key: YARN-472 URL: https://issues.apache.org/jira/browse/YARN-472 Project: Hadoop YARN Issue Type:

Re: Resource Manager Queue builds up

2013-03-12 Thread Vinod Kumar Vavilapalli
And what's the number of nodes you have? Number of concurrent running applications? ResourceManager log-level? A 1000 event-queue size isn't really a problem, till what size does it keep on increasing? And how did you find out that ResourceManager is freezing, meaning what are the symptoms you

Re: Resource Manager Queue builds up

2013-03-12 Thread Muntasir Raihan Rahman
I am running it on a 8 node emulab cluster. We have one master and 7 slaves, each with 3 containers. The capacity scheduler web interface becomes un-responsive, and the yarn logs show that the queue size keeps increasing. The number of concurrent running applications is 2-3, but they always occupy

Re: Resource Manager Queue builds up

2013-03-12 Thread Muntasir Raihan Rahman
Here is the tail of the latest yarn-log: 2013-03-12 19:43:09,584 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 25000 2013-03-12 19:45:32,588 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 26000 2013-03-12 19:47:55,634 INFO

Re: Resource Manager Queue builds up

2013-03-12 Thread Vinod Kumar Vavilapalli
Okay, I think you ran into some bug/bottleneck, but we need your help to figure this out. Can you wait till the queue builds up beyond some threshold, say 40K events, stop the load for some time, and see if the queue is emptied? Can you share more info on your benchmark/load so that we can

Failed: YARN-18 PreCommit Build #510

2013-03-12 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/YARN-18 Build: https://builds.apache.org/job/PreCommit-YARN-Build/510/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated