date:20140131


[ 
https://issues.apache.org/jira/browse/YARN-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887572#comment-13887572
 ] 

Hadoop QA commented on YARN-1578:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12626278/YARN-1578-2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2975//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2975//console

This message is automatically generated.

 Fix how to read history file in FileSystemApplicationHistoryStore
 -

 Key: YARN-1578
 URL: https://issues.apache.org/jira/browse/YARN-1578
 Project: Hadoop YARN
  Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
 Attachments: YARN-1578-2.patch, YARN-1578.patch, 
 application_1390978867235_0001, resoucemanager.log, screenshot.png, 
 screenshot2.pdf


 I carried out PiEstimator job at Hadoop cluster which applied YARN-321.
 After the job end and when I accessed Web UI of HistoryServer, it displayed 
 500. And HistoryServer daemon log was output as follows.
 {code}
 2014-01-09 13:31:12,227 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error 
 handling URI: 
 /applicationhistory/appattempt/appattempt_1389146249925_0008_01
 java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at 
 org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
 (snip...)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore.mergeContainerHistoryData(FileSystemApplicationHistoryStore.java:696)
 at 
 org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore.getContainers(FileSystemApplicationHistoryStore.java:429)
 at 
 org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl.getContainers(ApplicationHistoryManagerImpl.java:201)
 at 
 org.apache.hadoop.yarn.server.webapp.AppAttemptBlock.render(AppAttemptBlock.java:110)
 (snip...)
 {code}
 I confirmed that there was container which was not finished from 
 ApplicationHistory file.
 In ResourceManager daemon log, ResourceManager reserved this container, but 
 did not allocate it.
 When FileSystemApplicationHistoryStore reads container information without 
 finish data in history file, this problem occurs.
 In consideration of the case which there is not finish data, we should fix 
 how to read history file in FileSystemApplicationHistoryStore.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Moved] (YARN-1680) Reducer preemption does not happen if node is blacklisted, intern job get hanged.

2014-01-31 Thread Rohith (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohith moved MAPREDUCE-5734 to YARN-1680:
-

Affects Version/s: (was: 2.2.0)
   2.3.0
   2.2.0
  Key: YARN-1680  (was: MAPREDUCE-5734)
  Project: Hadoop YARN  (was: Hadoop Map/Reduce)

 Reducer preemption does not  happen if node is blacklisted, intern job get 
 hanged.
 --

 Key: YARN-1680
 URL: https://issues.apache.org/jira/browse/YARN-1680
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.2.0, 2.3.0
 Environment: SuSE 11 SP2 + Hadoop-2.3 
Reporter: Rohith

 There are 4 NodeManagers with 8GB each.Total cluster capacity is 32GB.Cluster 
 slow start is set to 1.
 Job is running reducer task occupied 29GB of cluster.One NodeManager(NM-4) is 
 become unstable(3 Map got killed), MRAppMaster blacklisted unstable 
 NodeManager(NM-4). All reducer task are running in cluster now.
 MRAppMaster does not preempt the reducers because for Reducer preemption 
 calculation, headRoom is considering blacklisted nodes memory. This makes 
 jobs to hang forever(ResourceManager does not assing any new containers on 
 blacklisted nodes but returns availableResouce considers cluster free 
 memory). 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

2014-01-31 Thread Rohith (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohith updated YARN-1680:
-

Summary: availableResources sent to applicationMaster in heartbeat should 
exclude blacklistedNodes free memory.  (was: Reducer preemption does not  
happen if node is blacklisted, intern job get hanged.)

Updated issue summary as per Yarn project.

 availableResources sent to applicationMaster in heartbeat should exclude 
 blacklistedNodes free memory.
 --

 Key: YARN-1680
 URL: https://issues.apache.org/jira/browse/YARN-1680
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.2.0, 2.3.0
 Environment: SuSE 11 SP2 + Hadoop-2.3 
Reporter: Rohith

 There are 4 NodeManagers with 8GB each.Total cluster capacity is 32GB.Cluster 
 slow start is set to 1.
 Job is running reducer task occupied 29GB of cluster.One NodeManager(NM-4) is 
 become unstable(3 Map got killed), MRAppMaster blacklisted unstable 
 NodeManager(NM-4). All reducer task are running in cluster now.
 MRAppMaster does not preempt the reducers because for Reducer preemption 
 calculation, headRoom is considering blacklisted nodes memory. This makes 
 jobs to hang forever(ResourceManager does not assing any new containers on 
 blacklisted nodes but returns availableResouce considers cluster free 
 memory). 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1617) Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate


[ 
https://issues.apache.org/jira/browse/YARN-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887652#comment-13887652
 ] 

Hudson commented on YARN-1617:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #467 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/467/])
YARN-1617. Remove ancient comment and surround LOG.debug in 
AppSchedulingInfo.allocate (Sandy Ryza) (sandy: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563004)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java


 Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate
 ---

 Key: YARN-1617
 URL: https://issues.apache.org/jira/browse/YARN-1617
 Project: Hadoop YARN
  Issue Type: Bug
  Components: scheduler
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 2.4.0

 Attachments: YARN-1617.patch


 {code}
   synchronized private void allocate(Container container) {
 // Update consumption and track allocations
 //TODO: fixme sharad
 /* try {
 store.storeContainer(container);
   } catch (IOException ie) {
 // TODO fix this. we shouldnt ignore
   }*/
 
 LOG.debug(allocate: applicationId= + applicationId +  container=
 + container.getId() +  host=
 + container.getNodeId().toString());
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-321) Generic application history service


[ 
https://issues.apache.org/jira/browse/YARN-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887647#comment-13887647
 ] 

Hudson commented on YARN-321:
-

SUCCESS: Integrated in Hadoop-Yarn-trunk #467 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/467/])
Updating trunk's YARN CHANGES.txt after YARN-321 merge. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1562950)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt


 Generic application history service
 ---

 Key: YARN-321
 URL: https://issues.apache.org/jira/browse/YARN-321
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Luke Lu
 Attachments: AHS Diagram.pdf, ApplicationHistoryServiceHighLevel.pdf, 
 Generic Application History - Design-20131219.pdf, HistoryStorageDemo.java


 The mapreduce job history server currently needs to be deployed as a trusted 
 server in sync with the mapreduce runtime. Every new application would need a 
 similar application history server. Having to deploy O(T*V) (where T is 
 number of type of application, V is number of version of application) trusted 
 servers is clearly not scalable.
 Job history storage handling itself is pretty generic: move the logs and 
 history data into a particular directory for later serving. Job history data 
 is already stored as json (or binary avro). I propose that we create only one 
 trusted application history server, which can have a generic UI (display json 
 as a tree of strings) as well. Specific application/version can deploy 
 untrusted webapps (a la AMs) to query the application history server and 
 interpret the json for its specific UI and/or analytics.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues


[ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887649#comment-13887649
 ] 

Hudson commented on YARN-1498:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #467 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/467/])
YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
(sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563021)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSParentQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetrics.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java


 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues


[ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887733#comment-13887733
 ] 

Hudson commented on YARN-1498:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1684 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1684/])
YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
(sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563021)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSParentQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetrics.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java


 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1617) Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate


[ 
https://issues.apache.org/jira/browse/YARN-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887736#comment-13887736
 ] 

Hudson commented on YARN-1617:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1684 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1684/])
YARN-1617. Remove ancient comment and surround LOG.debug in 
AppSchedulingInfo.allocate (Sandy Ryza) (sandy: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563004)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java


 Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate
 ---

 Key: YARN-1617
 URL: https://issues.apache.org/jira/browse/YARN-1617
 Project: Hadoop YARN
  Issue Type: Bug
  Components: scheduler
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 2.4.0

 Attachments: YARN-1617.patch


 {code}
   synchronized private void allocate(Container container) {
 // Update consumption and track allocations
 //TODO: fixme sharad
 /* try {
 store.storeContainer(container);
   } catch (IOException ie) {
 // TODO fix this. we shouldnt ignore
   }*/
 
 LOG.debug(allocate: applicationId= + applicationId +  container=
 + container.getId() +  host=
 + container.getNodeId().toString());
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-321) Generic application history service


[ 
https://issues.apache.org/jira/browse/YARN-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887731#comment-13887731
 ] 

Hudson commented on YARN-321:
-

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1684 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1684/])
Updating trunk's YARN CHANGES.txt after YARN-321 merge. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1562950)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt


 Generic application history service
 ---

 Key: YARN-321
 URL: https://issues.apache.org/jira/browse/YARN-321
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Luke Lu
 Attachments: AHS Diagram.pdf, ApplicationHistoryServiceHighLevel.pdf, 
 Generic Application History - Design-20131219.pdf, HistoryStorageDemo.java


 The mapreduce job history server currently needs to be deployed as a trusted 
 server in sync with the mapreduce runtime. Every new application would need a 
 similar application history server. Having to deploy O(T*V) (where T is 
 number of type of application, V is number of version of application) trusted 
 servers is clearly not scalable.
 Job history storage handling itself is pretty generic: move the logs and 
 history data into a particular directory for later serving. Job history data 
 is already stored as json (or binary avro). I propose that we create only one 
 trusted application history server, which can have a generic UI (display json 
 as a tree of strings) as well. Specific application/version can deploy 
 untrusted webapps (a la AMs) to query the application history server and 
 interpret the json for its specific UI and/or analytics.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues


[ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887747#comment-13887747
 ] 

Hudson commented on YARN-1498:
--

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1659 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1659/])
YARN-1498. Common scheduler changes for moving apps between queues (Sandy Ryza) 
(sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563021)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSParentQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetrics.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java


 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-321) Generic application history service


[ 
https://issues.apache.org/jira/browse/YARN-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887745#comment-13887745
 ] 

Hudson commented on YARN-321:
-

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1659 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1659/])
Updating trunk's YARN CHANGES.txt after YARN-321 merge. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1562950)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt


 Generic application history service
 ---

 Key: YARN-321
 URL: https://issues.apache.org/jira/browse/YARN-321
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Luke Lu
 Attachments: AHS Diagram.pdf, ApplicationHistoryServiceHighLevel.pdf, 
 Generic Application History - Design-20131219.pdf, HistoryStorageDemo.java


 The mapreduce job history server currently needs to be deployed as a trusted 
 server in sync with the mapreduce runtime. Every new application would need a 
 similar application history server. Having to deploy O(T*V) (where T is 
 number of type of application, V is number of version of application) trusted 
 servers is clearly not scalable.
 Job history storage handling itself is pretty generic: move the logs and 
 history data into a particular directory for later serving. Job history data 
 is already stored as json (or binary avro). I propose that we create only one 
 trusted application history server, which can have a generic UI (display json 
 as a tree of strings) as well. Specific application/version can deploy 
 untrusted webapps (a la AMs) to query the application history server and 
 interpret the json for its specific UI and/or analytics.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1617) Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate


[ 
https://issues.apache.org/jira/browse/YARN-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887750#comment-13887750
 ] 

Hudson commented on YARN-1617:
--

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1659 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1659/])
YARN-1617. Remove ancient comment and surround LOG.debug in 
AppSchedulingInfo.allocate (Sandy Ryza) (sandy: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563004)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java


 Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate
 ---

 Key: YARN-1617
 URL: https://issues.apache.org/jira/browse/YARN-1617
 Project: Hadoop YARN
  Issue Type: Bug
  Components: scheduler
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 2.4.0

 Attachments: YARN-1617.patch


 {code}
   synchronized private void allocate(Container container) {
 // Update consumption and track allocations
 //TODO: fixme sharad
 /* try {
 store.storeContainer(container);
   } catch (IOException ie) {
 // TODO fix this. we shouldnt ignore
   }*/
 
 LOG.debug(allocate: applicationId= + applicationId +  container=
 + container.getId() +  host=
 + container.getNodeId().toString());
   }
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1632) TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package


[ 
https://issues.apache.org/jira/browse/YARN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887868#comment-13887868
 ] 

Hudson commented on YARN-1632:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5082 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5082/])
YARN-1632. TestApplicationMasterServices should be under 
org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) 
(jeagles: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563157)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice/TestApplicationMasterService.java


 TestApplicationMasterServices should be under 
 org.apache.hadoop.yarn.server.resourcemanager package
 ---

 Key: YARN-1632
 URL: https://issues.apache.org/jira/browse/YARN-1632
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 0.23.9, 2.2.0
Reporter: Chen He
Assignee: Chen He
Priority: Minor
 Attachments: yarn-1632v2.patch


 ApplicationMasterService is under 
 org.apache.hadoop.yarn.server.resourcemanager package. However, its unit test 
 file TestApplicationMasterService is placed under 
 org.apache.hadoop.yarn.server.resourcemanager.applicationmasterservice 
 package which only contains one file (TestApplicationMasterService). 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1632) TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package

2014-01-31 Thread Jonathan Eagles (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Eagles updated YARN-1632:
--

Fix Version/s: 3.0.0
   2.4.0

 TestApplicationMasterServices should be under 
 org.apache.hadoop.yarn.server.resourcemanager package
 ---

 Key: YARN-1632
 URL: https://issues.apache.org/jira/browse/YARN-1632
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 0.23.9, 2.2.0
Reporter: Chen He
Assignee: Chen He
Priority: Minor
 Fix For: 3.0.0, 2.4.0

 Attachments: yarn-1632v2.patch


 ApplicationMasterService is under 
 org.apache.hadoop.yarn.server.resourcemanager package. However, its unit test 
 file TestApplicationMasterService is placed under 
 org.apache.hadoop.yarn.server.resourcemanager.applicationmasterservice 
 package which only contains one file (TestApplicationMasterService). 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1632) TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package


[ 
https://issues.apache.org/jira/browse/YARN-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887880#comment-13887880
 ] 

Hudson commented on YARN-1632:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5083 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5083/])
YARN-1632. TestApplicationMasterServices should be under 
org.apache.hadoop.yarn.server.resourcemanager package (Chen He via jeagles) 
(jeagles: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563160)
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice


 TestApplicationMasterServices should be under 
 org.apache.hadoop.yarn.server.resourcemanager package
 ---

 Key: YARN-1632
 URL: https://issues.apache.org/jira/browse/YARN-1632
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 0.23.9, 2.2.0
Reporter: Chen He
Assignee: Chen He
Priority: Minor
 Fix For: 3.0.0, 2.4.0

 Attachments: yarn-1632v2.patch


 ApplicationMasterService is under 
 org.apache.hadoop.yarn.server.resourcemanager package. However, its unit test 
 file TestApplicationMasterService is placed under 
 org.apache.hadoop.yarn.server.resourcemanager.applicationmasterservice 
 package which only contains one file (TestApplicationMasterService). 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs


[ 
https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887918#comment-13887918
 ] 

Karthik Kambatla commented on YARN-1461:


The findbugs issue is unrelated - comes from YARN-1498, which I reopened to fix 
it. 

[~zjshen] - I believe the latest patch incorporates all your suggestions. Mind 
taking another look? 

 RM API and RM changes to handle tags for running jobs
 -

 Key: YARN-1461
 URL: https://issues.apache.org/jira/browse/YARN-1461
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Attachments: yarn-1461-1.patch, yarn-1461-10.patch, 
 yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, 
 yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, 
 yarn-1461-9.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Reopened] (YARN-1498) Common scheduler changes for moving apps between queues


 [ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Kambatla reopened YARN-1498:



This seems to cause a findbugs issue - 
https://builds.apache.org/job/PreCommit-YARN-Build/2974//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html#Warnings_MT_CORRECTNESS

Can we make SchedulerApplicationAttempt#move() a synchronized method in an 
addendum patch? 

 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1479) Invalid NaN values in Hadoop REST API JSON response

2014-01-31 Thread Chen He (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chen He updated YARN-1479:
--

Attachment: Yarn-1479v2.patch

 Invalid NaN values in Hadoop REST API JSON response
 ---

 Key: YARN-1479
 URL: https://issues.apache.org/jira/browse/YARN-1479
 Project: Hadoop YARN
  Issue Type: Improvement
Affects Versions: 0.23.6, 2.0.4-alpha
Reporter: Kendall Thrapp
Assignee: Chen He
 Fix For: 2.3.0

 Attachments: Yarn-1479.patch, Yarn-1479v2.patch


 I've been occasionally coming across instances where Hadoop's Cluster 
 Applications REST API 
 (http://hadoop.apache.org/docs/r0.23.6/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_API)
  has returned JSON that PHP's json_decode function failed to parse.  I've 
 tracked the syntax error down to the presence of the unquoted word NaN 
 appearing as a value in the JSON.  For example:
 progress:NaN,
 NaN is not part of the JSON spec, so its presence renders the whole JSON 
 string invalid.  Hadoop needs to return something other than NaN in this case 
 -- perhaps an empty string or the quoted string NaN.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1479) Invalid NaN values in Hadoop REST API JSON response

2014-01-31 Thread Chen He (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chen He updated YARN-1479:
--

Attachment: (was: Yarn-1479v2.patch)

 Invalid NaN values in Hadoop REST API JSON response
 ---

 Key: YARN-1479
 URL: https://issues.apache.org/jira/browse/YARN-1479
 Project: Hadoop YARN
  Issue Type: Improvement
Affects Versions: 0.23.6, 2.0.4-alpha
Reporter: Kendall Thrapp
Assignee: Chen He
 Fix For: 2.3.0

 Attachments: Yarn-1479.patch


 I've been occasionally coming across instances where Hadoop's Cluster 
 Applications REST API 
 (http://hadoop.apache.org/docs/r0.23.6/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_API)
  has returned JSON that PHP's json_decode function failed to parse.  I've 
 tracked the syntax error down to the presence of the unquoted word NaN 
 appearing as a value in the JSON.  For example:
 progress:NaN,
 NaN is not part of the JSON spec, so its presence renders the whole JSON 
 string invalid.  Hadoop needs to return something other than NaN in this case 
 -- perhaps an empty string or the quoted string NaN.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1479) Invalid NaN values in Hadoop REST API JSON response


[ 
https://issues.apache.org/jira/browse/YARN-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888143#comment-13888143
 ] 

Hadoop QA commented on YARN-1479:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12626362/Yarn-1479v2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified test files.

  {color:red}-1 javac{color}.  The applied patch generated 1545 javac 
compiler warnings (more than the trunk's current 1540 warnings).

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 1 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2976//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-YARN-Build/2976//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Javac warnings: 
https://builds.apache.org/job/PreCommit-YARN-Build/2976//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2976//console

This message is automatically generated.

 Invalid NaN values in Hadoop REST API JSON response
 ---

 Key: YARN-1479
 URL: https://issues.apache.org/jira/browse/YARN-1479
 Project: Hadoop YARN
  Issue Type: Improvement
Affects Versions: 0.23.6, 2.0.4-alpha
Reporter: Kendall Thrapp
Assignee: Chen He
 Fix For: 2.3.0

 Attachments: Yarn-1479.patch


 I've been occasionally coming across instances where Hadoop's Cluster 
 Applications REST API 
 (http://hadoop.apache.org/docs/r0.23.6/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_API)
  has returned JSON that PHP's json_decode function failed to parse.  I've 
 tracked the syntax error down to the presence of the unquoted word NaN 
 appearing as a value in the JSON.  For example:
 progress:NaN,
 NaN is not part of the JSON spec, so its presence renders the whole JSON 
 string invalid.  Hadoop needs to return something other than NaN in this case 
 -- perhaps an empty string or the quoted string NaN.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1633) Define user-faced entity, entity-info and event objects

2014-01-31 Thread Zhijie Shen (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhijie Shen updated YARN-1633:
--

Attachment: YARN-1633.2.patch

Upload a new patch, which incorporates Vinod's comments.

 Define user-faced entity, entity-info and event objects
 ---

 Key: YARN-1633
 URL: https://issues.apache.org/jira/browse/YARN-1633
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
 Attachments: YARN-1633.1.patch, YARN-1633.2.patch


 Define the core objects of the application-timeline effort.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1530) [Umbrella] Store, manage and serve per-framework application-timeline data

2014-01-31 Thread Chris Riccomini (JIRA)

[
https://issues.apache.org/jira/browse/YARN-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888286#comment-13888286
]

Chris Riccomini commented on YARN-1530:
---

Had a look at the design doc. I think I'm starting to get it, but I have a
couple of questions:

1. Is the expectation that people will be able to use this for INFO, WARN,
ERROR type application logging?
2. Regarding app-specific UIs, is it going to be possible to embed app-specific
UIs with YARN's UI, instead of having to run an app-specific web-ui? There is
some mention of JS UIs, but it's a little unclear whether this would be
embedded in YARN, or served from somewhere (to quote the docs. If it's served
from the RM (or some other web-ui in YARN), will it be up to ops to decide
which libraries are embedded, or up to
3. What are the planned out of the box implementations for both the storage
and transport layers? REST+LevelDB+HBase? Are Flume and Kafka implementations
expected to happen outside of the YARN project?

[Umbrella] Store, manage and serve per-framework application-timeline data
--

Key: YARN-1530
URL: https://issues.apache.org/jira/browse/YARN-1530
Project: Hadoop YARN
Issue Type: Bug
Reporter: Vinod Kumar Vavilapalli
Attachments: application timeline design-20140108.pdf, application
timeline design-20140116.pdf, application timeline design-20140130.pdf

This is a sibling JIRA for YARN-321.
Today, each application/framework has to do store, and serve per-framework
data all by itself as YARN doesn't have a common solution. This JIRA attempts
to solve the storage, management and serving of per-framework data from
various applications, both running and finished. The aim is to change YARN to
collect and store data in a generic manner with plugin points for frameworks
to do their own thing w.r.t interpretation and serving.

--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects

2014-01-31 Thread Billie Rinaldi (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billie Rinaldi updated YARN-1659:
-

Attachment: YARN-1659-5.patch

New patch adds javadocs to the interface.

 Define ApplicationTimelineStore interface and store-facing entity, 
 entity-info and event objects
 

 Key: YARN-1659
 URL: https://issues.apache.org/jira/browse/YARN-1659
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
 Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, 
 YARN-1659-5.patch, YARN-1659.2.patch


 These will be used by ApplicationTimelineStore interface.  The web services 
 will convert the store-facing obects to the user-facing objects.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1633) Define user-faced entity, entity-info and event objects


[ 
https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888315#comment-13888315
 ] 

Hadoop QA commented on YARN-1633:
-

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12626413/YARN-1633.2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2977//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2977//console

This message is automatically generated.

 Define user-faced entity, entity-info and event objects
 ---

 Key: YARN-1633
 URL: https://issues.apache.org/jira/browse/YARN-1633
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
 Attachments: YARN-1633.1.patch, YARN-1633.2.patch


 Define the core objects of the application-timeline effort.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects


[ 
https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888324#comment-13888324
 ] 

Hadoop QA commented on YARN-1659:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12626417/YARN-1659-5.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2978//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2978//console

This message is automatically generated.

 Define ApplicationTimelineStore interface and store-facing entity, 
 entity-info and event objects
 

 Key: YARN-1659
 URL: https://issues.apache.org/jira/browse/YARN-1659
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
 Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, 
 YARN-1659-5.patch, YARN-1659.2.patch


 These will be used by ApplicationTimelineStore interface.  The web services 
 will convert the store-facing obects to the user-facing objects.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects

2014-01-31 Thread Zhijie Shen (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888329#comment-13888329
 ] 

Zhijie Shen commented on YARN-1659:
---

Billie, thanks for the patch. Here're some general comments:

1. Would you please add the javadoc at the class level, including public inner 
classes and enums?
2. Please add some simple javadoc for the setter/getter in the store-faced pojo 
classes as well?
3. Change event.ts-event.timestamp?
4. Like YARN-1633, write a simple test case for the pjo classes?
5. Make sure the license block start witl /**

 Define ApplicationTimelineStore interface and store-facing entity, 
 entity-info and event objects
 

 Key: YARN-1659
 URL: https://issues.apache.org/jira/browse/YARN-1659
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
 Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, 
 YARN-1659-5.patch, YARN-1659.2.patch


 These will be used by ApplicationTimelineStore interface.  The web services 
 will convert the store-facing obects to the user-facing objects.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1633) Define user-faced entity, entity-info and event objects


[ 
https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888335#comment-13888335
 ] 

Vinod Kumar Vavilapalli commented on YARN-1633:
---

Looks good now. +1. Checking this in.

 Define user-faced entity, entity-info and event objects
 ---

 Key: YARN-1633
 URL: https://issues.apache.org/jira/browse/YARN-1633
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
 Attachments: YARN-1633.1.patch, YARN-1633.2.patch


 Define the core objects of the application-timeline effort.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1633) Define user-faced entity, entity-info and event objects


[ 
https://issues.apache.org/jira/browse/YARN-1633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888341#comment-13888341
 ] 

Hudson commented on YARN-1633:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5089 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5089/])
YARN-1633. Defined user-facing entity, entity-info and event objects related to 
Application Timeline feature. Contributed by Zhijie Shen. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563356)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntities.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvent.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvents.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/package-info.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java


 Define user-faced entity, entity-info and event objects
 ---

 Key: YARN-1633
 URL: https://issues.apache.org/jira/browse/YARN-1633
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
 Fix For: 2.4.0

 Attachments: YARN-1633.1.patch, YARN-1633.2.patch


 Define the core objects of the application-timeline effort.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1637) Implement a client library for java users to post entities+events


[ 
https://issues.apache.org/jira/browse/YARN-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888369#comment-13888369
 ] 

Vinod Kumar Vavilapalli commented on YARN-1637:
---

Actually, should we also drop 'ATS' prefix from the user-facing events?

 Implement a client library for java users to post entities+events
 -

 Key: YARN-1637
 URL: https://issues.apache.org/jira/browse/YARN-1637
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
 Attachments: YARN-1637.1.patch


 This is a wrapper around the web-service to facilitate easy posting of 
 entity+event data to the time-line server.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects

[
https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888380#comment-13888380
]

Vinod Kumar Vavilapalli commented on YARN-1659:
---

Can you summarize the patch so that it's clear what the patch is focused on?

Some more comments
- Please make sure you add audience and stability annotations to all the new
classes.
- Most of these classes are internal and so don't need to be
@InterfaceAudience.Public?

The new objects
- Can we use user-facing EntityInfo, Event etc instead of creating a new
object? Or is this for decoupling user-APIs from our internal APIs.
- Is it needed as part of this patch? We can remove it and include it in the
follow-up patches where it is really needed.

GenericObjectMapper
- Writing long for a int?
- Is it needed as part of this patch? We can remove it and include it in the
follow-up patches where it is really needed.

ApplicationTimelineReader
- DEFAULT_LIMIT isn't used in the patch. For future? Document its use or
remove.
- Can we rename Field.RELATEDENTITIES etc to be hyphenated. Like
RELATED_ENTITIES?
- fields parameter - fieldsToRetrieve

Define ApplicationTimelineStore interface and store-facing entity,
entity-info and event objects

Key: YARN-1659
URL: https://issues.apache.org/jira/browse/YARN-1659
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch,
YARN-1659-5.patch, YARN-1659.2.patch

These will be used by ApplicationTimelineStore interface. The web services
will convert the store-facing obects to the user-facing objects.

--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1659) Define ApplicationTimelineStore interface and store-facing entity, entity-info and event objects


[ 
https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888381#comment-13888381
 ] 

Vinod Kumar Vavilapalli commented on YARN-1659:
---

I think I got confused. Clearly, the JIRA title talks about the object. Either 
we should pull out the new objects into a patch in itself or add tests 
indicating how they are tied to the ApplicationTimelineStore.

 Define ApplicationTimelineStore interface and store-facing entity, 
 entity-info and event objects
 

 Key: YARN-1659
 URL: https://issues.apache.org/jira/browse/YARN-1659
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
 Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, 
 YARN-1659-5.patch, YARN-1659.2.patch


 These will be used by ApplicationTimelineStore interface.  The web services 
 will convert the store-facing obects to the user-facing objects.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover


[ 
https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888391#comment-13888391
 ] 

Xuan Gong commented on YARN-1611:
-

bq. RM_CONF_STORE is actually specific to the FSRemoteConf. Rename it.

Renamed to FS_RM_CONF_STORE

bq. Put audience and stability annotations for the new classes. Mostly keep 
them private and unstable at this point of time.

Added private and unstable annotations for RemoteConfiguration, 
FileSystemBasedRemoteConfiguration, RemoteConfigurationFactory

bq. Create the CS_CONFIGURATION_FILE constant in YarnConfiguration and not 
AdminService. It can be useful beyond, but mark it private for now.

Move CS_CONFIGURATION_FILE from AdminService to YarnConfiguration. But I need 
to use CS_CONFIGURATION_FILE as the input for refreshQueue to get the 
configuration, so mark it as public instead of private.

bq.CapacitySchedulerConfiguration: Why do you need to reloadConfiguration() 
when useRemoteConfiguration is true?

Removed

bq.After this patch is goes in, if HA is enabled, we are forcing users to put 
configs remotely. I agree this is the long term goal, but in the short term, 
can we have an internal non-public flag to enable the remote-configuration 
stuff?

Create YarnConfiguration.RM_HA_REMOTE_CONFIGURATION_ENABLED and set 
YarnConfiguration.DEFAULT_RM_HA_REMOTE_CONFIGURATION_ENABLED as false. 


 Make admin refresh of capacity scheduler configuration work across RM failover
 --

 Key: YARN-1611
 URL: https://issues.apache.org/jira/browse/YARN-1611
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Xuan Gong
Assignee: Xuan Gong
 Attachments: YARN-1611.1.patch, YARN-1611.2.patch, YARN-1611.2.patch, 
 YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, 
 YARN-1611.6.patch


 Currently, If we do refresh* for a standby RM, it will failover to the 
 current active RM, and do the refresh* based on the local configuration file 
 of the active RM. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover


 [ 
https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuan Gong updated YARN-1611:


Attachment: YARN-1611.7.patch

 Make admin refresh of capacity scheduler configuration work across RM failover
 --

 Key: YARN-1611
 URL: https://issues.apache.org/jira/browse/YARN-1611
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Xuan Gong
Assignee: Xuan Gong
 Attachments: YARN-1611.1.patch, YARN-1611.2.patch, YARN-1611.2.patch, 
 YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, 
 YARN-1611.6.patch, YARN-1611.7.patch


 Currently, If we do refresh* for a standby RM, it will failover to the 
 current active RM, and do the refresh* based on the local configuration file 
 of the active RM. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Assigned] (YARN-1540) Add an easy way to turn on HA


 [ 
https://issues.apache.org/jira/browse/YARN-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Kambatla reassigned YARN-1540:
--

Assignee: Karthik Kambatla

 Add an easy way to turn on HA
 -

 Key: YARN-1540
 URL: https://issues.apache.org/jira/browse/YARN-1540
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.3.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla

 Users will have to modify the configuration significantly to turn on HA. It 
 would be nice to have a simpler way of doing this.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Assigned] (YARN-1536) Cleanup: Get rid of ResourceManager#get*SecretManager() methods and use the RMContext methods instead


 [ 
https://issues.apache.org/jira/browse/YARN-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Kambatla reassigned YARN-1536:
--

Assignee: Karthik Kambatla

 Cleanup: Get rid of ResourceManager#get*SecretManager() methods and use the 
 RMContext methods instead
 -

 Key: YARN-1536
 URL: https://issues.apache.org/jira/browse/YARN-1536
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: resourcemanager
Affects Versions: 2.3.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Minor
  Labels: newbie

 Both ResourceManager and RMContext have methods to access the secret 
 managers, and it should be safe (cleaner) to get rid of the ResourceManager 
 methods.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Assigned] (YARN-1370) Fair scheduler to re-populate container allocation state


 [ 
https://issues.apache.org/jira/browse/YARN-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Kambatla reassigned YARN-1370:
--

Assignee: Karthik Kambatla

 Fair scheduler to re-populate container allocation state
 

 Key: YARN-1370
 URL: https://issues.apache.org/jira/browse/YARN-1370
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Reporter: Bikas Saha
Assignee: Karthik Kambatla

 YARN-1367 and YARN-1368 enable the NM to tell the RM about currently running 
 containers and the RM will pass this information to the schedulers along with 
 the node information. The schedulers are currently already informed about 
 previously running apps when the app data is recovered from the store. The 
 scheduler is expected to be able to repopulate its allocation state from the 
 above 2 sources of information.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1498) Common scheduler changes for moving apps between queues

2014-01-31 Thread Sandy Ryza (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandy Ryza updated YARN-1498:
-

Attachment: YARN-1498-addendum.patch

 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498-addendum.patch, 
 YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues

2014-01-31 Thread Sandy Ryza (JIRA)


[ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888439#comment-13888439
 ] 

Sandy Ryza commented on YARN-1498:
--

Good catch.  Attached addendum patch.

 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498-addendum.patch, 
 YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1504) RM changes for moving apps between queues


[ 
https://issues.apache.org/jira/browse/YARN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888448#comment-13888448
 ] 

Hudson commented on YARN-1504:
--

SUCCESS: Integrated in Hadoop-trunk-Commit #5091 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/5091/])
YARN-1504. RM changes for moving apps between queues (Sandy Ryza) (sandy: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1563371)
* 
/hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppMoveEvent.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestMoveApplication.java


 RM changes for moving apps between queues
 -

 Key: YARN-1504
 URL: https://issues.apache.org/jira/browse/YARN-1504
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1504-1.patch, YARN-1504.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1499) Fair Scheduler changes for moving apps between queues

2014-01-31 Thread Sandy Ryza (JIRA)


 [ 
https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandy Ryza updated YARN-1499:
-

Attachment: YARN-1499.patch

 Fair Scheduler changes for moving apps between queues
 -

 Key: YARN-1499
 URL: https://issues.apache.org/jira/browse/YARN-1499
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: scheduler
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Attachments: YARN-1499.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues


[ 
https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888453#comment-13888453
 ] 

Hadoop QA commented on YARN-1498:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12626447/YARN-1498-addendum.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The following test timeouts occurred in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2980//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2980//console

This message is automatically generated.

 Common scheduler changes for moving apps between queues
 ---

 Key: YARN-1498
 URL: https://issues.apache.org/jira/browse/YARN-1498
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Fix For: 3.0.0

 Attachments: YARN-1498-1.patch, YARN-1498-addendum.patch, 
 YARN-1498.patch, YARN-1498.patch


 This JIRA is to track changes that aren't in particular schedulers but that 
 help them support moving apps between queues.  In particular, it makes sure 
 that QueueMetrics are properly updated when an app changes queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1667) Make admin refreshSuperUserGroupsConfiguration work across RM failover


 [ 
https://issues.apache.org/jira/browse/YARN-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuan Gong updated YARN-1667:


Attachment: YARN-1667.2.patch

 Make admin refreshSuperUserGroupsConfiguration work across RM failover
 --

 Key: YARN-1667
 URL: https://issues.apache.org/jira/browse/YARN-1667
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Xuan Gong
Assignee: Xuan Gong
 Attachments: YARN-1667.1.patch, YARN-1667.2.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (YARN-1499) Fair Scheduler changes for moving apps between queues


[ 
https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888458#comment-13888458
 ] 

Hadoop QA commented on YARN-1499:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12626449/YARN-1499.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 2 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

  
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestMaxRunningAppsEnforcer

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2981//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-YARN-Build/2981//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2981//console

This message is automatically generated.

 Fair Scheduler changes for moving apps between queues
 -

 Key: YARN-1499
 URL: https://issues.apache.org/jira/browse/YARN-1499
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: scheduler
Reporter: Sandy Ryza
Assignee: Sandy Ryza
 Attachments: YARN-1499.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1668) Make admin refreshAdminAcls work across RM failover


 [ 
https://issues.apache.org/jira/browse/YARN-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuan Gong updated YARN-1668:


Attachment: YARN-1668.2.patch

 Make admin refreshAdminAcls work across RM failover
 ---

 Key: YARN-1668
 URL: https://issues.apache.org/jira/browse/YARN-1668
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Xuan Gong
Assignee: Xuan Gong
 Attachments: YARN-1668.1.patch, YARN-1668.2.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1669) Make admin refreshServiceAcls work across RM failover


 [ 
https://issues.apache.org/jira/browse/YARN-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuan Gong updated YARN-1669:


Attachment: YARN-1669.2.patch

 Make admin refreshServiceAcls work across RM failover
 -

 Key: YARN-1669
 URL: https://issues.apache.org/jira/browse/YARN-1669
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Xuan Gong
Assignee: Xuan Gong
 Attachments: YARN-1669.1.patch, YARN-1669.2.patch






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1676) Make admin refreshUserToGroupsMappings of configuration work across RM failover