[jira] [Updated] (YARN-2124) ProportionalCapacityPreemptionPolicy cannot work because it's initialized before scheduler initialized

2014-06-10 Thread Wangda Tan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-2124:
-

Attachment: YARN-2124.patch

Uploaded patch addressed [~jianhe]'s suggestion, move getRMActiveService from 
ResourceManager to MockRM.

 ProportionalCapacityPreemptionPolicy cannot work because it's initialized 
 before scheduler initialized
 --

 Key: YARN-2124
 URL: https://issues.apache.org/jira/browse/YARN-2124
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager, scheduler
Affects Versions: 3.0.0
Reporter: Wangda Tan
Assignee: Wangda Tan
Priority: Critical
 Attachments: YARN-2124.patch, YARN-2124.patch


 When I play with scheduler with preemption, I found 
 ProportionalCapacityPreemptionPolicy cannot work. NPE will be raised when RM 
 start
 {code}
 2014-06-05 11:01:33,201 ERROR 
 org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
 Thread[SchedulingMonitor (ProportionalCapacityPreemptionPolicy),5,main] threw 
 an Exception.
 java.lang.NullPointerException
 at 
 org.apache.hadoop.yarn.util.resource.Resources.greaterThan(Resources.java:225)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.computeIdealResourceDistribution(ProportionalCapacityPreemptionPolicy.java:302)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.recursivelyComputeIdealAssignment(ProportionalCapacityPreemptionPolicy.java:261)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:198)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:174)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:72)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PreemptionChecker.run(SchedulingMonitor.java:82)
 at java.lang.Thread.run(Thread.java:744)
 {code}
 This is caused by ProportionalCapacityPreemptionPolicy needs 
 ResourceCalculator from CapacityScheduler. But 
 ProportionalCapacityPreemptionPolicy get initialized before CapacityScheduler 
 initialized. So ResourceCalculator will set to null in 
 ProportionalCapacityPreemptionPolicy. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (YARN-2124) ProportionalCapacityPreemptionPolicy cannot work because it's initialized before scheduler initialized

2014-06-05 Thread Wangda Tan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-2124:
-

Attachment: YARN-2124.patch

Attached a patch to solve this problem. Moved 
ProportionalCapacityPreemptionPolicy.init(...) from RMActiveService.init() to 
SchedulerMonitor.serviceInit(...). SchedulerMonitor will be always added after 
Scheduler added, so that ProportionalCapacityPreemptionPolicy will be 
initialized after SchedulerMonitor initialized.
Added a test to ProportionalCapacityPreemptionPolicy to make sure no regression 
of in the future.

 ProportionalCapacityPreemptionPolicy cannot work because it's initialized 
 before scheduler initialized
 --

 Key: YARN-2124
 URL: https://issues.apache.org/jira/browse/YARN-2124
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager, scheduler
Affects Versions: 3.0.0
Reporter: Wangda Tan
Assignee: Wangda Tan
Priority: Critical
 Attachments: YARN-2124.patch


 When I play with scheduler with preemption, I found 
 ProportionalCapacityPreemptionPolicy cannot work. NPE will be raised when RM 
 start
 {code}
 2014-06-05 11:01:33,201 ERROR 
 org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
 Thread[SchedulingMonitor (ProportionalCapacityPreemptionPolicy),5,main] threw 
 an Exception.
 java.lang.NullPointerException
 at 
 org.apache.hadoop.yarn.util.resource.Resources.greaterThan(Resources.java:225)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.computeIdealResourceDistribution(ProportionalCapacityPreemptionPolicy.java:302)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.recursivelyComputeIdealAssignment(ProportionalCapacityPreemptionPolicy.java:261)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:198)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:174)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:72)
 at 
 org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PreemptionChecker.run(SchedulingMonitor.java:82)
 at java.lang.Thread.run(Thread.java:744)
 {code}
 This is caused by ProportionalCapacityPreemptionPolicy needs 
 ResourceCalculator from CapacityScheduler. But 
 ProportionalCapacityPreemptionPolicy get initialized before CapacityScheduler 
 initialized. So ResourceCalculator will set to null in 
 ProportionalCapacityPreemptionPolicy. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)