[jira] [Updated] (YARN-3026) Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp

2015-07-24 Thread Wangda Tan (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-3026:
-
Attachment: YARN-3026.6.patch

Attached ver.6 addressed [~jianhe]'s comment.

 Move application-specific container allocation logic from LeafQueue to 
 FiCaSchedulerApp
 ---

 Key: YARN-3026
 URL: https://issues.apache.org/jira/browse/YARN-3026
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacityscheduler
Reporter: Wangda Tan
Assignee: Wangda Tan
 Attachments: YARN-3026.1.patch, YARN-3026.2.patch, YARN-3026.3.patch, 
 YARN-3026.4.patch, YARN-3026.5.patch, YARN-3026.6.patch


 Have a discussion with [~vinodkv] and [~jianhe]: 
 In existing Capacity Scheduler, all allocation logics of and under LeafQueue 
 are located in LeafQueue.java in implementation. To make a cleaner scope of 
 LeafQueue, we'd better move some of them to FiCaSchedulerApp.
 Ideal scope of LeafQueue should be: when a LeafQueue receives some resources 
 from ParentQueue (like 15% of cluster resource), and it distributes resources 
 to children apps, and it should be agnostic to internal logic of children 
 apps (like delayed-scheduling, etc.). IAW, LeafQueue shouldn't decide how 
 application allocating container from given resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3026) Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp

2015-07-23 Thread Wangda Tan (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-3026:
-
Attachment: YARN-3026.5.patch

Attached ver.5 fixed some offline suggestions from [~jianhe]:
- RMContextImpl#setActiveServiceContext, TestCapacityScheduler changes unneeded
- FiCaSchedulerApp#assignContainer - assignContainers

And one bug fix:
- In CS (before this patch), excessReservation won't be completed since 
ParentQueue.assignContainers doesn't copy the excessReservation field, so 
CapacityScheduler will not release such excessReservations.
- Currently excessReservation and unreserved containers produced by 
continousReservationLooking are all released at LeafQueue instead of 
{{CapacityScheduler#allocateContainersToNode}}
- Added test for this as well.

 Move application-specific container allocation logic from LeafQueue to 
 FiCaSchedulerApp
 ---

 Key: YARN-3026
 URL: https://issues.apache.org/jira/browse/YARN-3026
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacityscheduler
Reporter: Wangda Tan
Assignee: Wangda Tan
 Attachments: YARN-3026.1.patch, YARN-3026.2.patch, YARN-3026.3.patch, 
 YARN-3026.4.patch, YARN-3026.5.patch


 Have a discussion with [~vinodkv] and [~jianhe]: 
 In existing Capacity Scheduler, all allocation logics of and under LeafQueue 
 are located in LeafQueue.java in implementation. To make a cleaner scope of 
 LeafQueue, we'd better move some of them to FiCaSchedulerApp.
 Ideal scope of LeafQueue should be: when a LeafQueue receives some resources 
 from ParentQueue (like 15% of cluster resource), and it distributes resources 
 to children apps, and it should be agnostic to internal logic of children 
 apps (like delayed-scheduling, etc.). IAW, LeafQueue shouldn't decide how 
 application allocating container from given resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3026) Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp

2015-07-22 Thread Wangda Tan (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-3026:
-
Attachment: YARN-3026.4.patch

Attached ver.4 patch, addressed comments from [~jianhe].

 Move application-specific container allocation logic from LeafQueue to 
 FiCaSchedulerApp
 ---

 Key: YARN-3026
 URL: https://issues.apache.org/jira/browse/YARN-3026
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacityscheduler
Reporter: Wangda Tan
Assignee: Wangda Tan
 Attachments: YARN-3026.1.patch, YARN-3026.2.patch, YARN-3026.3.patch, 
 YARN-3026.4.patch


 Have a discussion with [~vinodkv] and [~jianhe]: 
 In existing Capacity Scheduler, all allocation logics of and under LeafQueue 
 are located in LeafQueue.java in implementation. To make a cleaner scope of 
 LeafQueue, we'd better move some of them to FiCaSchedulerApp.
 Ideal scope of LeafQueue should be: when a LeafQueue receives some resources 
 from ParentQueue (like 15% of cluster resource), and it distributes resources 
 to children apps, and it should be agnostic to internal logic of children 
 apps (like delayed-scheduling, etc.). IAW, LeafQueue shouldn't decide how 
 application allocating container from given resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3026) Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp

2015-07-20 Thread Wangda Tan (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-3026:
-
Attachment: YARN-3026.3.patch

Attached ver.3 patch, reworked on the patch, since we have a lot of code 
changes since ver.2.

 Move application-specific container allocation logic from LeafQueue to 
 FiCaSchedulerApp
 ---

 Key: YARN-3026
 URL: https://issues.apache.org/jira/browse/YARN-3026
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacityscheduler
Reporter: Wangda Tan
Assignee: Wangda Tan
 Attachments: YARN-3026.1.patch, YARN-3026.2.patch, YARN-3026.3.patch


 Have a discussion with [~vinodkv] and [~jianhe]: 
 In existing Capacity Scheduler, all allocation logics of and under LeafQueue 
 are located in LeafQueue.java in implementation. To make a cleaner scope of 
 LeafQueue, we'd better move some of them to FiCaSchedulerApp.
 Ideal scope of LeafQueue should be: when a LeafQueue receives some resources 
 from ParentQueue (like 15% of cluster resource), and it distributes resources 
 to children apps, and it should be agnostic to internal logic of children 
 apps (like delayed-scheduling, etc.). IAW, LeafQueue shouldn't decide how 
 application allocating container from given resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3026) Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp

2015-01-14 Thread Wangda Tan (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-3026:
-
Attachment: YARN-3026.2.patch

Rebased against trunk and fixed findbugs warning.

 Move application-specific container allocation logic from LeafQueue to 
 FiCaSchedulerApp
 ---

 Key: YARN-3026
 URL: https://issues.apache.org/jira/browse/YARN-3026
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacityscheduler
Reporter: Wangda Tan
Assignee: Wangda Tan
 Attachments: YARN-3026.1.patch, YARN-3026.2.patch


 Have a discussion with [~vinodkv] and [~jianhe]: 
 In existing Capacity Scheduler, all allocation logics of and under LeafQueue 
 are located in LeafQueue.java in implementation. To make a cleaner scope of 
 LeafQueue, we'd better move some of them to FiCaSchedulerApp.
 Ideal scope of LeafQueue should be: when a LeafQueue receives some resources 
 from ParentQueue (like 15% of cluster resource), and it distributes resources 
 to children apps, and it should be agnostic to internal logic of children 
 apps (like delayed-scheduling, etc.). IAW, LeafQueue shouldn't decide how 
 application allocating container from given resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3026) Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp

2015-01-12 Thread Wangda Tan (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wangda Tan updated YARN-3026:
-
Attachment: YARN-3026.1.patch

Attached ver.1 patch and kick Jenkins.

 Move application-specific container allocation logic from LeafQueue to 
 FiCaSchedulerApp
 ---

 Key: YARN-3026
 URL: https://issues.apache.org/jira/browse/YARN-3026
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacityscheduler
Reporter: Wangda Tan
Assignee: Wangda Tan
 Attachments: YARN-3026.1.patch


 Have a discussion with [~vinodkv] and [~jianhe]: 
 In existing Capacity Scheduler, all allocation logics of and under LeafQueue 
 are located in LeafQueue.java in implementation. To make a cleaner scope of 
 LeafQueue, we'd better move some of them to FiCaSchedulerApp.
 Ideal scope of LeafQueue should be: when a LeafQueue receives some resources 
 from ParentQueue (like 15% of cluster resource), and it distributes resources 
 to children apps, and it should be agnostic to internal logic of children 
 apps (like delayed-scheduling, etc.). IAW, LeafQueue shouldn't decide how 
 application allocating container from given resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)