[
https://issues.apache.org/jira/browse/MAPREDUCE-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745827#action_12745827
]
Hemanth Yamijala commented on MAPREDUCE-824:
--------------------------------------------
Looked at the test cases:
- Code seems duplicated between CapacitySchedulerUtils and
CapacityTaskScheduler and TestContainerQueue.
- In some test cases, when we create a queue, it is already adding a child to
the parent. So, why do we need additional calls to addChildren ?
- What's the difference between testConfiguredCapacity and testMinCapacity ?
- The test cases testing scheduling are nice. The comments are out of sync a
bit, and will be hard to maintain. Instead I suggest that we assert what we are
documenting in the tests itself, so that they themselves read as comments, and
will also always be in sync.
- As discussed, getCapacity() should not return max capacity any time. It
should always return the current capacity or limit, whichever is smaller.
Otherwise, the sort order of queues would be affected.
- areTasksInQueueOverLimit should be changed to something along these lines:
{code}
if (tsi.getMaxTaskLimit() > 0) {
if (tsi.getNumSlotsOccupied() >= tsi.getCapacity()) {
return true;
}
}
if (tsi.getMaxCapacity() > 0) {
if (tsi.getNumSlotsOccupied() >= tsi.getMaxCapacity()) {
return true;
}
}
return false;
{code}
- At the same time, testMaxCapacity should be removed. I would instead
recommend a test case that sets a max capacity on a queue, and checks
scheduling honors the decision.
> Support a hierarchy of queues in the capacity scheduler
> -------------------------------------------------------
>
> Key: MAPREDUCE-824
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-824
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Reporter: Hemanth Yamijala
> Attachments: HADOOP-824-1.patch, HADOOP-824-2.patch,
> HADOOP-824-3.patch
>
>
> Currently in Capacity Scheduler, cluster capacity is divided among the queues
> based on the queue capacity. These queues typically represent an organization
> and the capacity of the queue represents the capacity the organization is
> entitled to. Most organizations are large and need to divide their capacity
> among sub-organizations they have. Or they may want to divide the capacity
> based on a category or type of jobs they run. This JIRA covers the
> requirements and other details to provide the above feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
