[
https://issues.apache.org/jira/browse/MAPREDUCE-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767853#action_12767853
]
Hemanth Yamijala commented on MAPREDUCE-1105:
---------------------------------------------
This is looking fine. I have a few minor comments on the trunk patch:
- TaskSchedulingContext.toString has a spurious comment like "//If maxTaskLimit
is less than the capacity"
- Javadoc of areTasksInQueueOverCapacity needs some update. "check if
numofslotsoccupied+numSlotsPerTask is greater maxCapacity" is missing a 'than'.
maximum-capacity is spelt incorrectly as maxium-capacity at a few places.
- areTasksInQueueOverCapacity is returning false if tsi.getNumSlotsOccupied() +
noOfSlotsPerTask is equal to maxCapacity. The problem is if noOfSlotsPerTask is
0, then this means the number of occupied slots is already at maxCapacity and
we will end up scheduling more. This seems a bug. One approach that seems to
work is the call from assignTasks can pass in noOfSlotsPerTask as 1 instead of
0. This way, the API is interpreted as saying if we add 1 more slot to the
current number of slots, will we be over limit.
- In testHighMemoryBlockingWithMaxCapacity, memory for reduce tasks for job1
should be 1024 instead of 1.
- In the test cases related to max capacities, expectedStrings map is not being
cleared. Why not use the convenience API populateExpectedStrings.
> CapacityScheduler: It should be possible to set queue hard-limit beyond it's
> actual capacity
> --------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1105
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1105
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.21.0
> Reporter: Arun C Murthy
> Assignee: rahul k singh
> Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: MAPRED-1105-21-1.patch, MAPRED-1105-21-2.patch,
> MAPREDUCE-1105-version20-2.patch, MAPREDUCE-1105-version20.patch.txt
>
>
> Currently the CS caps a queue's capacity to it's actual capacity if a
> hard-limit is specified to be greater than it's actual capacity. We should
> allow the queue to go upto the hard-limit if specified.
> Also, I propose we change the hard-limit unit to be percentage rather than
> #slots.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
