[
https://issues.apache.org/jira/browse/MAPREDUCE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768363#comment-13768363
]
wanbin commented on MAPREDUCE-5510:
-----------------------------------
private void preemptReducesIfNeeded() {
if (reduceResourceReqt == 0) {
return; //no reduces
}
//check if reduces have taken over the whole cluster and there are
//unassigned maps
if (scheduledRequests.maps.size() > 0) {
int memLimit = getMemLimit();
int availableMemForMap = memLimit - ((assignedRequests.reduces.size() -
assignedRequests.preemptionWaitingReduces.size()) *
reduceResourceReqt);
//availableMemForMap must be sufficient to run atleast 1 map
if (availableMemForMap < mapResourceReqt) {
I consider availableMemForMap should get rid of
scheduledRequests.maps.size()*mapResourceReqt
> Root queue is full, leads to a job hung, it's reduces started,but some maps
> is pending
> ---------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5510
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5510
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 2.0.6-alpha
> Reporter: wanbin
>
> when it hadppen, I notice reduces were not preempt
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated
> containers 1
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned to reduce
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned
> container container_1377833725640_158209_01_004100 to
> attempt_1377833725640_158209_r_000258_0
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating
> schedule, headroom=582656
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator:
> completedMapPercent 0.9940104 totalMemLimit:2548736 finalMapMemLimit:70656
> finalReduceMemLimit:2478080 netScheduledMapMem:70656
> netScheduledReduceMem:2460672
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping up 1
> 2013-09-15 10:32:26,608 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling:
> PendingReds:212 ScheduledMaps:22 ScheduledReds:55 AssignedMaps:1
> AssignedReds:213 CompletedMaps:3817 CompletedReds:0 ContAlloc:4031 ContRel:0
> HostLocal:0 RackLocal:0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
