[
https://issues.apache.org/jira/browse/MAPREDUCE-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen resolved MAPREDUCE-6131.
-----------------------------------
Resolution: Invalid
> Integer overflow in RMContainerAllocator results in starvation of applications
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6131
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Kamal Kc
> Attachments: MAPREDUCE-6131-2.2.0.patch
>
>
> When processing large datasets, Hadoop encounters a scenario where all
> containers run reduce tasks and no map tasks are scheduled. The
> application does not fail but rather remains in this state without making
> any forward progress. It then has to be manually terminated.
> This bug is due to integer overflow in scheduleReduces() of
> RMContainerAllocator. The variable netScheduledMapMem overflows for
> large data sizes, takes negative value, and results in a large
> finalReduceMemLimit and a large rampup value. In almost all cases, this
> large rampup value is greater than the total number of reduce tasks.
> Therefore, the AM tries to assign all reduce tasks. And if the total number
> of reduce tasks is greater than the total container slots, then all slots are
> taken up by reduce tasks, leaving none for maps.
> With 128MB block size and 2GB map container size, overflow occurs with 128 TB
> data size. An example scenario for the reproduction is:
> - Input data size of 32TB, block size 128MB, Map container size = 10GB,
> reduce container size = 10GB, #reducers = 50, cluster mem capacity = 7 x
> 40GB, slowstart=0.0
> Better resolution might be to change the variables used in
> RMContainerAllocator from int to long. A simpler fix instead would be to
> only change the local variables of scheduleReduces() to long data types.
> Patch is attached for 2.2.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
