[
https://issues.apache.org/jira/browse/MAPREDUCE-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-4730:
----------------------------------
Attachment: MAPREDUCE-4730.patch
Patch to lower the number of map completion events reducers ask for at a time
from 10000 to 500. This is a short-term fix to allow 20x the number of
reducers to run in the same IPC response footprint as before.
Ran a sleep job test with 20000 mappers and 3000 reducers with the fix, and it
was able to complete with a standard AM size (1.5GB slot).
> AM crashes due to OOM while serving up map task completion events
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-4730
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4730
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
> Affects Versions: 0.23.3
> Reporter: Jason Lowe
> Priority: Blocker
> Attachments: MAPREDUCE-4730.patch
>
>
> We're seeing a repeatable OOM crash in the AM for a task with around 30000
> maps and 3000 reducers. Details to follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
