Jason Lowe created MAPREDUCE-5543:
-------------------------------------
Summary: In-memory map outputs can be leaked after shuffle
completes in 0.23
Key: MAPREDUCE-5543
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5543
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.1.0-beta, 0.23.9
Reporter: Jason Lowe
Assignee: Jason Lowe
Priority: Blocker
Fix For: 2.1.1-beta
MergeManagerImpl#close adds the contents of inMemoryMergedMapOutputs and
inMemoryMapOutputs to a list of map outputs that is subsequently processed, but
it does not clear those sets. This prevents some of the map outputs from being
garbage collected and significantly reduces the memory available for the
subsequent reduce phase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
