Protect Job Tracker against memory exhaustion due to very large InputSplit or
JobConf objects
---------------------------------------------------------------------------------------------
Key: MAPREDUCE-841
URL: https://issues.apache.org/jira/browse/MAPREDUCE-841
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Affects Versions: 0.20.1
Reporter: Hong Tang
Fix For: 0.21.0
JobTracker only needs to examine a subset of information contained by
InputSplit or JobConf objects. But currently JobTracker loads the complete
user-defined InputSplit and JobConf objects in memory. This design would leave
JobTracker susceptible to memory exhaustion particularly in cases when some
bugs in user code which could result in very large input splits or job conf
objects (e.g. PIG-901).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
