[
https://issues.apache.org/jira/browse/MAPREDUCE-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752074#action_12752074
]
Devaraj Das commented on MAPREDUCE-841:
---------------------------------------
BTW for the splits part, MAPREDUCE-181 (http://tinyurl.com/legzp9) is
introducing some changes.
> Protect Job Tracker against memory exhaustion due to very large InputSplit or
> JobConf objects
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-841
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-841
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Hong Tang
> Fix For: 0.21.0
>
>
> JobTracker only needs to examine a subset of information contained by
> InputSplit or JobConf objects. But currently JobTracker loads the complete
> user-defined InputSplit and JobConf objects in memory. This design would
> leave JobTracker susceptible to memory exhaustion particularly in cases when
> some bugs in user code which could result in very large input splits or job
> conf objects (e.g. PIG-901).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
