[
https://issues.apache.org/jira/browse/HADOOP-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564235#action_12564235
]
Doug Cutting commented on HADOOP-2751:
--------------------------------------
> I propose we bump up the child-vm default heapsize to 512M; too many people
> are getting burnt by 200M.
Why the rush? This is for trunk, not an urgent fix to a blocker, where a
short-term fix might be appropriate. If io.sort.mb is using more than the
100MB that's allocated to it by default, then we should fix that there, not
just by bumping the heap size.
Amazon EC2 is a recommended environment for Hadoop. It's standard instance has
1.7GB. Hadoop, by default, will run 2 map tasks and 2 reduce tasks at a time
per node, in addition to probably running a datanode and a tasktracker. That's
six JVMs. At 200MB each, these would take 1.2GB. But a 200MB heap requires
more than 200MB, so that sounds about right for a 1.7GB machine. Upping this
to 512MB would mean that the default configuration would require as much as 3GB.
The defaults should aim for the low side of average, so that things work
out-of-the-box for most folks without alteration. Only folks with really
outdated, underpowered hardware should need to change defaults to get things to
run out of the box, no?
> Increase map/reduce child tasks' heapsize from current default of 200M to 512M
> ------------------------------------------------------------------------------
>
> Key: HADOOP-2751
> URL: https://issues.apache.org/jira/browse/HADOOP-2751
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.16.0
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 0.17.0
>
>
> I guess we should look to check why we get OOMs with 200M, I'd suspect
> io.sort.mb hogs a lot of the default 200M. However, HADOOP-1867 should be the
> right way to solve it.
> For now, I propose we bump up the child-vm default heapsize to 512M; too many
> people are getting burnt by 200M.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
