[
https://issues.apache.org/jira/browse/MAPREDUCE-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284425#comment-14284425
]
Gera Shegalov commented on MAPREDUCE-5785:
------------------------------------------
Hi [~kasha], thanks for updating the patch.
nit: v8 patch introduced a line break in the class declaration
{code}
public class
TestMapReduceChildJVM {
{code}
The config convention/implementation equates an empty value in xml to the unset
value. So I think your v7 was fine in this respect.
If it looks too hacky to you, we can set memory.mb keys to {{-1}}, the default
you used in v8 or maybe more natural {{0}}. Then we can say {{memory.mb <= 0}}
implies {{not set}}.
I prefer fail-fast validation, so I am not in favor of catching
NumberFormatException. At least not in this patch. Otherwise, it's not clear
why are we not doing this for all numeric parameters.
> Derive heap size or mapreduce.*.memory.mb automatically
> -------------------------------------------------------
>
> Key: MAPREDUCE-5785
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5785
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mr-am, task
> Reporter: Gera Shegalov
> Assignee: Karthik Kambatla
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-5785.v01.patch, MAPREDUCE-5785.v02.patch,
> MAPREDUCE-5785.v03.patch, mr-5785-4.patch, mr-5785-5.patch, mr-5785-6.patch,
> mr-5785-7.patch, mr-5785-8.patch
>
>
> Currently users have to set 2 memory-related configs per Job / per task type.
> One first chooses some container size map reduce.\*.memory.mb and then a
> corresponding maximum Java heap size Xmx < map reduce.\*.memory.mb. This
> makes sure that the JVM's C-heap (native memory + Java heap) does not exceed
> this mapreduce.*.memory.mb. If one forgets to tune Xmx, MR-AM might be
> - allocating big containers whereas the JVM will only use the default
> -Xmx200m.
> - allocating small containers that will OOM because Xmx is too high.
> With this JIRA, we propose to set Xmx automatically based on an empirical
> ratio that can be adjusted. Xmx is not changed automatically if provided by
> the user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
