[
https://issues.apache.org/jira/browse/MAPREDUCE-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-5785:
-------------------------------------
Description:
Currently users have to set 2 memory-related configs for per Job / per task
type. One fist choses some container size mapreduce.*.memory.mb and then a
corresponding Xmx < mapreduce.*.memory.mb to make sure that the JVM with the
user code heap, and its native memory do not exceed this limit. If one forgets
to tune Xmx, MR-AM might be allocating big containers whereas the JVM will only
use the default -Xmx200m.
With this JIRA, we propose to set Xmx automatically base on an empirical ratio
that can be adjusted. Xmx is not changed automaically if provided by the user.
was:
Currently users have to set 2 memory-related configs for per Job / per task
type. One fist choses some container size mapreduce.*.memory.mb and then a
corresponding Xmx < mapreduce.*.memory.mb to make sure that the JVM with the
user code heap, and its native memory do not exceed this limit. If one forgets
to tune Xmx, MR-AM might be allocating big containers whereas the JVM will only
use the default -Xmx200m.
With this JIRA, we propose to set Xmx automatically base on an empirical ratio
that can be adjusted. Xmx is not changed if provided by the user.
> Derive task attempt JVM max heap size automatically from mapreduce.*.memory.mb
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5785
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5785
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mr-am, task
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
>
> Currently users have to set 2 memory-related configs for per Job / per task
> type. One fist choses some container size mapreduce.*.memory.mb and then a
> corresponding Xmx < mapreduce.*.memory.mb to make sure that the JVM with the
> user code heap, and its native memory do not exceed this limit. If one
> forgets to tune Xmx, MR-AM might be allocating big containers whereas the JVM
> will only use the default -Xmx200m.
> With this JIRA, we propose to set Xmx automatically base on an empirical
> ratio that can be adjusted. Xmx is not changed automaically if provided by
> the user.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
