[
https://issues.apache.org/jira/browse/HADOOP-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623579#action_12623579
]
Hemanth Yamijala commented on HADOOP-3759:
------------------------------------------
This patch addresses all comments from Arun and Owen. Regarding the following
comment from Devaraj:
bq. The other thing is the way we handle -Xmx in this setup. Assume a case
where the user hasn't specified any memory requirement for his job. The memory
that a task would get is proportional to the amount of memory in the TT/#slots.
Let's say for this cluster instance, it is 1G. Now if his -Xmx, which is an
absolute number, is above this, say 1.5G, would it work? Note that the task JVM
might work even with 1G. It is just the user happened to specify it as 1.5G.
As far as I could see, the JVM seems to be allocating the Xmx value in chunks.
So, it might not fail immediately, and might not at all if the task doesn't
require more than 1G. However, if it does come over 1G, it might get killed.
There seem to be two ways to handle this:
- Document this for the user to know and take care of in the job configuration.
We've done a similar thing for {{mapred.child.ulimit}}.
- When scheduling, we specifically look for Xmx value in
{{mapred.child.java.opts}} and if its set to a value higher than
{{ResourceStatus.getDefaultMemoryPerTask()}}, don't schedule the task on this
TT.
The latter option looks a little hacky (having to parse mapred.child.java.opts
etc) and also could behave differently in different conditions making it
difficult to debug. It seems simpler to just document this. On this line, I've
updated both the hadoop-default.xml documentation and the Forrest documentation
in Map/Red tutorial that mentions these options. Let me know if this seems fine.
Also, I've retained {{mapred.child.ulimit}}, since it could be used for
parameters other than memory.
> Provide ability to run memory intensive jobs without affecting other running
> tasks on the nodes
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-3759
> URL: https://issues.apache.org/jira/browse/HADOOP-3759
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.19.0
>
> Attachments: HADOOP-3759.patch, HADOOP-3759.patch, HADOOP-3759.patch,
> HADOOP-3759.patch, HADOOP-3759.patch, HADOOP-3759.patch
>
>
> In HADOOP-3581, we are discussing how to prevent memory intensive tasks from
> affecting Hadoop daemons and other tasks running on a node. A related
> requirement is that users be provided an ability to run jobs which are memory
> intensive. The system must provide enough knobs to allow such jobs to be run
> while still maintaining the requirements of HADOOP-3581.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
