[
https://issues.apache.org/jira/browse/YARN-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137903#comment-14137903
]
Varun Vasudev commented on YARN-2190:
-------------------------------------
[~chuanliu] thanks for the patch! Some questions and comments -
1. What is the behaviour of a process that tries to exceed the allocated
memory? Will it start swapping or will it be killed?
2. Your code assumes a 1-1 mapping of physical cores to vcores. This assumption
is/will be problematic, especially in heterogeneous clusters. You're better off
using the ratio of (container-vcores/node-vcores) to determine cpu limits.
3.
{noformat}
Index:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
===================================================================
---
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
(revision 1618292)
+++
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
(working copy)
@@ -38,6 +38,7 @@
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.yarn.api.records.ContainerId;
+import org.apache.hadoop.yarn.api.records.Resource;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
import
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container;
import
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerDiagnosticsUpdateEvent;
@@ -257,6 +258,11 @@
readLock.unlock();
}
}
+
+ protected String[] getRunCommand(String command, String groupId,
+ Configuration conf) {
+ return getRunCommand(command, groupId, conf, null);
+ }
Index:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
===================================================================
---
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
(revision 1618292)
+++
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
(working copy)
@@ -185,7 +185,7 @@
// Setup command to run
String[] command = getRunCommand(sb.getWrapperScriptPath().toString(),
- containerIdStr, this.getConf());
+ containerIdStr, this.getConf(), container.getResource());
LOG.info("launchContainer: " + Arrays.toString(command));
{noformat}
Can you explain why you are modifying DefaultContainerExecutor? You've added a
method for the old signature in ContainerExecutor.
4. Can you modify the comments/usage to specify the units of memory(bytes, MB,
GB)?
> Provide a Windows container executor that can limit memory and CPU
> ------------------------------------------------------------------
>
> Key: YARN-2190
> URL: https://issues.apache.org/jira/browse/YARN-2190
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Reporter: Chuan Liu
> Assignee: Chuan Liu
> Attachments: YARN-2190-prototype.patch, YARN-2190.1.patch,
> YARN-2190.2.patch, YARN-2190.3.patch, YARN-2190.4.patch, YARN-2190.5.patch
>
>
> Yarn default container executor on Windows does not set the resource limit on
> the containers currently. The memory limit is enforced by a separate
> monitoring thread. The container implementation on Windows uses Job Object
> right now. The latest Windows (8 or later) API allows CPU and memory limits
> on the job objects. We want to create a Windows container executor that sets
> the limits on job objects thus provides resource enforcement at OS level.
> http://msdn.microsoft.com/en-us/library/windows/desktop/ms686216(v=vs.85).aspx
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
