[jira] [Commented] (YARN-6456) Allow administrators to control available container runtimes and set defaults for all containers
[ https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630977#comment-16630977 ] Hudson commented on YARN-6456: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15068 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15068/]) YARN-6456. Added config to set default container runtimes. (eyang: rev b237a0dd44ab285941983648d7ef26b99b30d624) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DefaultLinuxContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/GpuResourceHandlerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java > Allow administrators to control available container runtimes and set defaults > for all containers > > > Key: YARN-6456 > URL: https://issues.apache.org/jira/browse/YARN-6456 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Miklos Szegedi >Assignee: Craig Condit >Priority: Major > Labels: Docker > Fix For: 3.2.0 > > Attachments: YARN-6456-ForceDockerRuntimeIfSupported.patch, > YARN-6456.001.patch, YARN-6456.002.patch, YARN-6456.003.patch, > YARN-6456.004.patch, YARN-6456.005.patch > > > > With LCE, there are multiple ContainerRuntimes available for handling > different types of containers; default, docker, java sandbox. Admins should > have the ability to override the user decision and set a single global > ContainerRuntime to be used for all containers. > Original Description: > {quote}One reason to use Docker containers is to be able to isolate different > workloads, even, if they run as the same user. > I have noticed some issues in the current design: > 1. DockerLinuxContainerRuntime mounts containerLocalDirs > {{nm-local-dir/usercache/user/appcache/application_1491598755372_0011/}} and > userLocalDirs {{nm-local-dir/usercache/user/}}, so that a container can see > and modify the files of another container. I think the application file cache > directory should be enough for the container to run in most of the cases. > 2. The whole cgroups directory is mounted. Would the container directory be > enough? > 3. There is no way to enforce exclusive use of Docker for all containers. > There should be an option that it is not the user but the admin that requires > to use Docker. > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6456) Allow administrators to control available container runtimes and set defaults for all containers
[ https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630930#comment-16630930 ] Eric Yang commented on YARN-6456: - +1 looks good to me for patch 005, will commit shortly. > Allow administrators to control available container runtimes and set defaults > for all containers > > > Key: YARN-6456 > URL: https://issues.apache.org/jira/browse/YARN-6456 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Miklos Szegedi >Assignee: Craig Condit >Priority: Major > Labels: Docker > Attachments: YARN-6456-ForceDockerRuntimeIfSupported.patch, > YARN-6456.001.patch, YARN-6456.002.patch, YARN-6456.003.patch, > YARN-6456.004.patch, YARN-6456.005.patch > > > > With LCE, there are multiple ContainerRuntimes available for handling > different types of containers; default, docker, java sandbox. Admins should > have the ability to override the user decision and set a single global > ContainerRuntime to be used for all containers. > Original Description: > {quote}One reason to use Docker containers is to be able to isolate different > workloads, even, if they run as the same user. > I have noticed some issues in the current design: > 1. DockerLinuxContainerRuntime mounts containerLocalDirs > {{nm-local-dir/usercache/user/appcache/application_1491598755372_0011/}} and > userLocalDirs {{nm-local-dir/usercache/user/}}, so that a container can see > and modify the files of another container. I think the application file cache > directory should be enough for the container to run in most of the cases. > 2. The whole cgroups directory is mounted. Would the container directory be > enough? > 3. There is no way to enforce exclusive use of Docker for all containers. > There should be an option that it is not the user but the admin that requires > to use Docker. > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6456) Allow administrators to control available container runtimes and set defaults for all containers
[ https://issues.apache.org/jira/browse/YARN-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630892#comment-16630892 ] Craig Condit commented on YARN-6456: [~eyang], title updated. > Allow administrators to control available container runtimes and set defaults > for all containers > > > Key: YARN-6456 > URL: https://issues.apache.org/jira/browse/YARN-6456 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Miklos Szegedi >Assignee: Craig Condit >Priority: Major > Labels: Docker > Attachments: YARN-6456-ForceDockerRuntimeIfSupported.patch, > YARN-6456.001.patch, YARN-6456.002.patch, YARN-6456.003.patch, > YARN-6456.004.patch, YARN-6456.005.patch > > > > With LCE, there are multiple ContainerRuntimes available for handling > different types of containers; default, docker, java sandbox. Admins should > have the ability to override the user decision and set a single global > ContainerRuntime to be used for all containers. > Original Description: > {quote}One reason to use Docker containers is to be able to isolate different > workloads, even, if they run as the same user. > I have noticed some issues in the current design: > 1. DockerLinuxContainerRuntime mounts containerLocalDirs > {{nm-local-dir/usercache/user/appcache/application_1491598755372_0011/}} and > userLocalDirs {{nm-local-dir/usercache/user/}}, so that a container can see > and modify the files of another container. I think the application file cache > directory should be enough for the container to run in most of the cases. > 2. The whole cgroups directory is mounted. Would the container directory be > enough? > 3. There is no way to enforce exclusive use of Docker for all containers. > There should be an option that it is not the user but the admin that requires > to use Docker. > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org