Abin Shahab created YARN-2466:
---------------------------------
Summary: Umbrella issue for Yarn launched Docker Containers
Key: YARN-2466
URL: https://issues.apache.org/jira/browse/YARN-2466
Project: Hadoop YARN
Issue Type: New Feature
Affects Versions: 2.4.1
Reporter: Abin Shahab
Assignee: Abin Shahab
Docker (https://www.docker.io/) is, increasingly, a very popular container
technology.
In context of YARN, the support for Docker will provide a very elegant solution
to allow applications to package their software into a Docker container (entire
Linux file system incl. custom versions of perl, python etc.) and use it as a
blueprint to launch all their YARN containers with requisite software
environment. This provides both consistency (all YARN containers will have the
same software environment) and isolation (no interference with whatever is
installed on the physical machine).
In addition to software isolation mentioned above, Docker containers will
provide resource, network, and user-namespace isolation.
Docker provides resource isolation through cgroups, similar to
LinuxContainerExecutor. This prevents one job from taking other jobs
resource(memory and CPU) on the same hadoop cluster.
User-namespace isolation will ensure that the root on the container is mapped
an unprivileged user on the host. This is currently being added to Docker.
Network isolation will ensure that one user’s network traffic is completely
isolated from another user’s network traffic.
Last but not the least, the interaction of Docker and Kerberos will have to be
worked out. These Docker containers must work in a secure hadoop environment.
Additional details are here:
https://wiki.apache.org/hadoop/dineshs/IsolatingYarnAppsInDockerContainers
--
This message was sent by Atlassian JIRA
(v6.2#6252)
