Allen Wittenauer commented on YARN-1964:

I took a look at the patch.  After some discussion/clarification from Abin and 
Dinesh, here are some issues we've identified with the current patch:

* yarn-site.xml names are not in the yarn.* hierarchy.

* The environment variables being imported are likely incomplete 
(HADOOP_PREFIX, at a minimum, should be added to the list).  

* Application Masters cannot currently be run from DockerContainerExecutor.  
The current patch launches them with the default launcher.  

* There isn't any special distributed cache handling, which could/will expose 
private data to everyone as well as risk confusing the node manager if the 
distributed cache contents are changed out from underneath it.  

* Same thing with logging.

* This hasn't been tested with any level of security (either 'run as the user' 
or fully kerberized).  There are likely more holes/problems that will be 
discovered after those tests.

* There is a question as to how to handle importing Java and the Hadoop setup 
into the container. There is a risk that just adding HADOOP_PREFIX and marking 
it as RO may break things in certain configurations (specifically logs, tmp, 
and pids dirs).

> Create Docker analog of the LinuxContainerExecutor in YARN
> ----------------------------------------------------------
>                 Key: YARN-1964
>                 URL: https://issues.apache.org/jira/browse/YARN-1964
>             Project: Hadoop YARN
>          Issue Type: New Feature
>    Affects Versions: 2.2.0
>            Reporter: Arun C Murthy
>            Assignee: Abin Shahab
>         Attachments: yarn-1964-branch-2.2.0-docker.patch, 
> yarn-1964-branch-2.2.0-docker.patch, yarn-1964-docker.patch, 
> yarn-1964-docker.patch, yarn-1964-docker.patch, yarn-1964-docker.patch, 
> yarn-1964-docker.patch
> Docker (https://www.docker.io/) is, increasingly, a very popular container 
> technology.
> In context of YARN, the support for Docker will provide a very elegant 
> solution to allow applications to *package* their software into a Docker 
> container (entire Linux file system incl. custom versions of perl, python 
> etc.) and use it as a blueprint to launch all their YARN containers with 
> requisite software environment. This provides both consistency (all YARN 
> containers will have the same software environment) and isolation (no 
> interference with whatever is installed on the physical machine).

This message was sent by Atlassian JIRA

Reply via email to