[ https://issues.apache.org/jira/browse/YARN-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541069#comment-14541069 ]
Vinod Kumar Vavilapalli commented on YARN-3611: ----------------------------------------------- Just looked at this and YARN-3291 JIRA. The main direction of both of these JIRAs as I understand is to reuse code from (a) the container-executor binary and (b) the java LinuxContainerExecutor code; so as to setup things correctly. This was one of my comments in the original docker integration JIRAs itself. May be it is just the JIRA title that is throwing us off a bit. Both JIRAs are advocating similar idea though. Can we establish a clear relationship between the two? [~sidharta-s] and [~ashahab]. > Support Docker Containers In LinuxContainerExecutor > --------------------------------------------------- > > Key: YARN-3611 > URL: https://issues.apache.org/jira/browse/YARN-3611 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn > Reporter: Sidharta Seethana > Assignee: Sidharta Seethana > > Support Docker Containers In LinuxContainerExecutor > LinuxContainerExecutor provides useful functionality today with respect to > localization, cgroups based resource management and isolation for CPU, > network, disk etc. as well as security with a well-defined mechanism to > execute privileged operations using the container-executor utility. Bringing > docker support to LinuxContainerExecutor lets us use all of this > functionality when running docker containers under YARN, while not requiring > users and admins to configure and use a different ContainerExecutor. > There are several aspects here that need to be worked through : > * Mechanism(s) to let clients request docker-specific functionality - we > could initially implement this via environment variables without impacting > the client API. > * Security - both docker daemon as well as application > * Docker image localization > * Running a docker container via container-executor as a specified user > * “Isolate” the docker container in terms of CPU/network/disk/etc > * Communicating with and/or signaling the running container (ensure correct > pid handling) > * Figure out workarounds for certain performance-sensitive scenarios like > HDFS short-circuit reads > * All of these need to be achieved without changing the current behavior of > LinuxContainerExecutor -- This message was sent by Atlassian JIRA (v6.3.4#6332)