[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401522#comment-15401522 ] Zhankun Tang commented on YARN-5360: [~sidharta-s], Thanks for the explaination. And I appreciate that if you can share the spark docker image. I would like to have a try. This JIRA is originally not only just dropping --user but also searching for a flexible interface different with YARN-4266 to decouple host user and Docker container user. The *main difference* is that this JIRA would like to expose this --user to application while YARN-4266 utilize whitelisted user as admin configuration to drop --user. But both changes would have several same implications like log aggregation, etc. Since it is not recommended to change existing use case or expose --user to application, I think we can move to YARN-4266 to discuss more details. > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398593#comment-15398593 ] Sidharta Seethana commented on YARN-5360: - To clarify a few things here - yes, customized images are required in some cases (especially in secure mode) to make apps work for certain users. This is a limitation we have to work with for the moment given the hadoop security model - it may not be reasonable or practical to drop it altogether except under controlled situations. Also, log aggregation does not work in secure mode if you drop "--user" (it works in non-secure mode, I think but I'll have to check the code/test again). Artifact deletion will not work if the artifacts are created as a different user in the docker container (artifact cleanup is done as the 'run as' user). In your first table above, the yarn/nobody case likely did not work because the sequenceiq image is based on centos (nobody uid=99) and the system you were testing on was not centos (ubuntu? nobody uid=65534). We have tested spark with other images on centos (I have test images on docker hub if you'd like to try). I am pretty sure [~templedf] has successfully run spark using the current implementation as well. ([~templedf] : please confirm) . If this the discussion here is now only about dropping the "--user" in certain cases, this is captured in YARN-4266. > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398563#comment-15398563 ] Zhankun Tang commented on YARN-5360: [~sidharta-s], The first testing result table shows us existing Docker image won't work if no corret user/uid created in it. What I cannot comprehend here is that you said "dropping --user" breaks existing use case. How did the users adopt LCE Docker? Added same user/UID in Docker image to work through it? Yes. Running Docker user won't approve the whole thing. And I agree with you that we shouldn't have different default behaviors. We should find a better solution. > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398111#comment-15398111 ] Sidharta Seethana commented on YARN-5360: - [~tangzhankun], dropping the '--user' works in this because you are running the container processes as root - once the processes are run root, the restrictive permissions on launch_container.sh don't matter. An image that is setup to use a different arbitrary user will not work. To reiterate, I don't believe it makes sense to have different default behavior for secure/non-secure modes - we should find a model that works across both. > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397027#comment-15397027 ] Zhankun Tang commented on YARN-5360: [~sidharta-s], thanks for pointing out this. Actually, different default behaviors seems awkward for me too. But dropping --user in non-secure mode seems no big issue breaking spark/MR per my testing. Could you please explain this a little more? I did some testings about the impacts dropping "--user" in non-secure mode bring to current MR/Dockerized MR and spark (I don't find any official document of Dockerized spark, please direct me to it if you know) job. Below are my testing results: ||non-secure mode LCE With "--user"|| |Cluster Environment:|Hadoop branch-2.8| |LCE Modifications:|*None*| |Testing tool:| "Pi" application for MR and "SparkPi" for spark| |Testing result: ||yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user||yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users||submitting user||run job user||"Pi" MR without Docker||Pi" MR with Docker image "sequenceiq/hadoop-docker:latest"||"SparkPi" without Docker||| | |yarn|(default)|yarn|yarn|(/)|(x) "Unable to find user yarn, exit code 7"|(/)| | |yarn|(default)|test|yarn|(/)|(x) "Unable to find user yarn, exit code 7"|(/)| | |yarn|false|test|test|(/)|(x) "Unable to find user test, exit code 7"|(/)| | |(default)|(default)|yarn|nobody|(/)|(x) "launch_container.sh: Permission denied. Unable to read from docker logs(ferror, feof): 0 1, exit code 126"|(/)| ||non-secure mode LCE Without "--user"|| |Cluster Environment:|Hadoop branch-2.8| |LCE Modifications:|*Only remove the "--user" from mandatory options* in constructor method of DockerRunCommand.java (line 37)| |Testing tool:| "Pi" application for MR and "SparkPi" for spark| |Testing result: ||yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user||yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users||submitting user||expected run job user||"Pi" MR without Docker||Pi" MR with Docker image "sequenceiq/hadoop-docker:latest"||"SparkPi" without Docker||| | |yarn|(default)|yarn|yarn|(/)|(/) (run as root)|(/)| | |yarn|(default)|test|yarn|(/)|(/) (run as root)|(/)| | |yarn|false|test|test|(/)|(/) (run as root)|(/)| | |(default)|(default)|yarn|nobody|(/)|(/) (run as root)|(/)| > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail:
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[
https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392734#comment-15392734
]
Sidharta Seethana commented on YARN-5360:
-
{quote}
For under what conditions to drop the "--user", I prefer that in non-secure
mode, we drop it by default and leave the chance of setting it to the
application thru interface (default yarn container launch is not affected,
still run the container based on configuration). But in secure mode,
application is not allowed to run Docker container as arbitrary user.
{quote}
I am afraid I disagree with this. Having such completely different default
behavior in secure vs non-secure mode can lead to a lot of confusion and break
existing supported use cases. Dropping --user by default, for example would
break spark/map-reduce in non-secure mode.
> Decouple host user and Docker container user
>
>
> Key: YARN-5360
> URL: https://issues.apache.org/jira/browse/YARN-5360
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Zhankun Tang
>Assignee: Zhankun Tang
>
> There is *a dependency between job submitting user and the user in the Docker
> image* in LCE currently. For instance, in order to run the Docker container
> as yarn user, we can choose set the
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn
> and leave
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users"
> default (true). Then LCE will choose yarn ( UID maybe 1001) as the user
> running jobs.
> LCE will mount the generated launch_container.sh (owned by the running job
> user) and /etc/passwd (*current the code is mounting to container's
> /etc/password, I think it's a mistake*) into the Docker container and
> utilizes "docker run --user=" option to get it done internally.
> Mounting /etc/passwd to the container is a not good choice due to override
> original users defined in Docker image. As far as I know, since Docker v1.8
> (or maybe earlier), the Docker run command "--user=" option accepts UID and
> *when passing UID, the user does not have to exist in the container*. So we
> could use UID instead of user name to construct the Docker run command to
> eliminate the dependency that create the same user in the Docker image. This
> enables LCE the ability to launch any Docker container safely regardless what
> users in it.
> But this is not enough to decouple host user and Docker container user. The
> final solution we are searching for are focused on allowing users to run
> their Docker images flexibly without involving dependencies of YARN and make
> sure the container won't bring in security risk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391563#comment-15391563 ] Zhankun Tang commented on YARN-5360: [~vvasudev], Thanks but I think set to 705 won't affect current LCE behavior, just provide the possiblity of dropping "--user". Please correct me If I missed something important. For under what conditions to drop the "--user", I prefer that in non-secure mode, we drop it by default and leave the chance of setting it to the application thru interface (default yarn container launch is not affected, still run the container based on configuration). But in secure mode, application is not allowed to run Docker container as arbitrary user. For the Docker container root user potential insecure issue, we can drop the risky capabilities of "mount operation", "change file owner/attributes" and so on by default to make sure that applications won't interfere with each other. If admin want to enable certain capabilities in configuration, then he's responsible for ensure security. > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391408#comment-15391408 ] Varun Vasudev commented on YARN-5360: - [~tangzhankun] - changing launch_container.sh permissions to 705 is not an option(even in non-secure mode). Lots of users run LinuxContainerExecutor in non-secure mode but with containers being launched as the user that submitted the job. The right solution is to figure out under what conditions is it appropriate to drop the "--user" option, which would allow user to bring their own Docker images, and how to handle the associated problems with that solution(such as local directory permissions, log aggregation, etc). Like [~sidharta-s] mentioned above running containers as root exposes a security risk that YARN didn't expose earlier. While it's possible to mitigate this risk, we need to make sure existing users aren't exposed to it inadvertently. > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391336#comment-15391336 ] Zhankun Tang commented on YARN-5360: [~sidharta-s], although running a container as root have security implications by default, we can still use capabilities(already enabled in YARN-4258), selinux and other linux kernel features to ensure the root user in container has much less privileges than the real “root” in host. So for me, it is not unsafe. But as you mentioned, YARN's own complexity here prevents us from simply removing "--user" option. Then let's focus on resolving these barriers which enforces it and provide more flexible interface of what user to run the container. For discussion, I list them as below. We need more flexibility and we have these barriers to cross: * 1. Mounting /etc/passwd to container ** We should provide a way to toggle it since it override the original users defined in Docker image. It shouldn't be mounted by default. ButI think this already can be done by application thru YARN-4595. What we need to do is just remove /etc/passwd bind-mount after this JIRA closed. * 2. Exclusive permissions of local host files/directories mounted into Docker container, especially "launch_container.sh" ** One simple way to solve this is just change the related permissions. For instance, if unsecure mode, set 705 to launch_container.sh to allow other group user run it and set 707 to log dirs to allow others to write log. If secure mode, just set permissions as it is now. * 3. Side effects that if we change the running user, things like log aggregation would fail ** One simple way is that modify the YARN expects. For instance, don't expect log permission same with submitting user if unsecure mode but check it when secure mode. And solving above items shouldn’t affect the secure mode requirement on user. Behaviors in secure mode are just same as it is now. After solving these barriers, we can provide application the interface in unsecure mode: * an environment variable YARN_CONTAINER_RUNTIME_DOCKER_USER for application to specify the running user in unsecure mode Thoughts? > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384786#comment-15384786 ] Sidharta Seethana commented on YARN-5360: - [~templedf], running a container as root does in fact have security implications (there are other things to consider in conjunction with this - capabilites, selinux and so on). There are (at least) a couple of reasons why --user is enforced currently : 1) YARN security model requires the launched process run as the designated user 2) Log aggregation/local permissions etc - some of these things would stop working if the generated logs have ownership that is different from what YARN expects. These are also the reasons that need to be considered for YARN-4266 > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[
https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384777#comment-15384777
]
Sidharta Seethana commented on YARN-5360:
-
[~zyluo],
{quote}
I think this is inconsistent with Docker's motto to "build, ship and run".
There is no point of using Docker if the user has to use every image as a base
to add the correct user.
{quote}
While that may be Docker's motto - the objective of YARN-3611, in my opinion
has never been to use docker for docker's sake - we needed to adapt it to the
YARN/hadoop world - hadoop security, log aggregation, localization - all of
these need to work.
> Decouple host user and Docker container user
>
>
> Key: YARN-5360
> URL: https://issues.apache.org/jira/browse/YARN-5360
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Zhankun Tang
>Assignee: Zhankun Tang
>
> There is *a dependency between job submitting user and the user in the Docker
> image* in LCE currently. For instance, in order to run the Docker container
> as yarn user, we can choose set the
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn
> and leave
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users"
> default (true). Then LCE will choose yarn ( UID maybe 1001) as the user
> running jobs.
> LCE will mount the generated launch_container.sh (owned by the running job
> user) and /etc/passwd (*current the code is mounting to container's
> /etc/password, I think it's a mistake*) into the Docker container and
> utilizes "docker run --user=" option to get it done internally.
> Mounting /etc/passwd to the container is a not good choice due to override
> original users defined in Docker image. As far as I know, since Docker v1.8
> (or maybe earlier), the Docker run command "--user=" option accepts UID and
> *when passing UID, the user does not have to exist in the container*. So we
> could use UID instead of user name to construct the Docker run command to
> eliminate the dependency that create the same user in the Docker image. This
> enables LCE the ability to launch any Docker container safely regardless what
> users in it.
> But this is not enough to decouple host user and Docker container user. The
> final solution we are searching for are focused on allowing users to run
> their Docker images flexibly without involving dependencies of YARN and make
> sure the container won't bring in security risk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383322#comment-15383322 ] Daniel Templeton commented on YARN-5360: I agree that UID isn't going to make things better. Usernames will be more portable. Running the container as root is not typically a security concern. That's the magic promise of containers. Dropping the --user altogether is a partial solution, though. I may want my container to run as a user that is unknown to YARN and not specified in the Dockerfile. For that reason, allowing users control over the --user option would be desirable. The main issue is that the launch_container.sh script is owned by the job owner with limited access. One common way around this issue is to pipe the contents of the file to "docker run ... /bin/bash -c". That will work for the script, but it might require some extra cleverness to use that approach to also bring over the tokens file. Another way to get around the access issues is running the container as root, thought that's a little limiting. If we can't work out a way to allow the script to execute while leaving --user configurable, the next best option would be YARN-4266 and requiring that the user be specified in the Dockerfile. [~sidharta-s], what was the original reason that --user was forced for all users? > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > Mounting /etc/passwd to the container is a not good choice due to override > original users defined in Docker image. As far as I know, since Docker v1.8 > (or maybe earlier), the Docker run command "--user=" option accepts UID and > *when passing UID, the user does not have to exist in the container*. So we > could use UID instead of user name to construct the Docker run command to > eliminate the dependency that create the same user in the Docker image. This > enables LCE the ability to launch any Docker container safely regardless what > users in it. > But this is not enough to decouple host user and Docker container user. The > final solution we are searching for are focused on allowing users to run > their Docker images flexibly without involving dependencies of YARN and make > sure the container won't bring in security risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5360) Decouple host user and Docker container user
[ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381650#comment-15381650 ] Zhankun Tang commented on YARN-5360: Change the title from "use UID .." to "Decouple host user and Docker container user" > Decouple host user and Docker container user > > > Key: YARN-5360 > URL: https://issues.apache.org/jira/browse/YARN-5360 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Zhankun Tang >Assignee: Zhankun Tang > > There is *a dependency between job submitting user and the user in the Docker > image* in LCE currently. For instance, in order to run the Docker container > as yarn user, we can choose set the > "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn > and leave > "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" > default (true). Then LCE will choose yarn ( UID maybe 1001) as the user > running jobs. > LCE will mount the generated launch_container.sh (owned by the running job > user) and /etc/passwd (*current the code is mounting to container's > /etc/password, I think it's a mistake*) into the Docker container and > utilizes "docker run --user=" option to get it done internally. > But I don't think mounting /etc/passwd to the container is a good choice. As > far as I know, since Docker v1.8 (or maybe earlier), the Docker run command > "--user=" option accepts UID and *when passing UID, the user does not have to > exist in the container*. So we should use UID instead of user name to > construct the Docker run command to eliminate the dependency that create the > same user in the Docker image. This enables LCE the ability to launch any > Docker container safely regardless what users in it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
