Zhankun Tang commented on YARN-4266:

Thanks for the review and poiting out the GID thing. [~templedf].
I have done testings based on quick implementations for the two ways of option 
3.2. Based on testings, a generic launch_container.sh serving all scenarios 
seems not possible. So leaving launch_container.sh and modify it only in LCE 
when white-listed user is preferred. This ensure that current use cases won't 
be affected. The only known limitation of this solution is that we need the 
Docker image to package "sudo" command in it because we leverage it to switch 
user in container at run-time.

But as you mentioned, there are still more concerns for my preferred solution:
* *What if the new UID/GID is already in use in the container?*
     The usermod and groupmod commands both has an "-o" option that allow using 
duplicate (non-unique) UID/GID. So we are able to change the UID and GUID to 
what we want without failure.
* *How can we run a container without modification?*
    The answer depends on the container application type(One thing to note is 
that if we allow a container run as root for white-listed users, we don't need 
to modify container. This needs discussion too but I guess it's hard to get 
     ** For applications in container that have dependencies on local host, 
like MRv2 application: 
     The applications would almost certain encounter the log file write 
permission and the secure token read permission issues due to the 
"bind-mounted" directory.
     ** For applications in container don't have dependencies on local host, 
like web application:
     We seem need only to solve the log write permission during my testing (not 
sure for the log aggregation). Or find another way which don't persist log in 
local host.
* To sum up, if we don't want to change container at all, *the known blocking 
issues here are how to collect log from container and how to feed the container 
secure token in a different way from "bind-mount" host directory*. IMO, it's 
nearly impossible because the only way to connect container and host seems 
"bind-mount" or it will involve modification to container.

But again, current MRv2/spark in container already needs UID modification in 
Docker images, requirements of  "sudo" installed in Docker images is not 
unacceptable to enable this white-listed user as a light weight solution.
Anyway, I'm open to any better solution. Ideas? [~templedf], [~vvasudev], 

> Allow whitelisted users to disable user re-mapping/squashing when launching 
> docker containers
> ---------------------------------------------------------------------------------------------
>                 Key: YARN-4266
>                 URL: https://issues.apache.org/jira/browse/YARN-4266
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Sidharta Seethana
>            Assignee: Zhankun Tang
>         Attachments: 
> YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping.pdf
> Docker provides a mechanism (the --user switch) that enables us to specify 
> the user the container processes should run as. We use this mechanism today 
> when launching docker containers . In non-secure mode, we run the docker 
> container based on 
> `yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user` and in 
> secure mode, as the submitting user. However, this mechanism breaks down with 
> a large number of 'pre-created' images which don't necessarily have the users 
> available within the image. Examples of such images include shared images 
> that need to be used by multiple users. We need a way in which we can allow a 
> pre-defined set of users to run containers based on existing images, without 
> using the --user switch. There are some implications of disabling this user 
> squashing that we'll need to work through : log aggregation, artifact 
> deletion etc.,

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to