[
https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376531#comment-15376531
]
Zhankun Tang commented on YARN-5360:
------------------------------------
[~sidharta-s], I understand your concerns of the side effect of using UID. I am
not sure of UID working for all cases although I haven't find any issue in my
testing currently. But as you said, *mounting /etc/passwd is not a good choice
due to override existing users in Docker image*. So I should change the title
of this JIRA so that we can focus on improving current running container user
mechanism of LCE.
If my understanding is correct, YARN-4266 means allowing white-listed users
defined in config file to run docker without "--user" option. Then
* If no user was defined in Dockerfile with "USER <user_name/UID>", the Docker
container would run as root by default. This has no permission and big security
issue I think.
* If there is user or UID defined in Dockerfile, then the container would run
as unpredictable user which would encounter permission issue if the
launch_container.sh is owned by a different UID from the one defined in
Dockerfile. And also some implications needs to work through.
So *YARN-4266 allowing a white-listed user config is also not enough*.
Anyway, all the solution we are searching for are focused on allowing users to
run their Docker images flexibly without involving dependencies of YARN and
make sure the container won't bring in security risk. We should *consider the
trade-off carefully*:
* 1. For flexibility which allowing user to specify which user they want to run
their container as, current configuration is not enough. Because *we should not
assume container running user* is same with the submitting lcoal host user. So
exposing the "--user=<username/UID>" as an environment variable in
ContainerLaunchContext at application submission time is one reasonable way.
** But simply exposing the "--user" won't work at current situation. Because
current LCE set “700” file permission to the "launch_container.sh"(other
directories has additional "s" bit) and *the owner is a user in the local host,
this is a key barrier blocking the flexibility currently*. I think one way to
solve this is use "705" permission to allow other group users to read and
execute the "launch_container.sh". The log directories should still can be
writable.
** So exposing "--user" to the application and when the environment variable
was set, just change the "launch_container.sh" permission to 705 will work. And
by default, if this environment variable was not set, utilize UID to run the
container should be flexible and work for most cases.
* 2. For security, I guess current LCE mounting the files with only owner
privileges is for security consideration, but when it comes to Docker,
Shouldn't the isolated Docker container be safe even we run user root in it?
Based on this assumption, we can just discard the "--user" and let Docker
handles which user (root or defined in Dockerfile) to run the container. YARN
only need to make sure the critical files/directories mounted in the container
are read-only(for instance, public scope cache).
To sum up, Our goal is to find the balance between run-as-user flexibility and
security. *Using UID is just a tiny step* for decoupling the container user
namespace from local host without other changes in current LCE. Future steps we
can choose above option 1. Or we just choose option2 for Docker. Any ideas?
PS. It's a good idea that we do more research of Docker user namespace/user
remapping before we find a better solution. I'll investigate that too.
> Use UID instead of user name to build the Docker run command
> ------------------------------------------------------------
>
> Key: YARN-5360
> URL: https://issues.apache.org/jira/browse/YARN-5360
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Zhankun Tang
> Assignee: Zhankun Tang
>
> There is *a dependency between job submitting user and the user in the Docker
> image* in LCE currently. For instance, in order to run the Docker container
> as yarn user, we can choose set the
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn
> and leave
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users"
> default (true). Then LCE will choose yarn ( UID maybe 1001) as the user
> running jobs.
> LCE will mount the generated launch_container.sh (owned by the running job
> user) and /etc/passwd (*current the code is mounting to container's
> /etc/password, I think it's a mistake*) into the Docker container and
> utilizes "docker run --user=<run_as_user>" option to get it done internally.
> But I don't think mounting /etc/passwd to the container is a good choice. As
> far as I know, since Docker v1.8 (or maybe earlier), the Docker run command
> "--user=" option accepts UID and *when passing UID, the user does not have to
> exist in the container*. So we should use UID instead of user name to
> construct the Docker run command to eliminate the dependency that create the
> same user in the Docker image. This enables LCE the ability to launch any
> Docker container safely regardless what users in it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
