[
https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398593#comment-15398593
]
Sidharta Seethana commented on YARN-5360:
-----------------------------------------
To clarify a few things here - yes, customized images are required in some
cases (especially in secure mode) to make apps work for certain users. This is
a limitation we have to work with for the moment given the hadoop security
model - it may not be reasonable or practical to drop it altogether except
under controlled situations. Also, log aggregation does not work in secure mode
if you drop "--user" (it works in non-secure mode, I think but I'll have to
check the code/test again). Artifact deletion will not work if the artifacts
are created as a different user in the docker container (artifact cleanup is
done as the 'run as' user).
In your first table above, the yarn/nobody case likely did not work because the
sequenceiq image is based on centos (nobody uid=99) and the system you were
testing on was not centos (ubuntu? nobody uid=65534). We have tested spark with
other images on centos (I have test images on docker hub if you'd like to try).
I am pretty sure [~templedf] has successfully run spark using the current
implementation as well. ([~templedf] : please confirm) .
If this the discussion here is now only about dropping the "--user" in certain
cases, this is captured in YARN-4266.
> Decouple host user and Docker container user
> --------------------------------------------
>
> Key: YARN-5360
> URL: https://issues.apache.org/jira/browse/YARN-5360
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Zhankun Tang
> Assignee: Zhankun Tang
>
> There is *a dependency between job submitting user and the user in the Docker
> image* in LCE currently. For instance, in order to run the Docker container
> as yarn user, we can choose set the
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn
> and leave
> "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users"
> default (true). Then LCE will choose yarn ( UID maybe 1001) as the user
> running jobs.
> LCE will mount the generated launch_container.sh (owned by the running job
> user) and /etc/passwd (*current the code is mounting to container's
> /etc/password, I think it's a mistake*) into the Docker container and
> utilizes "docker run --user=<run_as_user>" option to get it done internally.
> Mounting /etc/passwd to the container is a not good choice due to override
> original users defined in Docker image. As far as I know, since Docker v1.8
> (or maybe earlier), the Docker run command "--user=" option accepts UID and
> *when passing UID, the user does not have to exist in the container*. So we
> could use UID instead of user name to construct the Docker run command to
> eliminate the dependency that create the same user in the Docker image. This
> enables LCE the ability to launch any Docker container safely regardless what
> users in it.
> But this is not enough to decouple host user and Docker container user. The
> final solution we are searching for are focused on allowing users to run
> their Docker images flexibly without involving dependencies of YARN and make
> sure the container won't bring in security risk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
