[
https://issues.apache.org/jira/browse/YARN-7430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16248147#comment-16248147
]
Eric Yang commented on YARN-7430:
---------------------------------
{quote}
I don't believe that is true? I'm referring to YARN containers, not docker
containers in this case. YARN tasks will write their logs to the directory
specified by yarn.nodemanager.log-dirs.
{quote}
Yes, container-executor launch it, and prep the app logging directory as the
user who is supposed to run the container. In the old days, we have TaskLog
appender which captures stderr and stdout of mapreduce task. Regardless of the
technique, if the script to run container looks like:
{code}
docker run -it ... > /path/to/log
{code}
This will redirect docker output by the shell script. It depends on who
spawned the shell, and the resulting log output would be owned by the user who
spawned the shell. Root user can potentially end up with a file owned by root
user, which you stated that can not be cleaned up. How about we change the
code to:
{code}
sudo wrapper_script_to_docker_run.sh | tee -a /path/to/log
{code}
The second approach is the receiving end of the data can be the original user.
Hence, there is no permission problem with clean up.
> User and Group mapping are incorrect in docker container
> --------------------------------------------------------
>
> Key: YARN-7430
> URL: https://issues.apache.org/jira/browse/YARN-7430
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: security, yarn
> Affects Versions: 2.9.0, 3.0.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Attachments: YARN-7430.001.patch
>
>
> In YARN-4266, the recommendation was to use -u [uid]:[gid] numeric values to
> enforce user and group for the running user. In YARN-6623, this translated
> to --user=test --group-add=group1. The code no longer enforce group
> correctly for launched process.
> In addition, the implementation in YARN-6623 requires the user and group
> information to exist in container to translate username and group to uid/gid.
> For users on LDAP, there is no good way to populate container with user and
> group information.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
