[
https://issues.apache.org/jira/browse/YARN-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922888#comment-16922888
]
Eric Yang commented on YARN-9561:
---------------------------------
[~ebadger] thank you for the debugging session today. I got mapreduce pi to
run correctly after adding /etc/krb5.conf to default mount location. Some
improvements to make this better:
1. The current output looks like this when container run fails:
{code}
[2019-09-04 13:26:30.726]Exception from container-launch.
Container id: container_1567624987243_0004_01_000006
Exit code: 1
Exception message: Launch container failed
[2019-09-04 13:26:30.731]Container exited with a non-zero exit code 1. Error
file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
[2019-09-04 13:26:30.734]Container exited with a non-zero exit code 1. Error
file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
2019-09-04 13:26:31,455 INFO mapreduce.Job: Task Id :
attempt_1567624987243_0004_m_000003_0, Status : FAILED
{code}
Print a line of output before calling runc. Prelaunch initialization is
completed or print the formatted json in prelaunch.out. This helps to narrow
down the root of the problem is caused by user job configuration or bugs in
container-executor code. Docker runtime shows the command line for calling
docker. This helps to troubleshoot the actual problem sooner.
2. ENTRY_POINT support. Instead of calling out to launch_container.sh, it
would be nice to dup the stdout, stderr without launch_container.sh wrapper.
This helps to remove the requirement of bind mounting log or workdir
directories into the container for some use cases.
3. User defined properties integration:
YARN Docker integration have a list of [configurable
properties|https://hadoop.apache.org/docs/r3.2.0/hadoop-yarn/hadoop-yarn-site/DockerContainers.html#Application_Submission].
These settings do not work with runc container today. Without hinder
progress, I suggest to open new issues to improve integration.
4. YARN service uses the properties defined in #3 for customize YARN services
mount points, network to use, and privilege container flag. Similar feature
sets need new tickets to ensure the new runtime can integrate well with YARN
service programming interfaces.
> Add C changes for the new RuncContainerRuntime
> ----------------------------------------------
>
> Key: YARN-9561
> URL: https://issues.apache.org/jira/browse/YARN-9561
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Badger
> Assignee: Eric Badger
> Priority: Major
> Attachments: YARN-9561.001.patch, YARN-9561.002.patch,
> YARN-9561.003.patch, YARN-9561.004.patch
>
>
> This JIRA will be used to add the C changes to the container-executor native
> binary that are necessary for the new RuncContainerRuntime. There should be
> no changes to existing code paths.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
