[
https://issues.apache.org/jira/browse/YARN-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586601#comment-16586601
]
Shane Kumpf commented on YARN-8623:
-----------------------------------
I was able to run the below MR pi job with the modified _apache/hadoop-runner_
image, after a quick hack to the sudo rules.
{code:java}
YARN_EXAMPLES_JAR=$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar
IMAGE_ID="local/hadoop-runner-new:latest"
MOUNTS="/usr/local/hadoop:/usr/local/hadoop:ro,/etc/hadoop/conf:/etc/hadoop/conf:ro,/etc/passwd:/etc/passwd:ro,/etc/group:/etc/group:ro"
yarn jar $YARN_EXAMPLES_JAR pi \
-Dmapreduce.map.env.YARN_CONTAINER_RUNTIME_TYPE=docker \
-Dmapreduce.map.env.YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS=$MOUNTS \
-Dmapreduce.map.env.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$IMAGE_ID \
-Dmapreduce.reduce.env.YARN_CONTAINER_RUNTIME_TYPE=docker \
-Dmapreduce.reduce.env.YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS=$MOUNTS \
-Dmapreduce.reduce.env.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$IMAGE_ID 1
40000{code}
Hadoop bits were installed to {{/usr/local/hadoop}} on the host. Hadoop config
in {{/etc/hadoop/conf}} on the host. The appropriate mounts were added to
{{docker.allowed.ro-mounts}} and the image prefix to
{{docker.trusted.registries}} in {{container-executor.cfg}}
The above assumes the use of {{/etc/passwd}} and {{/etc/group}} for propagating
the user and group into the container. We should point to the other ways of
[managing user
propagation|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md#user-management-in-docker-container]
as part of this example documentation.
> Update Docker examples to use image which exists
> ------------------------------------------------
>
> Key: YARN-8623
> URL: https://issues.apache.org/jira/browse/YARN-8623
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Craig Condit
> Priority: Minor
> Labels: Docker
>
> The example Docker image given in the documentation
> (images/hadoop-docker:latest) does not exist. We could change
> images/hadoop-docker:latest to apache/hadoop-runner:latest, which does exist.
> We'd need to do a quick sanity test to see if the image works with YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
