[
https://issues.apache.org/jira/browse/YARN-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leitao Guo updated YARN-3282:
-
Attachment: YARN-3282.01.patch
When using DockerContainerExecutor, mapreduce jobs can set docker environment
variables via yarn.nodemanager.docker-container-executor.env after the patch.
e.g.
{code}
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples.jar wordcount
-Dyarn.app.mapreduce.am.env=yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0
-Dmapreduce.map.env=yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0
-Dmapreduce.reduce.env=yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0
-Dyarn.nodemanager.docker-container-executor.env=-v
/data/transcode:/data/tmp -v /etc/qcs:/etc/qcs -v /mnt:/mnt -e
VTC_MQTYPE=rabbitmq -e VTC_APP=ugc -e VTC_LOCATION=sh -e VTC_RUNTIME=vtc
/wordcount_input /wordcount_output
{code}
DockerContainerExecutor should support environment variables setting
Key: YARN-3282
URL: https://issues.apache.org/jira/browse/YARN-3282
Project: Hadoop YARN
Issue Type: Improvement
Components: applications, nodemanager
Affects Versions: 2.6.0
Reporter: Leitao Guo
Attachments: YARN-3282.01.patch
Currently, DockerContainerExecutor will mount yarn.nodemanager.local-dirs
and yarn.nodemanager.log-dirs to containers automatically. However
applications maybe need set more environment variables before launching
containers.
In our applications, just as the following command, we need to attach several
directories and set some environment variables to docker containers.
{code}
docker run -i -t -v /data/transcode:/data/tmp -v /etc/qcs:/etc/qcs -v
/mnt:/mnt -e VTC_MQTYPE=rabbitmq -e VTC_APP=ugc -e VTC_LOCATION=sh -e
VTC_RUNTIME=vtc sequenceiq/hadoop-docker:2.6.0 /bin/bash
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
