[
https://issues.apache.org/jira/browse/YARN-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699908#comment-16699908
]
Charo Zhang commented on YARN-9053:
-----------------------------------
[~Jim_Brennan] In the non-entrypoint case, we summit a GPU application by
{code:java}
yarn jar \
/data/bigdata/hadoop/client/hadoop-yarn-applications-distributedshell-3.1.1.jar
-jar \
/data/bigdata/hadoop/client/hadoop-yarn-applications-distributedshell-3.1.1.jar
\
-shell_env YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE=false \
-shell_env YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK=bridge \
-shell_env YARN_CONTAINER_RUNTIME_TYPE=docker \
-shell_env
hadoop_security_authentication_tbds_securekey=jNCmfGkKkvpRv48aN7X43UKRnCzyvYda \
-shell_env
YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=docker.oa.com:8080/public/tensorflow1.8.0-gpu-py2:yarn311
\
-shell_env YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS=/cephfs/group/:/cephfs/group:rw
-timeout 214748367 \
-shell_command "tailf /etc/hosts" -container_resources
memory-mb=1024,vcores=1,yarn.io/gpu=1 -num_containers 1
{code}
When we access container by docker exec, we can't find
"hadoop_security_authentication_tbds_securekey " variable in "env" list. If we
set YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE=true, the
"hadoop_security_authentication_tbds_securekey " existed in "env" list.
And "hadoop_security_authentication_tbds_securekey " has configured in
white-listed variables.
Besides, we don't think white-listed way is good, because it must restart
nodemanger when white-listed be changed, and we can not make sure how many
environment variables will be used from the start.
> Support set environment variables for Docker Containers In nonEntryPoint mode
> -----------------------------------------------------------------------------
>
> Key: YARN-9053
> URL: https://issues.apache.org/jira/browse/YARN-9053
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Affects Versions: 3.1.1
> Reporter: Charo Zhang
> Priority: Major
> Labels: Docker
> Attachments: YARN-9053.patch
>
>
> In yarn 3.1.1, users can only set environment variables with "-shell_env" in
> ENTRYPOINT mode, and variables must be registered in
> yarn.nodemanager.env-whitelist.
> But in nonEntryPoint mode, we should allow users to set environment variables
> like "-e KEY=VAULE" in docker run command, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
