[
https://issues.apache.org/jira/browse/YARN-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509094#comment-16509094
]
Wangda Tan commented on YARN-8220:
----------------------------------
Discussed with [~eyang] about this and did some tests:
Currently, YARN NM passes JAVA_HOME, HDFS_HOME, CLASSPATH environments before
launching Docker container no matter if ENTRY_POINT is used or not. This will
overwrite environments defined inside Dockerfile (by using \{{ENV}}). For
Docker container, it actually doesn't make sense to pass JAVA_HOME, HDFS_HOME,
etc. because inside docker image we have a separate Java/Hadoop installed or
mounted to exactly same directory of host machine.
I just filed YARN-8417 to revisit this behavior.
Once the above change is done, we actually don't need to presetup common
configs inside service spec or presetup.sh, everything could be done very
cleanly inside the Dockerfile.
For this patch:
Considering size of this patch, I suggest to get it merged before YARN-8417. We
can continuously improve it (like using ENV/ENTRY_POINT) after YARN-8417 and
feedbacks from others.
Really appreciate valuable inputs from [~eyang]!
> Running Tensorflow on YARN with GPU and Docker - Examples
> ---------------------------------------------------------
>
> Key: YARN-8220
> URL: https://issues.apache.org/jira/browse/YARN-8220
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Critical
> Attachments: YARN-8220.001.patch, YARN-8220.002.patch
>
>
> Tensorflow could be run on YARN and could leverage YARN's distributed
> features.
> This spec fill will help to run Tensorflow on yarn with GPU/docker
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
