[
https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623762#comment-16623762
]
Xun Liu commented on YARN-8563:
-------------------------------
I don't think submarine needs to care about image management and production
issues for the following reasons:
# YARN-3.x has supported runtime environment isolation through docker. If too
much consideration is given to supporting dynamic installation dependencies on
a base image, then the meaning of docker is lost.
# Submarine need to provide a docker-hub repository for the submarine project,
which provides several stable versions of the tensorflow version, plus a full
python library as the base image, which is sufficient for the user to use.
Because ordinary users are generally able to use the off-the-shelf docker image
to complete their work, it is satisfied.
If they are advanced users, they will customize it using our base image.
So I think that the production and management of the image is handled by the
user and does not need to be included in the submarine project. Submarine need
more focus on the high performance, stability and ease of use of the submarine.
> [Submarine] Support users to specify Python/TF package/version/dependencies
> for training job.
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-8563
> URL: https://issues.apache.org/jira/browse/YARN-8563
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Priority: Major
>
> YARN-8561 assumes all Python / Tensorflow dependencies will be packed to
> docker image. In practice, user doesn't want to build docker image. Instead,
> user can provide python package / dependencies (like .whl), Python and TF
> version. And Submarine can localize specified dependencies to prebuilt base
> Docker images.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
