Wangda Tan created SUBMARINE-570:
------------------------------------
Summary: Support run experiment/notebook with synced code
Key: SUBMARINE-570
URL: https://issues.apache.org/jira/browse/SUBMARINE-570
Project: Apache Submarine
Issue Type: New Feature
Components: code-sync
Reporter: Wangda Tan
In Submarine Experiment, there're needs to sync up code from Git/S3 to run
experiment.
For example, Experiment can be defined to:
{code:java}
experiment: #Or notebook, model-serving
name: "abc",
environment: "team-default-ml-env"
// Other fields
code:
sync_mode: git
url: "https://foo.com/training-job.git"{code}
And once the experiment is launched, code from
[https://foo.com/training-job.git] will be synced under /code directory of the
launched container.
Different sync_mode can be supported, it can be git, s3, hdfs, nfs, etc.
Similarily, notebook can sync code when a notebook session is launched, this
avoids user manual checkout code post notebook session launch.
More design can be found at the following design docs:
-
[https://github.com/apache/submarine/blob/master/docs/design/storage-implementation.md#store-code-for-experimentnotebookmodel-serving]
-
[https://github.com/apache/submarine/blob/master/docs/design/experiment-implementation.md#api-of-experiment]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
