[
https://issues.apache.org/jira/browse/SPARK-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194513#comment-14194513
]
Chris Heller edited comment on SPARK-2691 at 11/3/14 1:08 PM:
--------------------------------------------------------------
Here is the patch for the changes needed to support docker images in the fine
grained backend.
The approach taken here was to just populate the DockerInfo of the container
info if some properties were set in the properties file. This has no support
for versions of mesos which do not support docker, so it is very incomplete.
Additionally there is only support for image name, and volumes.
For volumes, you just provide a string, which takes a value similar in form to
the argument to 'docker run -v', ie. it is a comma separated list of
[host:]container[:mode] options. I think this is sufficient, it parallels the
command line, and so should be familiar.
I would suggest, for all options of the DockerInfo exposed, to mirror how
those options are set on the docker command line.
was (Author: chrisheller):
Here is the patch for the changes needed to support docker images in the fine
grained backend.
The approach taken here was to just populate the DockerInfo of the container
info if some properties were set in the properties file. This has no support
for versions of mesos which do not support docker, so it is very incomplete.
Additionally there is only support for image name, and volumes.
For volumes, you just provide a string, which takes a value similar in form to
the argument to 'docker run -v', ie. it is a comma separated list of
[host:]container[:mode] options. I think this is sufficient, it parallels the
command line, and so should be familiar.
> Allow Spark on Mesos to be launched with Docker
> -----------------------------------------------
>
> Key: SPARK-2691
> URL: https://issues.apache.org/jira/browse/SPARK-2691
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Assignee: Timothy Chen
> Labels: mesos
> Attachments: spark-docker.patch
>
>
> Currently to launch Spark with Mesos one must upload a tarball and specifiy
> the executor URI to be passed in that is to be downloaded on each slave or
> even each execution depending coarse mode or not.
> We want to make Spark able to support launching Executors via a Docker image
> that utilizes the recent Docker and Mesos integration work.
> With the recent integration Spark can simply specify a Docker image and
> options that is needed and it should continue to work as-is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
