[
https://issues.apache.org/jira/browse/SPARK-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14933291#comment-14933291
]
Ondřej Smola edited comment on SPARK-8734 at 9/28/15 1:41 PM:
--------------------------------------------------------------
I also thought about spark.mesos.executor.docker.parameter.env.1 - it will be
also inconsistent with other spark configuration entries as they are all comma
separated when multiple values are allowed.
I think main limitation is for env variables and labels ... we can model them
spark.mesos.executor.docker.parameter.env.FOO BAR
I dont think we can come up with consistent solution that will handle all
corner cases as docker parameters are free form and we have pretty limited
format without support for duplicate keys.
We can also look at what other frameworks on Mesos expose - such as Marathon,
Chronos, etc. - i think most users need ports,volumes, network type, env
variables, labels - this properties are not likely to change so we can give
them "special" handling and cover use cases of most users.
Edit: people -> users
was (Author: ondrej.smola):
I also thought about spark.mesos.executor.docker.parameter.env.1 - it will be
also inconsistent with other spark configuration entries as they are all comma
separated when multiple values are allowed.
I think main limitation is for env variables and labels ... we can model them
spark.mesos.executor.docker.parameter.env.FOO BAR
I dont think we can come up with consistent solution that will handle all
corner cases as docker parameters are free form and we have pretty limited
format without support for duplicate keys.
We can also look at what other frameworks on Mesos expose - such as Marathon,
Chronos, etc. - i think most people need ports,volumes, network type, env
variables, labels - this properties are not likely to change so we can give
them "special" handling and cover use cases of most users.
> Expose all Mesos DockerInfo options to Spark
> --------------------------------------------
>
> Key: SPARK-8734
> URL: https://issues.apache.org/jira/browse/SPARK-8734
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Chris Heller
> Priority: Minor
> Attachments: network.diff
>
>
> SPARK-2691 only exposed a few options from the DockerInfo message. It would
> be reasonable to expose them all, especially given one can now specify
> arbitrary parameters to docker.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
