[
https://issues.apache.org/jira/browse/SPARK-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14933193#comment-14933193
]
Chris Heller commented on SPARK-8734:
-------------------------------------
This approach eliminates the ability to pass values into the parameter which
would embed a comma. For instance if I wished to pass a ZooKeeper connection
string into a container by an environment variable, I would be unable to do so
if the comma is reserved to allow multiple parameters since the connection
string also reserves the comma for its delimiter.
Just as a thought exercise, what about a method such as:
spark.mesos.executor.docker.parameter.env.1: FOO=bar
spark.mesos.executor.docker.parameter.env.2: BAZ=quux
(where the .* suffix is just a unique value to prevent key folding by the hash
table)
This is ugly in that it exposes an implementation detail I'd rather not expose,
but I'd rather have to suffix a ".1", than do something like:
spark.mesos.executor.docker.parameter.env:
ZOOKEEPER=host1:2181\,host2:2181\,host3:2181, FOO=bar
Don't support comma delimited values, escape all commas, and unescape them
after splitting the value in the code, or rework the property name to encode
ordinal information to support multi-valued keys. Are these the only
alternatives?
> Expose all Mesos DockerInfo options to Spark
> --------------------------------------------
>
> Key: SPARK-8734
> URL: https://issues.apache.org/jira/browse/SPARK-8734
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Chris Heller
> Priority: Minor
> Attachments: network.diff
>
>
> SPARK-2691 only exposed a few options from the DockerInfo message. It would
> be reasonable to expose them all, especially given one can now specify
> arbitrary parameters to docker.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
