[
https://issues.apache.org/jira/browse/SPARK-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361556#comment-15361556
]
Ashic Mahtab commented on SPARK-16364:
--------------------------------------
Would it be more reasonable to have spark-submit upload the jar to the master,
which would expose it over http? Seeing that the master already runs a web
interface, it can then double up as a file server for the jars. Perhaps this
could be controlled with a flag?
> Allow spark-submit to upload jars to nodes in cluster mode
> ----------------------------------------------------------
>
> Key: SPARK-16364
> URL: https://issues.apache.org/jira/browse/SPARK-16364
> Project: Spark
> Issue Type: Improvement
> Components: Spark Submit
> Affects Versions: 1.6.2
> Environment: Stand-Alone cluster
> Reporter: Ashic Mahtab
> Priority: Minor
>
> Currently, it seems that with a stand-alone cluster, running spark-submit
> with --deploy-mode cluster requires the target jar to be visible from all
> nodes. It would be convenient if spark-submit had the option to copy the jar
> to the relevant nodes, instead of requiring it to already by present.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
