[ https://issues.apache.org/jira/browse/SPARK-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361556#comment-15361556 ]
Ashic Mahtab commented on SPARK-16364: -------------------------------------- Would it be more reasonable to have spark-submit upload the jar to the master, which would expose it over http? Seeing that the master already runs a web interface, it can then double up as a file server for the jars. Perhaps this could be controlled with a flag? > Allow spark-submit to upload jars to nodes in cluster mode > ---------------------------------------------------------- > > Key: SPARK-16364 > URL: https://issues.apache.org/jira/browse/SPARK-16364 > Project: Spark > Issue Type: Improvement > Components: Spark Submit > Affects Versions: 1.6.2 > Environment: Stand-Alone cluster > Reporter: Ashic Mahtab > Priority: Minor > > Currently, it seems that with a stand-alone cluster, running spark-submit > with --deploy-mode cluster requires the target jar to be visible from all > nodes. It would be convenient if spark-submit had the option to copy the jar > to the relevant nodes, instead of requiring it to already by present. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org