[
https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or updated SPARK-5388:
-----------------------------
Description:
The existing submission gateway in standalone mode is not compatible across
Spark versions. If you have a newer version of Spark submitting to an older
version of the standalone Master, it is currently not guaranteed to work. The
goal is to provide a stable REST interface to replace this channel.
The first cut implementation will target standalone cluster mode because there
are very few messages exchanged. The design, however, will be general enough to
eventually support this for other cluster managers too. Note that this is not
necessarily required in YARN because we already use YARN's stable interface to
submit applications there.
was:
The existing submission gateway in standalone mode is not compatible across
Spark versions. If you have a newer version of Spark submitting to an older
version of the standalone Master, it is currently not guaranteed to work. The
goal is to provide a stable REST interface to replace this channel.
A design doc will be posted shortly.
> Provide a stable application submission gateway
> -----------------------------------------------
>
> Key: SPARK-5388
> URL: https://issues.apache.org/jira/browse/SPARK-5388
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.0
> Reporter: Andrew Or
> Assignee: Andrew Or
> Priority: Blocker
>
> The existing submission gateway in standalone mode is not compatible across
> Spark versions. If you have a newer version of Spark submitting to an older
> version of the standalone Master, it is currently not guaranteed to work. The
> goal is to provide a stable REST interface to replace this channel.
> The first cut implementation will target standalone cluster mode because
> there are very few messages exchanged. The design, however, will be general
> enough to eventually support this for other cluster managers too. Note that
> this is not necessarily required in YARN because we already use YARN's stable
> interface to submit applications there.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
