[
https://issues.apache.org/jira/browse/SPARK-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725747#comment-14725747
]
Marcelo Vanzin commented on SPARK-10288:
----------------------------------------
Hi Saisai,
I have a question: why do you need this? Why would anyone choose to submit to
YARN using the REST backend instead of the usual {{YarnClient}}? What extra
features does it support, or what does it fix?
>From a Spark user's perspective, it makes no difference whether the underlying
>code is using REST or not. In the case of standalone, the rest server was
>added to make it easier to support multiple Spark versions submitting to the
>cluster manager; but YARN doesn't have that problem. The binary protocol is
>compatible across different YARN versions and so this different backend just
>seems to add complexity without offering any compelling features in return.
> Add a rest client for Spark on Yarn
> -----------------------------------
>
> Key: SPARK-10288
> URL: https://issues.apache.org/jira/browse/SPARK-10288
> Project: Spark
> Issue Type: New Feature
> Components: YARN
> Reporter: Saisai Shao
>
> This is a proposal to add rest client for Spark on Yarn. Currently Spark
> standalone and Mesos mode can support rest way of submitting applications,
> for Spark on Yarn, it still uses program way to do it. Since RM now (from
> 2.6) supports rest way of submitting application, so it would be better Spark
> on Yarn also support this way.
> Here is the design doc
> (https://docs.google.com/document/d/1m_P-4olXrp0tJ3kEOLZh1rwrjTfAat7P3fAVPR5GTmg/edit?usp=sharing).
> Currently I'm working on it, working branch is
> (https://github.com/jerryshao/apache-spark/tree/yarn-rest-support), the major
> part is already finished.
> Any comment is greatly appreciated, thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
