[
https://issues.apache.org/jira/browse/SPARK-18611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrian Bridgett updated SPARK-18611:
------------------------------------
Affects Version/s: 1.6.3
> mesos shuffle service on v2 isn't compatible with spark v1
> ----------------------------------------------------------
>
> Key: SPARK-18611
> URL: https://issues.apache.org/jira/browse/SPARK-18611
> Project: Spark
> Issue Type: Bug
> Components: Shuffle
> Affects Versions: 1.6.3
> Reporter: Adrian Bridgett
> Priority: Minor
>
> In SPARK-12583 heartbeating was added to the shuffle service.
> This unfortunately changes registerDriver with a different signature so we
> can't use sparkv1 clients with sparkv2 mesos shuffle service.
> http://apache-spark-developers-list.1001551.n3.nabble.com/YARN-Shuffle-service-and-its-compatibility-td17222.html
> is a slightly different issue, but it does suggest that it's agreed that
> compatibility is a good idea in these cases.
> https://github.com/apache/spark/pull/13279 was created a while ago to
> backport this to 1.6.x but was rejected (too invasive). However I don't
> believe this issue (v1/v2 compatibility) was recognised back then so perhaps
> it's time to reconsider?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
