Github user skonto commented on the issue:
https://github.com/apache/spark/pull/19390
@susanxhuynh @ArtRand.
In order to deal with stripping allocation info for non multi-role
schedulers:
https://www.mail-archive.com/[email protected]/msg38329.html
I suggest to go with the 2nd option:
"(2) Any schedulers that update the protobuf, but leave their resource
matching logic as is, also continue to work since they ignore the
allocation info."
I am not sure of moving with full multi-role support for the Spark
Scheduler for several reasons:
a) In 1.3 tag of mesos the multi-role field is not recommended for use:
https://github.com/apache/mesos/blob/904729ed37802545ec650e46177e49e1c35c84a8/include/mesos/mesos.proto#L346-L348
b) Benefits for Spark Scheduler itself are minimal (I see a benefit for
the dispatcher to support multi-tenancy as marathon does).
(https://issues.apache.org/jira/browse/MESOS-1763)
When backwards compatibility wont be broken we can move to the multi-role
support. Thoughts?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
