Github user squito commented on the issue:
https://github.com/apache/spark/pull/20640
thats fine with me, but as I'm neither a user of mesos nor am I in touch w/
many mesos users, I'd like wait a bit for more opinions, given the
ramifications of this change. (that shouldn't block work on the better version,
if anybody wants to take that on ...) @IgorBerman @susanxhuynh
blacklisting in yarn (at least, what spark does internally already) is
really not much more sophisticated, at least before SPARK-16630; spark does
tell yarn which hosts it has blacklisted so it wants to avoid for future
executors, but thats about it. Yarn itself is doing a little more as well, as
it has its own disk health checker etc., and it'll try to exclude resources
from *all* applications if it thinks they are bad. but that is independent of
changes within spark itself.
also I'd like to see a jira for mesos to discuss the other improvements
we've discussed here to be more like SPARK-16630 so we don't forget about it.
@skonto can you file that jira and try to capture some of the points that have
been raised in the discussion here? (or maybe that jira exists already?)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
