Github user squito commented on the issue:
https://github.com/apache/spark/pull/20640
@susanxhuynh good point about changing default behavior. I'd rather have
the change so we have more unified behavior between mesos and other cluster
managers. But I have never run spark on mesos or even had much interaction
with users of spark on mesos, so I will defer to others judgement. Another
option: we could leave the old behavior, unless a user sets
spark.blacklist.enabled=true. its a little wonky, but that also guarantees you
always get some blacklisting.
I've also been considering turning blacklisting on by default in spark 2.4.
So far I've had good feedback from users running it (though we'll get way more
feedback when its on by default).
btw, one hole in the general blacklisting is handling cases when the
executors fail to even start: https://issues.apache.org/jira/browse/SPARK-16630
would that be covered by mesos with the old code? just want to make sure
we aren't losing that ability.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
