Github user squito commented on the issue:
https://github.com/apache/spark/pull/21068
@tgravescs on the blacklist ratio for task-based blacklisting -- there is
nothing, but there are some related jiras:
[SPARK-22148](https://issues.apache.org/jira/browse/SPARK-22148) &
[SPARK-15815](https://issues.apache.org/jira/browse/SPARK-15815)
to be honest I have doubts about the utility of the ratio ... if you really
want to make sure blacklisting doesn't lead to starvation, you've got to have
some other mechanism, as you could easily have the remaining nodes be occupied
or have insufficient resources.
Kubernetes doesn't do anything with the node blacklisting currently:
[SPARK-23485](https://issues.apache.org/jira/browse/SPARK-23485)
Mesos already has a notion of blacklisting nodes for failing to allocate
containers, but its currently at odds with the task-based blacklist.
https://github.com/apache/spark/pull/20640 is somewhat stalled because
blacklisting based on allocation failures is missing in a general sense.
In any case, I still think we shouldn't make the code more complex for
something other clusters managers *might* use in the future, and that the
current overall organization is fine.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
