Github user chemikadze commented on the issue:
https://github.com/apache/spark/pull/7786
@vanzin If those would be implemented, would it have any change to get
merged? We use preemption quite a lot and current behavior is not the best we
can get: logs sometimes getting overfilled with preemption side effects (RPC
errors, etc), getting logs hard to read and confusing some users. I agree that
depending on task size, effect might be both positive and negative (longer ones
anyway won't be able to complete be wasting resources, but lots of shorter ones
will not get chance to run). Does it just mean it should be configurable
behavior (spark.yarn.releasePreemptedContainers=true/false)?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
