Github user squito commented on the issue: https://github.com/apache/spark/pull/23223 @attilapiros > I mean if node blacklisting in Spark would be perfectly aligned to YARN then it would be just redundant to have it in Spark in the first place. I'm not super familiar with exactly how the blacklisting works in yarn itself -- it looks like its only going to blacklist the node for the AM, not other nodes for general containers. I don't totally follow where the `KILLED_BY_RESOURCEMANAGER` status is generated, but it does seem like a good idea to protect against this, maybe there is a race where the container is created by the RM, but before it reports back to the driver it gets killed with KILLED_BY_RESOURCEMANAGER. (another reason I'm curious if @Ngone51 is actually seeing this cause problems, or just noticed a case to improve.)
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org