Github user squito commented on the issue:
https://github.com/apache/spark/pull/23223
@attilapiros
> I mean if node blacklisting in Spark would be perfectly aligned to YARN
then it would be just redundant to have it in Spark in the first place.
I'm not super familiar with exactly how the blacklisting works in yarn
itself -- it looks like its only going to blacklist the node for the AM, not
other nodes for general containers. I don't totally follow where the
`KILLED_BY_RESOURCEMANAGER` status is generated, but it does seem like a good
idea to protect against this, maybe there is a race where the container is
created by the RM, but before it reports back to the driver it gets killed with
KILLED_BY_RESOURCEMANAGER. (another reason I'm curious if @Ngone51 is actually
seeing this cause problems, or just noticed a case to improve.)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
