Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/15249
@squito I agree, you are right - even the functionality is blacklisting in
both cases, the problem each is trying to tackle are quite different.
Transient issues (resource, shutdown, etc) versus more permanent failure
modes (bad memory, disk error, disk full, etc).
You are correct that existing config is trying to handle former, and is
suboptimal for latter (actually, it is not designed to handle it). So the value
user will need to set, as you elaborated, if he is to use this config is going
to be fairly meaningless (and affects its ability to handle the former).
I think it is logical to split implementation for the two usecases - with
only config namespace being same (since it is blacklist'ing from user pov !).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
