Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/21895
I don't think the problem you mentioned is a big problem.
1. For the blacklist mechanism, we can have a time-based reviving mechanism
to check if permission is changed, compared to check file permission for all
the files, the cost would not be so high. Also as you mentioned, the permission
is seldom changed, so it is fine without change.
2. I don't think this is a problem, try-catch with proper log should be
enough.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
