Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/546#issuecomment-41512227
Just fly-by-commenting here, but the performance cost of doing a few
directory traversals is not relevant in this code path. It gets called only
once in the lifetime of a broadcast variable. It think the bigger issues are
around correctness and consistency...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
