Github user adrian-ionescu commented on the issue:
https://github.com/apache/spark/pull/17465
LGTM
I think it's ok that it doesn't take into account partition pruning, since
that's not really part of this physical operator. Another thing to note here is
that `listFiles()` may often return from cache. We could consider exposing file
cache hits & misses as another metric.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
