Github user Parth-Brahmbhatt commented on the issue:
https://github.com/apache/spark/pull/14655
@gatorsmile not sure if its the same issue. The issue you are pointing at
talks about storing the actual partition level stats, which could be used by
this PR but until its available we could rely on file system level statistics.
Also given this is config driven which is disabled by default it should have no
perf impact.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
