Github user lianhuiwang commented on the issue:
https://github.com/apache/spark/pull/13494
@cloud-fan Yes, I think what you said is right. as Hive/Prestodb, if
queries that did some functions (example: MIN/MAX) or distinct aggregates on
partition column and the value of config 'spark.sql.optimizer.metadataOnly' is
true, then we can use the metadata-only optimization.
But now we cannot check it in FileSourceStrategy. In the long run, at
firstly we need put partition filter into optimizer like #13373. and then we
can check metadataOnly optimizer and did it.
So this PR just did this optimizer in FileSourceStrategy. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
