szehon-ho edited a comment on pull request #2358:
URL: https://github.com/apache/iceberg/pull/2358#issuecomment-804450547
This is an Implementation for Predicate Push down for Partition Table. It
is based on a conversation first with @RussellSpitzer
and then with some community in
https://github.com/apache/iceberg/issues/2326.
It is not the ultimate solution to the problem , which may need the Spark
view catalog as discussed in the issue. But it should be complementary as the
same logic can be used in Files Metadata table.
Also, as we don't know when view-catalog comes, this at least makes it so
that we won't have partition query with a filter taking longer than an actual
query with that filter :)
If this change is the right way, next step would be implementing the same
partition-filter on files-table to reduce the metadata scanned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
