szehon-ho commented on pull request #2358: URL: https://github.com/apache/iceberg/pull/2358#issuecomment-804450547
This is an Implementation for Predicate Push down for Partition Table. It is based on a conversation first with @russell-spitzer and then with some community in https://github.com/apache/iceberg/issues/2326. It is not the ultimate solution to the problem , which may need the Spark view catalog as discussed in the issue. But it should be complementary as the same logic can be used in Files Metadata table. Also, as we don't know when view-catalog comes, this at least makes it so that we won't have partition query with a filter taking longer than an actual query with that filter :) If this change is the right way, next step would be implementing the same partition-filter on files-table to reduce the metadata scanned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
