shangxinli commented on pull request #1566: URL: https://github.com/apache/iceberg/pull/1566#issuecomment-721292039
@shardulm94 I am also curious about why Iceberg reimplements those filters. @rdblue Can you cast some light on this? I can see the pros and cons of reimplementing. The concern for reimplementing is that it creates duplication and fragments. For short-term workaround it is fine, but for the long run, I think deduping to one implementation makes more sense. When new filters are added in Parquet, we need to reimplement them here. Column index is one, and bloom filer is another one on the way, and so on. If some filters in Iceberg but not in Parquet, we can bring in to the Parquet community to add them. What do you think? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
