kbendick commented on PR #4721: URL: https://github.com/apache/iceberg/pull/4721#issuecomment-1121793472
> @southernriver, when can Iceberg metadata be used to evaluate `endsWith`? The only case I can think of is when evaluating for identity partitioned data values, but that's an extremely narrow use case. I don't think it is worth the complexity of adding more predicates unless Iceberg metadata can be used to skip more data. Yeah I do agree. It would have to be identity partitions and the metrics would need to be stored longer than the standard truncation width. I pulled this down and played with it and it’s a very narrow use case. There are cases where it can have benefit, but we should weigh the complexity of the implementation over the value added. The use cases that will see a performance benefit from this are very narrow, but as it stands now I think the added complexity could lead to incorrect results potentially. Like the truncation right now etc. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
