dshogan-amzn commented on PR #19015: URL: https://github.com/apache/iceberg/pull/19015#issuecomment-6993998778
> I like the idea, but I'm not sure it's worth the amount of new API surface. Could we focus on keeping the sorted ness as an implementation detail and out of the API? I'm also not clear on what your benchmarks are trying to show. - Updated the metrics to make it a bit more clear the added planning costs while maintaining the benefits. This should *hopefully* show that the suspected planning costs that caused the initial restriction to 200 can mostly be avoided and we can remove entirely. If not we should at least to be able to significantly raise it (10,000?) - For the API, the sorted-literal helper is now package-private in expressions, and the Parquet row-group filter has its own private copy, so the sorting is an implementation detail and this PR adds no public API. Let me know if that's all that was requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
