toutane commented on PR #2671: URL: https://github.com/apache/iceberg-rust/pull/2671#issuecomment-5193387421
> I think you've done a great job pushing through on this topic. I know there have been many iterations and I think it's paid off because this looks like a very clean PR. I don't have committer access, so I can only provide an extra "approve" that doesn't carry weight. > > The DataFusion parts specifically look very good to me. I am unable to test at scale myself, so I will have to rely on others or on the included tests. > > The comments I have are all documentation, not code issues. @timsaucer, thanks a lot for your message, really appreciate it! For testing, we're going to shadow real traffic in our infra. It won't be a standard benchmark, but it should at least tell us whether things are improving on real-world queries. That said, I think having a standard way to benchmark this kind of optimization would be really valuable for the `iceberg-datafusion` crate. Not sure if that's already been discussed somewhere? Thanks again for the comments - I opened three issues to track the TODOs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
