andygrove commented on PR #4019: URL: https://github.com/apache/datafusion-comet/pull/4019#issuecomment-4459495347
> > It may make sense to break down #3720 into smaller categories. > > Sure, we can do that, but I don't think it changes the fact that we want to remove `native_iceberg_compat` scan. It has been deprecated since 0.15.0, is no longer tested in CI, and no longer works - many TPC-H queries fail if we select this scan. If we want to reverse course on removing it, we should re-enable all of the tests and fix the regressions. Note that this scan also has limitations that `native_datafusion` does not have. These are documented in the compatibility guide. It also has worse performance. > > Once #4229 is merged, I think we are down to 7 tests ignored, and these are edge/corner cases which are already documented in the compatibility guide. These could eventually be fixed with upstream changes to arrow-rs and datafusion if we really think these are worth the investment. I burned some tokens working on https://github.com/apache/datafusion-comet/pull/4229 overnight and maybe have all the tests unignored for 4.1.1 ... I'll keep my eye on CI today ... would be great to get reviews on that one so we can unblock this work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
