andygrove commented on PR #7544: URL: https://github.com/apache/datafusion-comet/pull/7544#issuecomment-6771476652
On the two points that were not tied to a line: I ran `CometNativeExecutorsSuite` against the cluster built from `native/native-executors` in this PR, with a libcomet built from this PR with the `native-executors` feature, on TPC-H data at scale factor 1. All 15 tests that are not TPC-H queries pass, and so do TPC-H q1, q6, q11 and q15. The other 18 TPC-H queries all join, and fail while the scheduler plans them with `CometFragmentExec input N has K partitions, expected 1`. The unmodified head fails the same way. This PR is the driver side split out of #7289, so I don't expect it to run TPC-H yet. The cluster that runs it is in #7289. The suite's header and the docs page now say which TPC-H tests need that cluster, and the PR description has the numbers. I added `CometNativeExecutorsPlanSuite`, which is in the regular CI matrix. It takes a normal Comet plan with AQE off and checks the stage graph (the scan stage's partition plans, hash, single-partition and broadcast inputs, a scan with no files), the refusals (a sort that stays in Spark, a hash exchange keyed by an expression), and the scalar subquery rewrite. It needs neither the feature-built libcomet nor the cluster binaries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
