andygrove opened a new pull request, #4873: URL: https://github.com/apache/datafusion-comet/pull/4873
## Which issue does this PR close? N/A. Documentation update. ## Rationale for this change The Comet vs Gluten comparison document referenced an older single-node TPC-H benchmark for performance and did not surface several of Comet's differentiating strengths. This updates the performance comparison to cite independent third-party benchmarking and adds sections covering areas where Comet has meaningful advantages. ## What changes are included in this PR? - **Performance**: Replace the single-node TPC-H numbers with a reference to the independent AWS Labs benchmark comparing Comet 0.16.0 with Gluten 1.6.0 on TPC-DS 3TB, which found similar overall performance with Gluten roughly 9% faster, and note the wide per-query variation and our expectation that the gap closes over time. - **Codegen Dispatch**: New section describing Comet's codegen dispatcher, which runs Spark's own generated code inside the native pipeline for correctness-by-default and to avoid full query-stage fallbacks. Gluten has no direct equivalent. - **Regular Expression Compatibility**: New section explaining that Comet is 100% compatible with Spark regex by default (via the codegen dispatcher running the JVM `java.util.regex` engine), whereas Velox's C++ RE2 engine cannot match Java regex semantics such as backreferences and lookaround. - **Scala and Java UDF Support**: New section noting that Comet runs existing JVM Scala/Java scalar UDFs inside the native pipeline without rewrites and without forcing the enclosing operator back to Spark. - **Underlying Execution Engine**: Expand the Rust vs C++ discussion to cover Rust's compile-time memory-safety guarantees. - **Community and Governance**: New section on Comet being built on Apache DataFusion, a vendor-neutral ASF community, versus Velox being primarily Meta-driven. - Update the Summary to reflect the revised performance framing and the additional strengths. ## How are these changes tested? Documentation only. No tests required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
