baibaichen opened a new pull request, #13364: URL: https://github.com/apache/gluten/pull/13364
## What changes are proposed in this pull request? Add catalog-driven microbenchmarks comparing Spark JVM and Velox execution for scalar expressions, aggregates and single-specification window functions. Markdown defines cases and input generators; CLI/JSON options control execution, with optional async-profiler output. Both engines plan from the same optimized logical plan. JVM execution pulls local Spark RDD iterators; native execution uses ExprSet, GroupingSet and iterator-driven Sort/Window operators. Stages use local shuffle files without submitting Spark jobs. Extract shared Velox query configuration to preserve production session settings, including independent nondeterministic expression state. Include correctness comparisons, lifecycle checks, a launcher and usage documentation. This measures local execution paths, not end-to-end SQL. Current limits: at most one Exchange, one actual reducer and one window specification. Native Partial does not implement production adaptive abandonment or memory-budget adjustment; selected known HLL and collection-order differences skip result comparison. ## How was this patch tested? - Latest rebased HEAD (67facf8a9), Spark 4.1 / Scala 2.13 / JDK 17: eight benchmark/helper suites, **383 passed, 0 failed, 20 ignored**, no canceled or aborted tests. - After the previous rebase: ScalarFunctionsValidateSuite.monotonically_increasing_id passed with ANSI disabled, covering single/repeated calls and asserting native projection execution; validates the query-configuration conflict resolution. - Earlier validation: native build, Spark 3.5 / Scala 2.12 / JDK 8 reactor test compilation and window CLI smoke tests passed. Spark 3.5 was not rerun after the latest rebase. - Manual scalar, aggregate and window performance runs and targeted CPU profiles completed. These are local measurements, not CI performance guarantees. Possible daemon-thread leak warnings remain. Window memory-pressure reclamation was not validated. Draft for CI feedback; no unrelated full-repository test run. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (model version not exposed in this session) Earlier commits also record Claude Code and GitHub Copilot co-authorship; their model versions are not recorded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
