u70b3 opened a new pull request, #4972: URL: https://github.com/apache/datafusion-comet/pull/4972
## Which issue does this PR close? Part of #4942. ## Rationale for this change `spark_unscaled_value` is injected by Spark's `DecimalAggregates` rule for every `sum`/`avg` over narrow decimals, so it sits on the TPC-DS decimal aggregation hot path. The array branch used a per-element `Int64Builder` loop. ## What changes are included in this PR? - Replace the per-element builder loop with the vectorized Arrow `unary` kernel: the Decimal128 values buffer is mapped to Int64 in a single pass and the null buffer is carried over untouched (zero-copy). The scalar branch is unchanged. - Add a criterion benchmark (`benches/unscaled_value.rs`) covering no-null / sparse-null (1 in 10) / dense-null (1 in 2) shapes at 8192 rows, following the scalar expression optimization guide. Benchmark results (8192 rows, Decimal128(20,2), median): | shape | before | after | speedup | |---|---|---|---| | no nulls | 29.5 µs | 3.2 µs | ~9.3x | | sparse nulls (1/10) | 35.5 µs | 3.0 µs | ~11.7x | | dense nulls (1/2) | 30.5 µs | 2.9 µs | ~10.4x | ## How are these changes tested? - Added unit tests for the array path (values and null distribution are bit-exact, including nulls) and the scalar path; the function previously had no direct tests. - Full `datafusion-comet-spark-expr` suite passes (497 tests), plus `clippy --all-targets -D warnings` and `fmt`. - Benchmark shows a strict improvement on all three shapes with no regression (criterion change −89% to −92%, p=0.00). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
