mightsleep opened a new pull request, #11527: URL: https://github.com/apache/arrow-rs/pull/11527
# Which issue does this PR close? - Part of #11437. # Rationale for this change `merge` with one scalar side is what DataFusion runs for `CASE WHEN c THEN expr ELSE literal END`, and today it extends `MutableArrayData` once per row for the scalar side, about 5 to 6 ns per row. The existing benchmark covers array against non-null scalar for i32 and the byte types on random masks. Before changing the kernel, this adds the shapes the next PR (primitives and decimals) needs to be measured on: - array without nulls against a scalar, and array against a NULL scalar (`CASE WHEN ... THEN array END`) - masks made of runs (average 64 and 512, 50 % and 10 % true), as from a clustered or sorted column - Decimal128 Boolean and Utf8View follow in a small benchmark PR before the PRs that change those paths. # What changes are included in this PR? - `create_clustered_array`, the same run model as the `filter_bits` benchmark from #11271, and four run masks. - The scalar-side scenarios run on random and run masks for every type. Decimal128 runs only those scenarios, the existing types keep all of them. Existing benchmark ids are unchanged. - 500 ms warm up and 1 s of measurement per group instead of 3 s and 5 s. One merge of 8192 rows takes microseconds, and 448 cases at the default would run about an hour. With this the whole suite takes about 16 minutes. An A/A run of the i32 group moved by 1.6 % on average. Baseline on main: i32 with a scalar side 5 to 6 ns per row, a NULL scalar 8.7 ns per row, and a run mask (`runs_of_512_10pct_true`) still 43 µs for 8192 rows, as the kernel goes row by row inside runs too. # Are these changes tested? Benchmark only. `cargo bench -p arrow --bench merge_kernels --features test_utils` runs all 448 cases. # Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
