mightsleep opened a new pull request, #11559: URL: https://github.com/apache/arrow-rs/pull/11559
# Which issue does this PR close? - Part of #11437. # Rationale for this change `zip` with a scalar on one side extends `MutableArrayData` once per scalar row and once per run of array rows, 6 to 7 ns a row on random masks. Callers include DataFusion's Spark `%` and `pmod`, `unnest` and `union_extract`. # What changes are included in this PR? For a primitive array and a scalar of the same type: - Values are a branch-free select per row over blocks of 64 rows. Mask words that are all set or all clear copy or fill. 16 and 32 byte values are 2 or 4 `i64` lanes. - The lane masks are computed into an array before the select. Computed inside it, LLVM emits a branch per row, which mispredicts on random masks. - Nulls in closed form. With a null scalar the array's values buffer is shared. Other types keep the `MutableArrayData` path. # Are these changes tested? `test_zip_primitive_with_scalar_matches_general_path` compares against `zip_impl` for 8 types (including `Decimal256` and `Interval(MonthDayNano)`), with offsets, nulls in the array and the mask, a null and a valid scalar, on either side. ns per row, 8192 rows, a pool of 512 random masks so the branch predictor cannot learn them: | type | main | this PR | |---|---|---| | i8 | 6.0 to 7.3 | 0.32 | | i32 | 6.3 to 7.4 | 0.30 | | f64 | 6.7 to 7.6 | 0.38 | | Decimal128 | 6.3 to 7.4 | 0.66 | | i32, null scalar | 9 to 10 | 0.05 | `zip_kernels` (i32) on GitHub runners (Neoverse-N2, Apple M1, EPYC 9V74 default and x86-64-v3), main / this PR: 22 to 60x on random masks, 1.07 to 1.35x when every row comes from the array. # Are there any significant user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
