dtenedor opened a new pull request, #12278: URL: https://github.com/apache/arrow-rs/pull/12278
# Which issue does this PR close? None. # Rationale for this change The BYTE_STREAM_SPLIT decoder's join loop indexes `src[i + j * stride]` and `dst[i * TYPE_SIZE + j]`. For 8- and 16-byte types those bounds checks stay in the loop and it runs scalar, at 2.3 ns per f64. For f32 the same loop happens to vectorize, at under 0.2 ns per value. # What changes are included in this PR? For types 8 bytes and wider, the decoder reads from one slice per stream, each exactly as long as the output. Every index is then provably in bounds, and the loop vectorizes. Narrower types keep the existing loop, because the per-stream form made f32 decoding 3.5x slower. The encoder is unchanged. Neither this form nor an explicit 8x8 byte transpose in `u64` registers made it faster. # Are these changes tested? Covered by the existing byte stream split tests. Decoded output was also checked to be bit-identical to main's for f32, f64, i32, i64 and FLBA widths 2, 3, 4, 8 and 16, at lengths around multiples of 8 and 64. Intel Xeon Platinum 8559C, default x86-64 target, median of 5 alternating rounds: | Benchmark | main | PR | Speedup | | :--- | ---: | ---: | ---: | | `encoding`: decoding f64 | 38.13 µs | 20.56 µs | 1.85x | | `encoding`: decoding FixedLenByteArray(16) | 314.76 µs | 292.62 µs | 1.08x | | `encoding`: decoding f32, FixedLenByteArray(2) | | | 1.00x | | `encoding`: encoding, all types (unchanged code) | | | 1.00-1.06x | | `arrow_reader`: Int64Array / UInt64Array `byte_stream_split encoded` | | | 1.35-1.36x | | `arrow_reader`: Int32Array `byte_stream_split encoded` | | | 1.00x | None of the measured inputs got slower. Not measured on aarch64. # Are there any significant user-facing changes? No. # AI usage disclosure An AI coding agent (Claude Code) generated the code and this description, and ran the benchmarks and the output checks. This PR is a draft until the submitter has reviewed it. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
