WaterWhisperer commented on issue #10245: URL: https://github.com/apache/arrow-rs/issues/10245#issuecomment-6706270952
> I updated the checklist in the description to reflect current `main`: > > * Two callsites remain, both with open PRs: > > * `arrow-arith/src/arity.rs` `try_binary` nullable path ([perf: optimize loops for primitive types binary kernels. #10518](https://github.com/apache/arrow-rs/pull/10518)) > * `arrow-select/src/zip.rs` byte-array path ([Replace OffsetBufferBuilder with Vec in scalar byte-array zip #10913](https://github.com/apache/arrow-rs/pull/10913)) > > Once those two PRs are merged (or otherwise resolved) we can close this epic 🎉 I looked into the remaining nullable `try_binary` callsite referenced in #10518. The local patch only replaces `BufferBuilder` with `vec![O::Native::default(); len]` and converts the result with `.into()`. A focused benchmark on base commit showed repeatable regressions in some cases. It calls `try_binary` directly with checked addition, fixed seeded inputs, and nulls on the left input only. Input generation is outside the timed operation. Environment: Intel i7-13700H, Linux x86_64, Rust 1.99.1, glibc 2.39, default release settings and system allocator. Both versions were pinned to the same CPU core, with the powersave governor unchanged. Each case was measured six times per version, alternating execution order and shuffling case order between rounds, using 100 Criterion samples and a 2-second measurement target per run. Selected results below are average execution times across the six runs: - Decimal256, 8,192 rows, 10% nulls: 30.03 → 33.33 µs, **11.0% slower**. - Decimal256, 65,536 rows, 10% nulls: 284.00 → 305.35 µs, **7.5% slower**. - Decimal256, 1,048,576 rows, 10% nulls: 17.42 → 17.68 ms, **1.4% slower**. - Int32, 128 rows, all null: 87.64 → 101.36 ns, **15.7% slower**. For Decimal256 with 50% nulls, the corresponding changes at those three batch sizes were +10.3%, +6.5%, and +0.8%. Larger Int32 cases were generally close to unchanged. The Decimal256 regressions at 8,192 and 65,536 rows occurred in all six comparisons. At one million rows, the differences were much smaller. One unchanged no-null Decimal256 control also showed an approximately 10% swing in one comparison, so I would treat small differences cautiously. These results are limited to this machine, toolchain, and workload. The nullable benchmark mentioned in #10518 was not included in that PR, so I cannot compare the two setups directly or explain its reported gains conclusively. Would it make sense to keep `BufferBuilder` at this callsite for now, pending results from other environments or a replacement with more consistent benefits? I have retained the focused benchmark and raw results locally and can share them for reproduction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
