timmyb32r commented on PR #11879:
URL: https://github.com/apache/arrow-rs/pull/11879#issuecomment-6878693638
@mbutrovich Thanks for all the work you put into exploring the alternatives,
running the benchmarks, and explaining the results. It’s been really helpful.
I’ve added the benchmarks you requested as the first commit and added
`Closes #12136` to the description.
I also explored another approach that avoids `padding_dirty`. My preference
would be to avoid adding persistent builder state solely to recover
performance: it introduces another invariant that the mutation methods need to
maintain.
The idea is to initialize new bytes directly and combine clearing with the
value write for partial bytes:
```rust
#[inline]
pub fn append(&mut self, v: bool) {
let index = self.len;
if index & 7 == 0 {
self.buffer.push(u8::from(v));
self.len += 1;
return;
}
self.len += 1;
let bit = index & 7;
let byte = &mut self.buffer.as_slice_mut()[index / 8];
*byte = (*byte & ((1_u8 << bit) - 1)) | (u8::from(v) << bit);
}
```
This avoids the general growth calculation for seven out of eight appends.
`append_slice` combines clearing with its first write. `append_word` retains
its existing OR loop and clears dirty padding in the old partial byte before
writing. Public `advance` also clears the old partial-byte padding.
The implementation and regression tests are on a separate branch: **[full
diff against the benchmark-only
baseline](https://github.com/timmyb32r/arrow-rs/compare/551b03358...codex/boolean-builder-no-padding-flag)**.
Local results on an **Apple M3 Pro, rustc 1.99.1, thin LTO, 16 codegen
units**, using your per-value append benchmarks. Values are µs per 65,536
appends, including builder creation and `finish`; the rows were measured in
separate local runs.
| Implementation | All false | All true | Random |
|---|---:|---:|---:|
| Main at the shared baseline + benchmarks | 49.24 | 50.90 | 217.05 |
| Current fix in this PR | 65.78 | 65.58 | 65.62 |
| Alternative without the flag | 49.18 | 49.36 | 48.31 |
The full workspace tests, Clippy, and the targeted Miri regression tests
pass. Your Xeon results showed why local measurements alone aren’t enough,
particularly for all-false input, so I’d like to compare this approach on the
benchmark bot.
Would you prefer that I update this PR with the alternative, or put it in a
separate draft PR for comparison?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]