timmyb32r commented on PR #11879:
URL: https://github.com/apache/arrow-rs/pull/11879#issuecomment-6878693638

   @mbutrovich Thanks for all the work you put into exploring the alternatives, 
running the benchmarks, and explaining the results. It’s been really helpful.
   
   I’ve added the benchmarks you requested as the first commit and added 
`Closes #12136` to the description.
   
   I also explored another approach that avoids `padding_dirty`. My preference 
would be to avoid adding persistent builder state solely to recover 
performance: it introduces another invariant that the mutation methods need to 
maintain.
   
   The idea is to initialize new bytes directly and combine clearing with the 
value write for partial bytes:
   
   ```rust
   #[inline]
   pub fn append(&mut self, v: bool) {
       let index = self.len;
       if index & 7 == 0 {
           self.buffer.push(u8::from(v));
           self.len += 1;
           return;
       }
   
       self.len += 1;
       let bit = index & 7;
       let byte = &mut self.buffer.as_slice_mut()[index / 8];
       *byte = (*byte & ((1_u8 << bit) - 1)) | (u8::from(v) << bit);
   }
   ```
   
   This avoids the general growth calculation for seven out of eight appends. 
`append_slice` combines clearing with its first write. `append_word` retains 
its existing OR loop and clears dirty padding in the old partial byte before 
writing. Public `advance` also clears the old partial-byte padding.
   
   The implementation and regression tests are on a separate branch: **[full 
diff against the benchmark-only 
baseline](https://github.com/timmyb32r/arrow-rs/compare/551b03358...codex/boolean-builder-no-padding-flag)**.
   
   Local results on an **Apple M3 Pro, rustc 1.99.1, thin LTO, 16 codegen 
units**, using your per-value append benchmarks. Values are µs per 65,536 
appends, including builder creation and `finish`; the rows were measured in 
separate local runs.
   
   | Implementation | All false | All true | Random |
   |---|---:|---:|---:|
   | Main at the shared baseline + benchmarks | 49.24 | 50.90 | 217.05 |
   | Current fix in this PR | 65.78 | 65.58 | 65.62 |
   | Alternative without the flag | 49.18 | 49.36 | 48.31 |
   
   The full workspace tests, Clippy, and the targeted Miri regression tests 
pass. Your Xeon results showed why local measurements alone aren’t enough, 
particularly for all-false input, so I’d like to compare this approach on the 
benchmark bot.
   
   Would you prefer that I update this PR with the alternative, or put it in a 
separate draft PR for comparison?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to