This is an automated email from the ASF dual-hosted git repository.
Jefffrey pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
The following commit(s) were added to refs/heads/main by this push:
new 3abbc04e42 Allow cross-crate inlining for `BitSliceIterator` (#10588)
3abbc04e42 is described below
commit 3abbc04e429d8d13f3dae1e0086f23142ad8cec9
Author: Connor Tsui <[email protected]>
AuthorDate: Tue Aug 18 02:28:44 2026 -0400
Allow cross-crate inlining for `BitSliceIterator` (#10588)
## Which issue does this PR close?
- Closes #10587.
## Rationale for this change
Downstream crates built without LTO cannot inline the `BitSliceIterator`
hot path. This change also makes `BitSliceIterator::next` consistent
with the annotated `next` implementations on `BitIterator` and
`BitIndexIterator`.
[Vortex](https://github.com/vortex-data/vortex) discovered this gap
while consuming `arrow-buffer` 58.4.0 from a separate crate.
## What changes are included in this PR?
Adds `#[inline]` to `BitSliceIterator::{new, advance_to_set_bit, next}`
and `UnalignedBitChunk::iter`.
## Are these changes tested?
Yes.
<details>
<summary>Instruction-count measurements</summary>
[Vortex's](https://github.com/vortex-data/vortex) original
`arrow-buffer` 58.4.0 experiment measured 13,729 marginal instructions
per iteration unmodified and 8,565 with fat LTO and one codegen unit.
Annotating only `BitSliceIterator::next` reduced the count to 13,151,
while annotating the broader pre-bisection set reduced it to 10,026. The
bisection on current `main` narrows that set to the four methods in this
PR.
A local reproduction on current `main` measured the proposed change with
a separate consumer crate:
| Methods marked `#[inline]` | Marginal instructions per iteration |
| ---------------------------- | -----------------------------------: |
| None | 15,028 |
| Four methods in this change | 7,584 |
Removing any one of the four attributes gives up part of the
improvement:
| Method without `#[inline]` | Marginal instructions per iteration |
| -------------------------------------------- |
-----------------------------------: |
| `BitSliceIterator::new` | 8,898 |
| `BitSliceIterator::advance_to_set_bit` | 14,015 |
| `BitSliceIterator::next` | 15,006 |
| `UnalignedBitChunk::iter` | 8,448 |
`UnalignedBitChunk::new` was also tested because it constructs this
path. Adding it to the four-method set increases the result from 7,584
to 8,331 instructions, or 9.8%, so this PR does not annotate it.
The consumer iterates contiguous true runs over a 16,384-bit buffer at
1% density. It uses `opt-level = 3`, 16 codegen units, and no LTO.
Callgrind counts from runs of 1,000 and 2,000 iterations were
differenced to remove startup and setup instructions.
These are instruction counts, not wall-clock measurements.
</details>
On aarch64 macOS, the optimized `arrow-buffer` rlib changes from
1,799,592 to 1,808,184 bytes, an increase of 8,592 bytes or 0.48%. The
median release compile time across five forced builds with warm
dependencies remains 1.19 seconds.
## Are there any user-facing changes?
No.
Signed-off-by: Connor Tsui <[email protected]>
---
arrow-buffer/src/util/bit_chunk_iterator.rs | 1 +
arrow-buffer/src/util/bit_iterator.rs | 3 +++
2 files changed, 4 insertions(+)
diff --git a/arrow-buffer/src/util/bit_chunk_iterator.rs
b/arrow-buffer/src/util/bit_chunk_iterator.rs
index 2c4e684917..460349e7a6 100644
--- a/arrow-buffer/src/util/bit_chunk_iterator.rs
+++ b/arrow-buffer/src/util/bit_chunk_iterator.rs
@@ -157,6 +157,7 @@ impl<'a> UnalignedBitChunk<'a> {
}
/// Returns an iterator over the chunks
+ #[inline]
pub fn iter(&self) -> UnalignedBitChunkIterator<'a> {
self.prefix
.into_iter()
diff --git a/arrow-buffer/src/util/bit_iterator.rs
b/arrow-buffer/src/util/bit_iterator.rs
index c2e65de0c8..6286e8f42b 100644
--- a/arrow-buffer/src/util/bit_iterator.rs
+++ b/arrow-buffer/src/util/bit_iterator.rs
@@ -195,6 +195,7 @@ pub struct BitSliceIterator<'a> {
impl<'a> BitSliceIterator<'a> {
/// Create a new [`BitSliceIterator`] from the provided `buffer`,
/// and `offset` and `len` in bits
+ #[inline]
pub fn new(buffer: &'a [u8], offset: usize, len: usize) -> Self {
let chunk = UnalignedBitChunk::new(buffer, offset, len);
let mut iter = chunk.iter();
@@ -215,6 +216,7 @@ impl<'a> BitSliceIterator<'a> {
///
/// Where `chunk_offset` is the bit offset to the current `u64` chunk
/// and `bit_offset` is the offset of the first `1` bit in that chunk
+ #[inline]
fn advance_to_set_bit(&mut self) -> Option<(i64, u32)> {
loop {
if self.current_chunk != 0 {
@@ -232,6 +234,7 @@ impl<'a> BitSliceIterator<'a> {
impl Iterator for BitSliceIterator<'_> {
type Item = (usize, usize);
+ #[inline]
fn next(&mut self) -> Option<Self::Item> {
// Used as termination condition
if self.len == 0 {
