mbutrovich commented on code in PR #11818:
URL: https://github.com/apache/arrow-rs/pull/11818#discussion_r4443577905
##########
arrow-buffer/src/util/bit_chunk_iterator.rs:
##########
@@ -168,10 +170,46 @@ impl<'a> UnalignedBitChunk<'a> {
/// Counts the number of ones
pub fn count_ones(&self) -> usize {
- self.iter().map(|x| x.count_ones() as usize).sum()
+ let edges = self.prefix.into_iter().chain(self.suffix);
+ edges.map(|x| x.count_ones() as usize).sum::<usize>() +
count_ones_words(self.chunks)
Review Comment:
> does it matter that this seemingly will now read the first byte/word and
then the last byte/word and then the middle?
It doesn't read memory out of order. `prefix` and `suffix` are `Option<u64>`
fields that `UnalignedBitChunk::new` already loaded
([bit_chunk_iterator.rs](https://github.com/apache/arrow-rs/blob/110c2e8254c12c7a2ed03a55a3328df86ec92306/arrow-buffer/src/util/bit_chunk_iterator.rs#L37-L39)),
so `count_ones` reads only `chunks` from the buffer. I tried prefix, chunks,
suffix to match `iter()`, but it was 1.05x to 1.17x slower at 448, 960, and
1024 bits on both architectures
([aarch64](https://github.com/apache/arrow-rs/pull/11818#issuecomment-6579864744),
[aarch64
aligned](https://github.com/apache/arrow-rs/pull/11818#issuecomment-6579867554),
[x86_64](https://github.com/apache/arrow-rs/pull/11818#issuecomment-6579882403),
[x86_64
aligned](https://github.com/apache/arrow-rs/pull/11818#issuecomment-6579870403)),
so I kept the original order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
