mbutrovich commented on code in PR #26507:
URL: https://github.com/apache/datafusion/pull/26507#discussion_r4360495918
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -1266,6 +1266,94 @@ pub fn offset_span<O: ArrowNativeType>(offsets:
&OffsetBuffer<O>) -> (usize, usi
(start, offsets.last().as_usize() - start)
}
+/// Counts the set bits that precede a sequence of non-decreasing positions in
a
+/// bitmap, reading each 64-bit word of the bitmap once.
+///
+/// This suits a single pass over the rows of a list, where each call costs a
+/// mask and a popcount when the position is in the current word. `words`
yields
+/// the bitmap's 64-bit words, starting with its first bit, such as
+/// [`BooleanBuffer::bit_chunks`] with [`BitChunks::iter_padded`].
+///
+/// [`BooleanBuffer::bit_chunks`]: arrow::buffer::BooleanBuffer::bit_chunks
+/// [`BitChunks::iter_padded`]:
arrow::util::bit_chunk_iterator::BitChunks::iter_padded
+///
+/// ```
+/// # use arrow::buffer::BooleanBuffer;
+/// # use datafusion_common::utils::SetBitCounter;
+/// let bits = BooleanBuffer::from(vec![true, false, true, true, false]);
+/// let mut counter = SetBitCounter::new(bits.bit_chunks().iter_padded());
+/// assert_eq!(counter.count_before(1), 1);
+/// assert_eq!(counter.count_range(1, 5), 2);
+/// ```
+pub struct SetBitCounter<I> {
+ /// The bitmap's remaining words, with the final word padded with zeros.
+ words: I,
+ /// The word containing the most recent position.
+ word: u64,
+ /// Position of the first bit of `word`.
+ word_start: usize,
+ /// Set bits before `word_start`.
+ count_before_word: usize,
+}
+
+impl<I: Iterator<Item = u64>> SetBitCounter<I> {
+ pub fn new(mut words: I) -> Self {
Review Comment:
> By reading, in a release build a call with `end < word_start` wraps `end -
self.word_start` and the `while` loop does not terminate in practice. Please
document it on the type, or use a cheap `assert!`. I did not run this.
I ran it against a copy of `SetBitCounter` from this PR, built with `rustc
-O`. Over three all-ones words, `count_before(130)` returns `130`, and a
following `count_before(10)` had not returned after 20 seconds. Should this be
an `assert!` now that other crates call it? It adds a compare per call, so a
`vector_math` run would show whether it costs anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
