mbutrovich commented on code in PR #26507:
URL: https://github.com/apache/datafusion/pull/26507#discussion_r4360493252
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -1266,6 +1266,94 @@ pub fn offset_span<O: ArrowNativeType>(offsets:
&OffsetBuffer<O>) -> (usize, usi
(start, offsets.last().as_usize() - start)
}
+/// Counts the set bits that precede a sequence of non-decreasing positions in
a
+/// bitmap, reading each 64-bit word of the bitmap once.
+///
+/// This suits a single pass over the rows of a list, where each call costs a
+/// mask and a popcount when the position is in the current word. `words`
yields
+/// the bitmap's 64-bit words, starting with its first bit, such as
+/// [`BooleanBuffer::bit_chunks`] with [`BitChunks::iter_padded`].
+///
+/// [`BooleanBuffer::bit_chunks`]: arrow::buffer::BooleanBuffer::bit_chunks
+/// [`BitChunks::iter_padded`]:
arrow::util::bit_chunk_iterator::BitChunks::iter_padded
+///
+/// ```
+/// # use arrow::buffer::BooleanBuffer;
+/// # use datafusion_common::utils::SetBitCounter;
+/// let bits = BooleanBuffer::from(vec![true, false, true, true, false]);
+/// let mut counter = SetBitCounter::new(bits.bit_chunks().iter_padded());
+/// assert_eq!(counter.count_before(1), 1);
+/// assert_eq!(counter.count_range(1, 5), 2);
+/// ```
+pub struct SetBitCounter<I> {
Review Comment:
> Because `SetBitCounter` becomes public in `datafusion-common`, switching
later means a deprecation cycle.
On second thought, it doesn't. DataFusion's [API
policy](https://github.com/apache/datafusion/blob/ebf48581898530f01d36e5580a29c8b8df3012de/docs/source/contributor-guide/api-health.md?plain=1#L50-L80)
covers items that are `pub` only because DataFusion crates share them: marked
`#[doc(hidden)]` with the "Public Only for Internal Use" note, they can be
removed without a deprecation period. `SetBitCounter` and `rows_have_nulls` are
`pub` so that `datafusion-functions-nested` and `datafusion-spark` can call
them. Could you mark both that way? Switching to an arrow-rs API later would
then be a plain dealetion. The rustdoc @comphead asked for on `new` still
applies, since hidden items keep their doc comments and doctests.
On the arrow-rs side, a new API can go into a [minor
release](https://github.com/apache/arrow-rs/blob/df4dc3ede59bd43b5f71847231212381d068bb72/CONTRIBUTING.md?plain=1#L208-L213).
60.1.0 is being cut now, but 60.2.0 (apache/arrow-rs#10526) is released from
`main` and has nothing scheduled yet, so it's a realistic target. DataFusion
would then only need to raise its `arrow` requirement to 60.2.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
