wesm commented on pull request #7514: URL: https://github.com/apache/arrow/pull/7514#issuecomment-650151220
> @wesm I don't know what you mean by the `BitBlockCounter` treatment. Ah sorry, welcome back =) We created some facilities to improve the performance of processing validity bitmaps https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/bit_block_counter.h The idea is that you can scan forward 64 bits at a time and determine efficiently (using hardware popcount) whether a "block" of 64 values has any nulls or not (or if the block is all null, too). For those 64 values you can skip null checking all together and use the fast path. This speeds up processing _significantly_ for data that is mostly not null or mostly null ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
