Have you though about implementing something like RoaringBatchIterator ( https://richardstartin.github.io/posts/roaringbitmap-performance-tricks#batch-iteration). Maybe something like PositionBatchIterator which uses that in the background could help. 🤔
On Fri, Sep 11, 2026, 16:49 전대홍 <[email protected]> wrote: > Short version, since my original was too long - thanks to the reviewer on > the PR for saying so. > > Problem Spark's vectorized reader asks the position delete index > "is row N deleted?" once per row - 5,000 times per batch - > while those positions are a contiguous ascending range. It > re-resolves the same Roaring container every time. > > Change Pass the range once. PositionDeleteIndex gets a > forEachInRange default method; the default implementation is > the current per-row loop, so nothing external has to change. > ColumnarBatchUtil takes it only when there are no equality > deletes. > > Effect 2.6-9.3x less delete-check CPU. End to end, 13-19% on narrow > all-integer projections, up to 29% for a single column on a > dedicated instance, and inside my noise floor at 10+ columns. > One rule: it pays off than ~21% > of scan CPU. > > Links PR https://github.com/apache/iceberg/pull/18027 > Data https://github.com/apache/iceberg/issues/18026 > > The part I would most like opinions on is the interface addition itself - > whether growing PositionDeleteIndex with another default method is the > right call, or whether the traversre. > > One correction to how this is easy to read: more deletes is not more > benefit. The largest gain is just below 6.25%, where the Roaring container > is still a sorted array at its deepest; past that it flips to a bitmap and > gets cheaper on its own. > >>
