jayzhan211 commented on issue #24704: URL: https://github.com/apache/datafusion/issues/24704#issuecomment-5666235532
> Having nested Vec harm performance due to the extra indirection and add cost for the low cardinality case Two things that might address the low-cardinality concern: 1. **Flat until the first block fills.** State is a plain Vec<T> until it reaches block capacity B. At that point the Vec becomes block 0 as-is (no copy, since B is a power of two and doubling lands on it exactly), and further growth appends new blocks. Below B nothing changes from today: same Vec, same values[idx], same emit. 2. **The decode is cheap, and only happens once the state is big anyway.** With power-of-two B, block = idx >> k, offset = idx & mask, no division. If B is sized in bytes (~1â2 MB per block, i.e. 128Kâ256K groups for a u64 state) instead of target_batch_size rows, then by the time we're in blocked mode the state no longer fits in cache and the update loop is bound by cache misses, not by the extra shift/mask. Block size and emit batch size don't need to be the same number; a block can be emitted as batch_size slices. Whether we're in flat or blocked mode is checked once per update_batch, not per row, so there's no branch in the hot loop. I will find time to check other sections -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
