Hello Prajwal, Thank you for the detailed proposal. I enjoyed reading it and have a few questions/comments.
1. On motivation, can you provide context on how this differs with aggregation merge engine’s roaring bitmap implementation [1]? Specifically, motivation part states that “users requiring high cardinality unique counting … must execute client-side aggregation”. Aggregation merge engine performs aggregation on server-side. The motivation section should clarify how the proposed changes improve or complement aggregation merge engine, which seems to have been considered as Section 2 references FIP-21 Aggregation Merge Engine. Adding this context will help readers understand the motivation of the proposal better. 2. Can you clarify the NULL semantics section specifically on the decision on why BITMAP_OR(bitmap, NULL) returns NULL but BITMAP_OR_AGG only returns null when all rows are NULL? 3. Why is the scope limited to 32 bit bitmaps? Adding the rationale behind these e.g. how (if any) support of 64bit bitmaps would increase implementation complexity. Articulating these may help other contributors understand the complexity and perhaps come up with suggestions on how to address them. Best regards Keith Lee [1] https://fluss.apache.org/blog/releases/0.9/#2-storage-level-processing--semantics On Mon, 9 Mar 2026 at 05:31, Prajwal Banakar <[email protected]> wrote: > Hi Devs, > > I have pushed a working prototype to my public fork demonstrating the > BitmapType integrated with FieldRoaringBitmap32Agg. This includes four > passing unit tests. > > The link to the prototype is available in the Google Doc, and you can also > find it here: > https://github.com/Prajwal-banakar/fluss/tree/RoaringBitmap-prototype > > The Google Doc link remains the same. I look forward to your feedback. > > Best regards, > > Prajwal Banakar > > > On Sun, 1 Mar, 2026, 11:49 am Prajwal Banakar, <[email protected] > > > wrote: > > > Hi everyone, > > > > I would like to start a discussion on the proposal for Native Bitmap > > Integration & Stateless Pushdown Aggregation. > > > > This proposal enables end-to-end native support for the BITMAP type in > > Fluss and integrates it with the existing aggregation merge engine to > > support server-side bitmap union pushdown. The goal is to reduce network > > transfer and offload DISTINCT-style aggregation from Flink to the > > TabletServer. > > > > Key highlights of the proposal include: > > > > - Type System: Promoting BITMAP to a first-class logical type. > > - UDF Suite: Introducing BITMAP_BUILD, BITMAP_OR_AGG, and > > BITMAP_CARDINALITY (aligned with FLIP-556 and StarRocks semantics). > > - Optimizer: Planner-based pushdown via applyAggregates in the Flink > > connector. > > - Safety: No changes to LogRecordBatch or WAL, making this strictly > > additive and migration-free. > > > > You can find the full proposal document here: > > > > > https://docs.google.com/document/d/1sDhfkmo-w-UTvo2n3rsY1lytSSryswfkI83cSdka8s0/edit?usp=sharing > > > > I would appreciate feedback on the public interfaces, pushdown > > constraints, and overall scope. > > > > Best regards, > > Prajwal Banakar > > >
