linrrzqqq opened a new pull request, #70774:
URL: https://github.com/apache/doris/pull/70774

   ### Background
   
   Multiple consumers can access the same bitmap column. Measuring its 
serialized size optimized the underlying Roaring map in place, and copying a 
BitmapValue could also modify the source's sharing flag.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Bugfix
   
   Keep reads of a shared bitmap independent of other consumers while 
preserving compact output at the boundaries that need it.
   
   ### Details
   
   - **Change map:** Bitmap ownership, encoding and aggregate-state reads are 
in bitmap_value.h and bitmap_intersect.h. Producer, Segment and SerDe changes 
use these APIs. Block serialization prepares compact bitmap columns for 
Exchange.
   - **Frequency:** The race requires overlapping accesses to shared bitmap 
data. The previous optimization also ran during single-threaded sizing.
   - **Root cause:** Const size and copy operations could modify the source 
bitmap or its sharing state.
   - **Why existing tests missed it:** Sequential value checks exercise the 
contents of the set without checking concurrent access to its representation.
   - **Fix:** Detach shared Roaring maps on mutation. Size and write the 
current representation directly. Optimize completed aggregate results and 
bitmaps constructed from arrays or strings. When compact bytes are requested 
for an unoptimized shared input, size and write one local optimized copy.
   - **Other paths and branches:** Segment readers reuse compact native bitmap 
representations. Ordinary decoding skips optional length calculation. 
Intersection-state readers request the consumed byte count for both numeric and 
string keys. Exchange sizes and writes the same compact bitmap snapshot, 
including bitmaps nested in other columns.
   - **Behavior change:** An intermediate bitmap may retain its original layout 
until a compact output is requested. The encoding versions and bitmap set 
contents are preserved.
   
   ### Testing
   
   The Release build and 109 ASAN block and bitmap tests passed. The LZ4 block 
test checks both the allocation size and the input to the codec, round-trips a 
batch, and checks that the source stays unchanged. A targeted TSan run passed 
with four threads copying, sizing and writing the same source, checking both 
the original and compact bytes. Earlier validation included the bitmap scalar 
and bitmap_agg SQL suites.
   
   ### Release note
   
   Shared bitmap values can be sized and serialized concurrently without 
modifying their representation.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details).
   - Does this need documentation: No.
   - Pre-submission status: Ready.
   
   ### AI Disclosure
   
   - Tool: Codex (GPT-6).
   - Generated: Code, tests and commit message.
   - Human review: Pending.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to