jacktengg opened a new pull request, #69012:
URL: https://github.com/apache/doris/pull/69012
### What problem does this PR solve?
Issue Number: N/A
Related PR: #68312, #68219
Problem Summary: With four hash buckets and four pipeline tasks, a group
containing bitmap inputs {1,2}, NULL and {2,3} returns {2} in one-stage
aggregation but an empty bitmap in two-stage aggregation. Filtering NULL
restores {2}. The bitmap-only intermediate representation drops is_first, so a
NULL-only partial state becomes indistinguishable from an initialized empty
bitmap and erases the valid intersection during merge. The recent aggregate
state reader and negative integer fixes leave this behavior intact.
Add bitmap_intersect_v2 with a serialized initialization flag and reuse the
standard string state readers. Skip uninitialized source states during merge
while retaining the intersection semantics of a genuine empty bitmap. Map
BITMAP_INTERSECT SQL to the new implementation and preserve the original BE
implementation for old plans and stored states. Resolve new AGG_STATE type
declarations at the SQL parser boundary so persisted bitmap_intersect columns
retain their original decoder; expose bitmap_intersect_v1 for those states.
### Release note
BITMAP_INTERSECT now ignores NULL-only partial states during multi-stage
aggregation. Existing bitmap_intersect AGG_STATE columns retain their original
format and can be read with bitmap_intersect_v1_merge. The original and v2
state formats are distinct and mixing them reports a type mismatch.
### Check List (For Author)
- Test: Regression test / Unit Test / Manual test
- ASAN BE and FE builds completed through build.sh.
- 16 BE bitmap unit tests passed through run-be-ut.sh, including both
NULL wrappers, direct and serialized merges, selected rows, streaming, empty
bitmaps and initialization restored before add().
- 38 FE DataTypeTest tests passed through run-fe-ut.sh, including state
alias resolution and preservation of historical catalog types.
- Three regression suites passed through run-regression-test.sh:
bitmap_functions/test_bitmap_intersect_null_partial,
bitmap_functions/test_bitmap_intersect and
correctness_p0/test_bitmap_intersect. New output was generated by the standard
regression runner and then checked in comparison mode.
- The original three SQL queries all return {2}. A state table created
before upgrading remains readable through bitmap_intersect_v1_merge.
- clang-format 16 checks, BE header hygiene, clang-tidy on changed lines
and source whitespace checks passed; FE Checkstyle passed during the builds.
Runner-generated .out files retain their empty TSV fields and separators.
- Behavior changed: Yes, NULL-only partial states no longer erase a bitmap
intersection. The SQL function selects the v2 implementation; original stored
states use the explicit v1 entry point.
- Does this need documentation: No, the existing NULL-ignoring contract is
restored; state format compatibility is described in the release note.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
