mrhhsg opened a new pull request, #70670:
URL: https://github.com/apache/doris/pull/70670

   ### Background
   
   The legacy BE local-shuffle planner propagates shuffled-consumer and 
bucket-distribution requirements independently. Aggregation operators choose 
local-exchange partition expressions from FE distribution metadata or their 
grouping expressions.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Bugfix
   
   Problem Summary: A bucket-preserving aggregation can receive a 
bucket-distribution requirement without a shuffled-consumer requirement. If 
tablets are bucketed by `k1` but the aggregation groups by `(k1, k2)`, the 
legacy planner may hash by both grouping keys and route rows outside a BE's 
owned buckets. Use the FE distribution key whenever bucket distribution must be 
preserved, while keeping the existing grouping-key fallback.
   
   ### Details
   
   - **Frequency:** Multi-stage DISTINCT aggregations using legacy BE local 
shuffle when BEs own only a subset of the table's buckets.
   - **Root cause:** Ordinary and distinct-streaming aggregation selected FE 
distribution expressions only for a shuffled-consumer requirement, overlooking 
the independently propagated bucket-distribution requirement.
   - **Why existing tests missed it:** They did not combine a bucket-preserving 
child, a false shuffled-consumer flag, differing grouping and bucket keys, and 
partial bucket ownership.
   - **Fix:** Select FE distribution expressions when either shuffled-consumer 
or bucket-distribution is required; retain the original grouping-key fallback 
when FE distribution expressions are unavailable or neither requirement applies.
   - **Other paths and branches:** FE-planned and disabled local shuffle retain 
their existing behavior. This PR does not change release branches.
   
   ### Testing
   
   - Before the fix, reproduced `Rows mismatched / BUCKET_HASH_SHUFFLE` locally 
with three BEs, partial bucket ownership, and aggregation phases 3 and 4; 
control queries with local shuffle disabled or FE planning enabled returned the 
expected result.
   - Built BE and FE locally with ASAN, including FE checkstyle.
   - Ran focused BE aggregation, local-exchange, and pipeline unit tests: 19 
passed.
   - Generated and verified the SQL regression expected output, then repeated 
the suite five times; also verified it with two partial-bucket ownership 
layouts across three BEs. All runs passed.
   - Ran three adjacent local-shuffle suites and the four-phase aggregation 
suite; all passed.
   - Ran BE formatting, build hygiene, clang-tidy, and diff checks; all passed. 
Full CI, cloud mode, and mixed-version execution were not run.
   
   ### Release note
   
   Affected multi-stage DISTINCT aggregations no longer fail from incorrect 
bucket routing under legacy BE local shuffle.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details)
   - Does this need documentation: No
   - Pre-submission status: Partially verified: local ASAN build, focused unit 
tests, and three-BE regressions passed; full CI and cloud execution were not 
run.
   
   ### AI Disclosure
   
   - Tool: Codex
   - Generated: Code, tests, and this description
   - Human review: Pending


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to