mrhhsg opened a new pull request, #68318:
URL: https://github.com/apache/doris/pull/68318
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: `Counts::merge` (the sample container behind `percentile`,
`percentile_array` and `PERCENTILE_STATE/UNION/MERGE`) only moved the raw
`_nums` of the source state into `_sorted_nums_vec`. Two in-memory merges
were therefore wrong:
1. The source state was itself a merged state: its samples live only in
`_sorted_nums_vec`, `_nums` is empty, so every sample was dropped.
2. The destination still held raw samples in `_nums`: `terminate` with a
single sorted run overwrote `_nums` with that run, the multi-run path
ignored `_nums`, `serialize` ignored `_sorted_nums_vec` when `_nums` was
non-empty, and unsorted raw samples could be pushed as a "sorted" run.
Bucketed hash aggregation merges per-instance hash-table states directly in
the source operator, which hits both cases. For example, with rows
`(shard, v) = (0,0),(1,0),(1,10)`, `agg_phase=1`,
`enable_bucketed_hash_agg=true` and `parallel_pipeline_task_num=4`,
SELECT PERCENTILE_MERGE(s) FROM (
SELECT shard, PERCENTILE_UNION(PERCENTILE_STATE(v, 0.625)) s
FROM t GROUP BY shard) q;
returned 0 or 6.25 non-deterministically instead of 2.5. Plain
`percentile(v, q) ... GROUP BY` under bucketed aggregation lost samples too.
Fix: `merge` now moves the raw samples of both sides (sorted if needed) into
`_sorted_nums_vec` and takes over all sorted runs of the source.
`serialize` and `terminate` fold any remaining raw samples into the sorted
runs before merging them. The serialized format is unchanged.
### Release note
Fix wrong and non-deterministic results of percentile / percentile_array /
PERCENTILE_UNION when states are merged by bucketed hash aggregation.
### Check List (For Author)
- Test:
- Unit Test: PercentileUtilTest (3 new cases fail before the fix, all 16
pass after)
- Regression test: query_p0/aggregate/percentile_bucketed_agg_merge
(fails
3/3 runs on an unfixed BE, passes 3/3 with the fix; results match the
non-bucketed controls)
- Manual test: the reproduction above returns 2.5 in 8/8 runs
- Behavior changed: Yes (percentile results under bucketed hash aggregation
are now correct)
- Does this need documentation: No
https://claude.ai/code/session_01Afs6xcnnuT7cUpnkiLEJ11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
