yujun777 opened a new pull request, #69456: URL: https://github.com/apache/doris/pull/69456
### What problem does this PR solve? Issue Number: N/A (tracked in https://github.com/apache/doris/issues/65418) Related PR: #69286 Problem Summary: This is PR 1 of the incremental MIN/MAX top-K work. It adds the four functions that the IVM delta/apply plan needs; the FE IVM integration is a separate PR. - `topk_min_count(x, k)` / `topk_max_count(x, k)`: an aggregate returning `STRUCT<values ARRAY<STRUCT<value T, count BIGINT>>, bound T NULL>` — the k smallest (largest) distinct non-NULL values with their exact row counts, ordered by value. `bound IS NOT NULL` is the truncation flag: it holds the k-th value when more than k distinct values were seen, and is NULL otherwise. With `k = 0` no list is built and `bound` is the extreme value of this side, so an append-only delta can be applied from the insert extreme alone. `k` is a constant `BIGINT` with `0 <= k <= 65536`. - `topk_merge_min` / `topk_merge_max`: the scalar apply step. Given the pre-image state `(S, b)`, the insert/delete top-K lists `(I, b_i)` and `(D, b_d)`, `n`, `dn` and `k`, it computes `bE = best(b, b_i, b_d)`, drops every value outside `bE` (the range whose counts both sides know exactly), computes `S + I - D`, removes non-positive counts, keeps the first k values and returns `(values, bound, n, exhausted)`. A negative count means the state invariant is broken and returns an internal error; `exhausted` asks the caller to fall back to a complete refresh. The aggregates keep a sorted `(value, count)` state with checked int64 counting, so partial aggregation and merge stay exact (`topK(A ∪ B) ⊆ topK(A) ∪ topK(B)`). FLOAT/DOUBLE order NaN as greater than every number so the distinct list keeps a strict order. The scalar disables the default NULL handling: a NULL list is an empty list, a NULL bound means "not truncated", and the result struct itself is never NULL. ### Release note Add the `topk_min_count`/`topk_max_count` aggregate functions and the `topk_merge_min`/`topk_merge_max` scalar functions. ### Check List (For Author) - Test: Regression test - `regression-test/suites/query_p0/aggregate/topk_min_max_count.groovy`: k = 0/1/2/8/65536/65537, empty input, all-NULL groups, NOT NULL input types, string/decimal/datetime, -0.0/NaN, agreement with MIN/MAX, two-phase partial+final merge, every `topk_merge_*` branch (pre-image/insert/delete bounds, cancellation, truncation, exhausted, all-NULL) and the error paths. - FE unit test `TopKFunctionSignatureTest`. - BE unit test `agg_topk_count_test.cpp` (k = 0, truncation, MIN/MAX merge, serialize/deserialize round-trip, count overflow). - Behavior changed: No (new functions only) - Does this need documentation: Yes (function docs will come with the IVM user documentation) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
