lzydmxy opened a new pull request, #66241:
URL: https://github.com/apache/doris/pull/66241
Add uniq_theta, a theta-sketch based approximate distinct-count aggregate
returning BIGINT, ported from ClickHouse uniqTheta and backed by the Apache
DataSketches library (contrib/datasketches-cpp submodule added in #63143). The
aggregate reuses the generic agg_state combinator framework so uniq_theta_state
/ uniq_theta_merge / uniq_theta_union are auto-derived with no extra wiring.
Includes BE gtest, FE unit test, and regression suites.
### What problem does this PR solve?
Issue Number: close #66240
Related PR: #63143
Problem Summary:
Doris only has HyperLogLog-based `approx_count_distinct` / `ndv`, which
supports
`union` but not clean intersection or difference on the serialized state.
This PR adds **`uniq_theta`**, an approximate distinct-count aggregate
backed by
the Apache DataSketches Theta Sketch (ported from ClickHouse `uniqTheta`).
Theta
sketches add capabilities HLL lacks: set operations (union / intersect /
difference) on the serialized sketch, precompute-once/query-many via
mergeable
`agg_state`, and cross-engine-compatible sketch bytes. Relative error is
~3.125%
(95% confidence, default 4096 nominal entries), exact for small
cardinalities.
The `contrib/datasketches-cpp` submodule and its build wiring were already
added
by #63143, so this PR only adds the aggregate itself.
### Release note
Add aggregate function uniq_theta, a Theta-sketch (Apache DataSketches)
based
approximate distinct-count function, together with its uniq_theta_state /
uniq_theta_merge / uniq_theta_union agg_state combinators.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
https://github.com/apache/doris-website/pull/4019
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
