This is an automated email from the ASF dual-hosted git repository.
tisonkun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datasketches-rust.git
The following commit(s) were added to refs/heads/main by this push:
new 3b555c7 refactor(req): remove ReqUnion (#242)
3b555c7 is described below
commit 3b555c70b0ba56598ccac814aaee50e64d67ae5a
Author: tison <[email protected]>
AuthorDate: Sat Aug 29 08:00:02 2026 +0800
refactor(req): remove ReqUnion (#242)
---
CHANGELOG.md | 4 +-
README.md | 26 +++----
datasketches/src/req/mod.rs | 2 -
datasketches/src/req/sketch.rs | 18 +++++
datasketches/src/req/union.rs | 89 ----------------------
tests-integration/tests/req_test/main.rs | 1 -
tests-integration/tests/req_test/union.rs | 121 ------------------------------
7 files changed, 33 insertions(+), 228 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8c62c90..02ffab6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,11 +9,11 @@ All significant changes to this project will be documented in
this file.
* `ThetaIntersection::to_sketch` and `TupleIntersection::to_sketch` now return
`Option`. Callers must handle `None` until the intersection receives its first
successful update.
* `BloomFilterBuilder`, `ThetaSketchBuilder`, `ThetaUnionBuilder`,
`TupleSketchBuilder`, and `TupleUnionBuilder` now validate their configuration
when `build` is called, and `build` returns `Result`. Callers must propagate or
handle construction errors.
* `BloomFilterBuilder::{MIN_NUM_BITS, MAX_NUM_BITS, MIN_NUM_HASHES,
MAX_NUM_HASHES}` are no longer public. Callers should pass configurations to
`build` and handle `InvalidArgument` instead of prevalidating against these
constants.
-* Fallible sketch and operator constructors now return `Result` directly from
`new` or `with_seed`. `ReqSketch`, `ReqUnion`, and `TDigestMut` no longer
provide `try_new`, and the Count-Min parameter suggestion methods also return
`Result`.
+* Fallible sketch and operator constructors now return `Result` directly from
`new` or `with_seed`. `ReqSketch` and `TDigestMut` no longer provide `try_new`,
and the Count-Min parameter suggestion methods also return `Result`.
### New features
-* Add Relative Error Quantiles (REQ) sketches behind the `req` feature,
including configurable high- or low-rank accuracy, rank, quantile, PMF, and CDF
queries, merging and unions, and C++/Java-compatible serialization.
+* Add Relative Error Quantiles (REQ) sketches behind the `req` feature,
including configurable high- or low-rank accuracy, rank, quantile, PMF, and CDF
queries, merging, and C++/Java-compatible serialization.
### Performance improvements
diff --git a/README.md b/README.md
index 35e964e..674d361 100644
--- a/README.md
+++ b/README.md
@@ -31,7 +31,7 @@
[docs-url]: https://docs.rs/datasketches
[msrv-badge]: https://img.shields.io/badge/MSRV-1.86.0-green?logo=rust
[license-badge]: https://img.shields.io/crates/l/datasketches
-[license-url]: LICENSE
+[license-url]: https://www.apache.org/licenses/LICENSE-2.0
[actions-badge]:
https://github.com/apache/datasketches-rust/actions/workflows/ci.yml/badge.svg
[actions-url]:
https://github.com/apache/datasketches-rust/actions/workflows/ci.yml
@@ -63,17 +63,17 @@ Enable multiple algorithms by listing their features
together, such as `features
## Available sketches
-| Feature | Main types | Use case |
-| --- | --- | --- |
-| `bloom` | `BloomFilter` | Space-efficient probabilistic set membership with
a configurable false-positive rate. |
-| `countmin` | `CountMinSketch` | Approximate point-frequency queries over a
stream. |
-| `cpc` | `CpcSketch`, `CpcUnion`, `CpcWrapper` | Highly compact
distinct-count estimation and unions. |
-| `frequencies` | `FrequentItemsSketch` | Heavy-hitter discovery with upper
and lower frequency bounds. |
-| `hll` | `HllSketch`, `HllUnion` | Fast distinct-count estimation and unions.
|
-| `req` | `ReqSketch`, `ReqUnion` | Relative-error quantile, rank, PMF, and
CDF queries with configurable high- or low-rank accuracy. |
-| `tdigest` | `TDigestMut`, `TDigest` | Quantile and rank estimation, with
high accuracy near distribution tails. |
-| `theta` | `ThetaSketch` and set operations | Distinct counts, set
expressions, and Jaccard similarity. |
-| `tuple` | `TupleSketch` and set operations | Theta-style keys with
user-defined summaries attached to retained entries. |
+| Feature | Main types | Use case
|
+| ------------- | ------------------------------------- |
-------------------------------------------------------------------------------------------------
|
+| `bloom` | `BloomFilter` | Space-efficient
probabilistic set membership with a configurable false-positive rate.
|
+| `countmin` | `CountMinSketch` | Approximate
point-frequency queries over a stream.
|
+| `cpc` | `CpcSketch`, `CpcUnion`, `CpcWrapper` | Highly compact
distinct-count estimation and unions.
|
+| `frequencies` | `FrequentItemsSketch` | Heavy-hitter
discovery with upper and lower frequency bounds.
|
+| `hll` | `HllSketch`, `HllUnion` | Fast distinct-count
estimation and unions. |
+| `req` | `ReqSketch` | Relative-error
quantile, rank, PMF, and CDF queries with configurable high- or low-rank
accuracy. |
+| `tdigest` | `TDigestMut`, `TDigest` | Quantile and rank
estimation, with high accuracy near distribution tails.
|
+| `theta` | `ThetaSketch` and set operations | Distinct counts, set
expressions, and Jaccard similarity. |
+| `tuple` | `TupleSketch` and set operations | Theta-style keys
with user-defined summaries attached to retained entries.
|
See the [API documentation](https://docs.rs/datasketches) for configuration,
accuracy guarantees, serialization, and examples for each algorithm.
@@ -106,4 +106,4 @@ To report a security vulnerability, follow the [ASF
security reporting process](
## License
-Licensed under the [Apache License, Version 2.0](LICENSE).
+Licensed under the [Apache License, Version 2.0][license-url].
diff --git a/datasketches/src/req/mod.rs b/datasketches/src/req/mod.rs
index 399236c..37f26ca 100644
--- a/datasketches/src/req/mod.rs
+++ b/datasketches/src/req/mod.rs
@@ -28,13 +28,11 @@ mod iter;
mod serialization;
mod sketch;
mod sorted_view;
-mod union;
mod value;
pub use self::iter::ReqSketchIterator;
pub use self::sketch::ReqSketch;
pub use self::sorted_view::SortedView;
-pub use self::union::ReqUnion;
pub use self::value::ReqValue;
/// Default value of `k` if not specified. Roughly 1% relative error at 95%
confidence.
diff --git a/datasketches/src/req/sketch.rs b/datasketches/src/req/sketch.rs
index 6c0249b..3ff0a53 100644
--- a/datasketches/src/req/sketch.rs
+++ b/datasketches/src/req/sketch.rs
@@ -269,6 +269,24 @@ impl<T: ReqValue> ReqSketch<T> {
/// # Errors
///
/// Returns an error if the two sketches have different `rank_accuracy`.
+ ///
+ /// # Examples
+ ///
+ /// ```
+ /// use datasketches::req::ReqSketch;
+ ///
+ /// let mut first = ReqSketch::default();
+ /// first.update(1.0_f64);
+ ///
+ /// let mut second = ReqSketch::default();
+ /// second.update(2.0_f64);
+ ///
+ /// let mut combined = ReqSketch::default();
+ /// combined.merge(&first).unwrap();
+ /// combined.merge(&second).unwrap();
+ ///
+ /// assert_eq!(combined.n(), 2);
+ /// ```
pub fn merge(&mut self, other: &Self) -> Result<(), Error> {
if self.rank_accuracy != other.rank_accuracy {
return Err(Error::invalid_argument(
diff --git a/datasketches/src/req/union.rs b/datasketches/src/req/union.rs
deleted file mode 100644
index b0a95f9..0000000
--- a/datasketches/src/req/union.rs
+++ /dev/null
@@ -1,89 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-//! REQ union — combines REQ sketches into a single result.
-
-use crate::error::Error;
-use crate::req::RankAccuracy;
-use crate::req::sketch::ReqSketch;
-use crate::req::value::ReqValue;
-
-/// Combines multiple [`ReqSketch`]es into a single result.
-///
-/// Internally wraps a `ReqSketch` configured for union semantics.
-#[derive(Debug, Clone)]
-pub struct ReqUnion<T: ReqValue> {
- inner: ReqSketch<T>,
-}
-
-impl<T: ReqValue> Default for ReqUnion<T> {
- fn default() -> Self {
- Self {
- inner: ReqSketch::default(),
- }
- }
-}
-
-impl<T: ReqValue> ReqUnion<T> {
- /// Creates a new union with the given `k` and rank accuracy.
- ///
- /// # Errors
- ///
- /// Returns an error if `k` is invalid (see [`ReqSketch::new`]).
- pub fn new(k: u16, rank_accuracy: RankAccuracy) -> Result<Self, Error> {
- Ok(Self {
- inner: ReqSketch::new(k, rank_accuracy)?,
- })
- }
-
- /// Returns the configured `k` parameter.
- pub fn k(&self) -> u16 {
- self.inner.k()
- }
-
- /// Returns the configured rank accuracy.
- pub fn rank_accuracy(&self) -> RankAccuracy {
- self.inner.rank_accuracy()
- }
-
- /// Returns true if the union has not yet absorbed any sketch.
- pub fn is_empty(&self) -> bool {
- self.inner.is_empty()
- }
-
- /// Merges a sketch into the union.
- ///
- /// # Errors
- ///
- /// Returns an error if the sketch's `rank_accuracy` differs from the
union's.
- pub fn merge(&mut self, sketch: &ReqSketch<T>) -> Result<(), Error> {
- self.inner.merge(sketch)
- }
-
- /// Extracts the merged result as a [`ReqSketch`].
- ///
- /// Equivalent to C++ `req_union::get_result`. Renamed per the workspace's
- /// CPC PR #81 precedent.
- pub fn to_sketch(&self) -> ReqSketch<T> {
- self.inner.clone()
- }
-
- /// Resets the union to empty.
- pub fn reset(&mut self) {
- self.inner.reset();
- }
-}
diff --git a/tests-integration/tests/req_test/main.rs
b/tests-integration/tests/req_test/main.rs
index 38de714..4633a38 100644
--- a/tests-integration/tests/req_test/main.rs
+++ b/tests-integration/tests/req_test/main.rs
@@ -23,4 +23,3 @@ mod property;
mod query;
mod sorted_view_api;
mod structure;
-mod union;
diff --git a/tests-integration/tests/req_test/union.rs
b/tests-integration/tests/req_test/union.rs
deleted file mode 100644
index 1469931..0000000
--- a/tests-integration/tests/req_test/union.rs
+++ /dev/null
@@ -1,121 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-//! Integration tests for ReqUnion.
-
-use datasketches::req::RankAccuracy;
-use datasketches::req::ReqSketch;
-use datasketches::req::ReqUnion;
-use datasketches::req::SearchCriteria;
-use googletest::assert_that;
-use googletest::prelude::anything;
-use googletest::prelude::err;
-use googletest::prelude::near;
-
-#[test]
-fn union_equivalent_to_chained_merge() {
- let make = |range: std::ops::Range<u64>| -> ReqSketch<f64> {
- let mut sketch = ReqSketch::default();
- for i in range {
- sketch.update(i as f64);
- }
- sketch
- };
- let s1 = make(0..1000);
- let s2 = make(1000..2000);
- let s3 = make(2000..3000);
-
- let mut union: ReqUnion<f64> = ReqUnion::default();
- union.merge(&s1).expect("merge should succeed");
- union.merge(&s2).expect("merge should succeed");
- union.merge(&s3).expect("merge should succeed");
- let union_result = union.to_sketch();
-
- let mut via_merge: ReqSketch<f64> = ReqSketch::default();
- via_merge.merge(&s1).expect("merge should succeed");
- via_merge.merge(&s2).expect("merge should succeed");
- via_merge.merge(&s3).expect("merge should succeed");
-
- assert_eq!(union_result.n(), via_merge.n());
- assert_eq!(union_result.min_item(), via_merge.min_item());
- assert_eq!(union_result.max_item(), via_merge.max_item());
-
- let true_median = 1499.5_f64;
- let tolerance = 0.05 * 3000.0;
- let q_union = union_result
- .quantile(0.5, SearchCriteria::Inclusive)
- .expect("quantile should succeed");
- let q_merge = via_merge
- .quantile(0.5, SearchCriteria::Inclusive)
- .expect("quantile should succeed");
-
- assert_that!(q_union, near(true_median, tolerance));
- assert_that!(q_merge, near(true_median, tolerance));
-}
-
-#[test]
-fn empty_union_returns_empty_sketch() {
- let union: ReqUnion<f64> = ReqUnion::default();
- assert!(union.to_sketch().is_empty());
-}
-
-#[test]
-fn reset_clears_union_state() {
- let mut sketch: ReqSketch<f64> = ReqSketch::default();
- for i in 0..100 {
- sketch.update(i as f64);
- }
-
- let mut union: ReqUnion<f64> = ReqUnion::default();
- union.merge(&sketch).expect("merge should succeed");
- assert!(!union.is_empty());
-
- union.reset();
- assert!(union.is_empty());
-}
-
-#[test]
-fn new_validates_k() {
- assert_that!(
- ReqUnion::<f64>::new(3, RankAccuracy::HighRank),
- err(anything())
- );
- assert!(ReqUnion::<f64>::new(12, RankAccuracy::HighRank).is_ok());
-}
-
-#[test]
-fn empty_union_uses_default_configuration() {
- let union: ReqUnion<f64> = ReqUnion::default();
- assert_eq!(union.k(), 12);
- assert_eq!(union.rank_accuracy(), RankAccuracy::HighRank);
-}
-
-#[test]
-fn union_keeps_default_k_when_merging_mismatched_sketch() {
- // The union retains its own k even when fed a sketch built with a
different k.
- let mut other = ReqSketch::<f64>::new(16, RankAccuracy::HighRank).unwrap();
- for i in 0..50 {
- other.update(i as f64);
- }
-
- let mut union: ReqUnion<f64> = ReqUnion::default();
- union.merge(&other).expect("merge should succeed");
-
- let result = union.to_sketch();
- assert_eq!(result.k(), 12);
- assert_eq!(result.n(), 50);
-}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
