Aschenbath opened a new pull request, #322: URL: https://github.com/apache/datasketches-rust/pull/322
Closes #321. ## What - Adds `TupleUnion::serialize(&self, ordered: bool) -> Vec<u8>`. It needs `P::Summary: TupleSummaryValue` but not `Clone`. - `UnionState` has one result-preparation routine (theta filtering, trimming to nominal capacity, ordering), and it takes a projection. `to_compact_sketch_state` projects with `E::clone`, so the Theta path compiles to the same code as before. The new `to_borrowed_compact_sketch_state` (gated on `tuple`) projects to `&E`. `SketchEntry` is implemented for `&E` to support this. - The compact Tuple encoder moved into `serialize_compact_state`, which is generic over `E: Borrow<TupleEntry<S>>`. `CompactTupleSketch::serialize` and `TupleUnion::serialize` both use it, so the format logic exists only once. When `ordered` is `true`, the output is byte-identical to `union.to_sketch(true).serialize()`. For unordered results with trimming, entries can come out in a different order than in the cloning path, because `select_nth_unstable` may pick a different partition scheme depending on the element size. The retained entries, theta, seed hash and flags are the same. The doc comment states this. I started with `TupleUnion` only, as the issue suggests. `TupleIntersection` and mutable-sketch compaction aren't touched. ## Tests Added to `tests-integration/tests/serde_tests/tuple.rs`: - `union_serialization_matches_compact_result` (`u64` summaries) compares `serialize` with `to_sketch(..).serialize()` for both orderings. The cases are: a fresh union, an empty input, a single entry with combined summaries, exact mode (30 entries), trimming to nominal capacity (lg_k 5 → 32 entries), a sampled union (p = 0.5), and a non-empty result with every entry screened by theta. Ordered results must match byte for byte. Unordered results must have the same entries, theta, seed hash and flags. - `union_serialization_with_heap_backed_summaries_supports_later_merges` uses a variable-size `Vec<String>` summary with a custom codec, in estimation mode. It round-trips the result, merges the restored sketch with another sketch, and compares against merging `to_sketch()` directly. `cargo x test`, `cargo +nightly clippy --tests --all-features --all-targets --workspace -- -D warnings`, `cargo +nightly fmt --all --check` and `typos` pass, and `cargo +1.86.0 check -p datasketches --all-features` (MSRV) passes too. I ran these on Windows x86_64. I didn't run `taplo` or `hawkeye` locally: no TOML changed, and the new benchmark file has the license header. ## Benchmarks New `benchmarks/tuple/serde.rs` compares `serialize` with `to_sketch(..).serialize()`. One input sketch goes into a union (default lg_k 12). Medians from Divan with `AllocProfiler`, unordered (ordered is similar): Heap-backed summary (4 short `String`s per key): | rows | `to_sketch().serialize()` | `serialize()` | allocations | |---:|---:|---:|---:| | 4 | 954 ns | 252 ns | 22 → 2 | | 100 | 23.5 µs | 3.8 µs | 502 → 2 | | 80,000 | 2.20 ms | 0.25 ms | 31,757 → 2 | `u64` summary, from two runs with 500 samples: | rows | `to_sketch().serialize()` | `serialize()` | |---:|---:|---:| | 4 | 105–113 ns | 138–140 ns | | 100 | 0.81–0.90 µs | 1.08–1.11 µs | | 80,000 | 26–29 µs | 29–38 µs | So the gain is for summaries that are expensive to clone. For small `Copy` summaries the borrowed path is about 15–25% slower at small sizes, a few hundred nanoseconds. Cloning 16-byte entries into a contiguous vector is about as cheap as collecting pointers, and the encoder then has to dereference each pointer. The CHANGELOG entry only claims the heap-backed case. If you'd rather keep `u64` users on the cloning path, I can document that on `serialize`. Another option is to collect `(hash, &summary)` pairs so that sorting doesn't dereference. --- This PR was written by an AI agent (Claude Code): the code, tests, benchmarks and this description. The verification above is what the agent ran. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
