tisonkun opened a new pull request, #234: URL: https://github.com/apache/datasketches-rust/pull/234
## Summary - implement `Extend<f64>` for bulk mutable T-Digest updates - defer compression when merging at most 32 unit-weight centroids, so repeated small partial merges share one compression cycle - keep weighted and larger inputs on the existing eager merge path to preserve its memory bound and overlapping-input behavior - extend the public benchmarks around the batching boundary and bulk updates ## Design A centroid with weight one is distribution-equivalent to one raw update for the next compression. A small all-unit-weight source can therefore be appended to the accumulator's unmerged tail without changing the serialized format or query semantics. The existing compression threshold still bounds the deferred work. The 32-centroid cutoff is a performance and memory policy rather than a correctness boundary. Larger inputs retain eager compression because deferring them can accumulate a much larger temporary buffer, especially when sorted ranges overlap. The specialized path is kept out of the eager merge loop under whole-program optimization. The buffer owns the ordering operation: source unmerged values precede its compressed prefix, matching the stable tie order of the existing eager merge. ## Benchmarks 64 serialized partials, median of 5-second Divan runs on Apple Silicon: | rows / partial | disjoint before | disjoint after | overlapping before | overlapping after | | ---: | ---: | ---: | ---: | ---: | | 8 | 28.23 us | 4.90 us | 33.32 us | 7.86 us | | 32 | 46.74 us | 15.97 us | 53.32 us | 32.02 us | | 64 | 62.61 us | 62.20 us | 70.90 us | 71.24 us | The eager 64-centroid path stays within about 0.5% in both directions. Peak live allocation grows from 4.2 KB to 16.4 KB for 8-centroid inputs and from 4.6 KB to 59.9 KB for 32-centroid inputs; the 64-centroid path remains at 9.2 KB. For a contiguous update stream, `Extend<f64>` measured 12.52 us versus 12.56 us for 1,000 scalar updates and 2.239 ms versus 2.277 ms for 100,000 scalar updates. Its larger purpose is to give iterator-based callers one bulk boundary without forcing `update` to be inlined at every call site. ## Validation - `cargo x check` - `cargo x test` - `cargo x lint` - `cargo x bench` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
