jaideeppyne opened a new pull request, #237: URL: https://github.com/apache/datasketches-rust/pull/237
`HllSketch::lower_bound` returns `estimate / (1 + rse)` for a sketch in HLL mode. The Java, C++, and Go implementations take `fmax` of that against the number of non-zero registers, and we don't. A non-zero register was set by at least one distinct item, so we can report a lower bound below a count the sketch has already proven. List and set modes were already correct, since `Container::lower_bound` clamps to its coupon count. The estimate and the upper bound are unchanged, and the upper bound has no such floor in any implementation. Concretely, at `lg_k=4`, `HLL_4`, 12 sequential values, 8 registers are non-zero and the 3-std-dev lower bound was 6.38 instead of 8. @tisonkun you asked on #224 how I noticed the last one, so: same method. I serialize Rust sketches and read them back into the C++ core through the Python binding, then diff the estimate and both bounds. Over 6156 sketches spanning HLL_4/6/8, lg_k 4 to 16, and both the direct and union paths, every estimate and every upper bound already matched, while 2161 of 18468 lower bounds diverged. All of them agree after this change. The isolation was clean: every mismatching sketch was in HLL mode, and no LIST or SET sketch ever disagreed. Tests: `tests-integration/tests/hll_test/bounds.rs` pins twelve reference vectors taken from the C++ implementation, covering all three target types and both the HIP and out-of-order paths; 13 of its 36 assertions fail without the fix. Two unit tests next to the estimator assert the floor directly, including the `cur_min > 0` case where it is the full `k`. `cargo x test` and `cargo x lint` are clean. AI assistance disclosure: I used Claude to help run the cross-language differential harness and draft the fix; I verified the divergence, the reference values, and the before/after results myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
