leerho opened a new pull request, #787: URL: https://github.com/apache/datasketches-java/pull/787
`ReqSketch.CreateSortedView` weighted each compactor's items by `1 << c.getLgWeight()`, an `int` shift assigned to a `long`. From level 31 up, the weight was wrong: level 31 gave -2^31, and level 32 gave 1 because the shift wraps at 32. Ranks and quantiles of sketches deep enough to reach level 31 were wrong. The shift is now `1L << c.getLgWeight()`, matching C++, which uses `1ULL << compactor.get_lg_weight()`. Level 31 needs on the order of 10^11 updates, so it takes a very large stream to see this, but nothing prevents it. ### Tests New `highLevelWeights` in `ReqSketchSortedViewTest`: a sketch with one item in each of 33 compactors (levels 0 to 32), in both HRA and LRA modes, checks each item's weight (2^h), its inclusive natural rank and the total N. It fails on main and passes here; the REQ tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
