jaideeppyne opened a new pull request, #767:
URL: https://github.com/apache/datasketches-java/pull/767
Fixes #756.
## Problem
Querying a heap `KllItemsSketch` before serializing it makes `heapify` /
`wrap` return a corrupted sorted view (wrong quantiles, duplicated min/max,
wrong retained count in the view).
## Root cause
`KllItemsSketch.CreateSortedView.getSV()` sorts level-0 and then records
that it did:
```java
final T[] srcQuantiles = getTotalItemsArray();
...
if (!isLevelZeroSorted()) {
Arrays.sort(srcQuantiles, srcLevelsArr[0], srcLevelsArr[1], comparator);
if (!hasMemorySegment()) { setLevelZeroSorted(true); }
}
```
For the heap Items variant, `KllHeapItemsSketch.getTotalItemsArray()`
returns a **defensive copy**, so the sort lands on the copy while
`setLevelZeroSorted(true)` is set on the live sketch. Serialization writes that
flag; `heapify` / `wrap` trust it and skip sorting.
The Doubles path is fine because
`KllHeapDoublesSketch.getDoubleItemsArray()` returns the live array, which is
what makes the comment at `KllDoublesSketch` CreateSortedView true.
## Fix
Minimal change: do **not** call `setLevelZeroSorted(true)` in Items
`CreateSortedView`. Only a copy was sorted, so the live level-0 remains
unsorted and the serialized flag must stay false. An alternative would be
returning the live array from `getTotalItemsArray()` to match Doubles; this PR
prefers not claiming the live sketch was sorted when it was not.
## Tests
Added regression tests in `KllItemsSketchSerDeTest`:
- query then `heapify` — sorted view quantiles/weights and probed ranks match
- query then `wrap` — same
- query after enough updates to force compaction (`k=8`, `n=8`) — heapify
and wrap match
Also asserts `isLevelZeroSorted()` stays false after the query on the heap
sketch.
## AI disclosure
I used Cursor / Grok to help draft the patch and open this PR via the GitHub
API (no local clone). I verified the root cause against the sources cited in
#756 before editing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
