leerho opened a new pull request, #776:
URL: https://github.com/apache/datasketches-java/pull/776
## Summary
`EmptyCompactSketch` decides whether the first 8 bytes of a serial version 3
compact image are a valid empty sketch, using a mask (which bits are examined)
and a test (their required values). This PR corrects both so that every empty
image written by an open-source Java or C++ release is accepted, while mangled
images are still rejected.
| | 9.0.0 | This PR |
|---|---|---|
| Mask | `0X00_00_EB_00_00_FF_FF_FFL` | `0X00_00_EF_FF_FF_FF_FF_FFL` |
| Test | `0X00_00_0A_00_00_03_03_01L` | `0X00_00_0E_00_00_03_03_01L` |
| Legacy test (preamble longs 3) | none | `0X00_00_0E_00_00_03_03_03L` |
| Array written | `{1, 3, 3, 0, 0, 0x1E, 0, 0}` | unchanged |
The mask and test now satisfy two rules, checked by a unit test: the test
lies within the mask, and our own array passes.
## Changes
- **Ordered flag ignored.** C++ before 3.3.0 and Java before 1.0.0 wrote
empty images without it (`0x0E`). The current mask on `main` (from a9954b988,
not in any release) requires it.
- **Bytes 3 and 4 must be zero.** They have been "not used by compact" since
0.1.0. 9.0.0 ignored them.
- **Reserved flag bits 0, 6 and 7 must be clear.** Any future use of them
requires a new serial version.
- **Empty flag checked.** Both callers had already confirmed it, so this
only makes the test complete.
- **Legacy 24-byte form accepted.** Before 1.0.0, Java wrote empty sketches
with p < 1 with preamble longs 3 and theta = p. `wrap` has rejected these since
2019. They are now accepted if they have no retained entries and carry the
expected seed hash, and are read as `EmptyCompactSketch` with theta = 1.0.
- **`CompactThetaSketch.heapify` now applies the same check as `wrap`.**
Before, it accepted any image with the empty flag set.
No serial version change: the writer is unchanged. Serial version 4 is not
affected, because empty sketches are never written compressed.
## Testing
- Full suite passes (2290 tests).
- New `EmptyTest` cases check the two rules, accept 5 historical images and
reject 12 mangled ones, through `heapify`, `wrap(MemorySegment)` and
`wrap(byte[])`.
- Real images read as empty by `heapify` and both `wrap` overloads: 2 from
sketches-core 0.9.0 (p = 0.5, ordered and unordered), and 60 written by C++
1.0.0-incubating, 2.0.0-incubating, 3.0.0, 3.2.0, 3.3.0 and current master (10
empty cases each: update compact, union, intersection and A-not-B, with p = 1
and 0.5, ordered and unordered).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
