leerho commented on issue #478:
URL:
https://github.com/apache/datasketches-java/issues/478#issuecomment-1829000038
@adarshsanjeev,
I'm not familiar with the HllSketchHolderObjectStrategy in Druid.
I think your "isSaveToConvertToNullSketch(ByteBuffer, int)" is too
complicated. But I may not be aware of other issues you are trying to deal
with, so the following suggestions may or may not be useful to you.
My recommended approach would be to wrap the sketch, which doesn't
deserialize the sketch, and then you can use the HllSketch API to check for
empty:
//All of our sketches require LittleEndian Byte Order
Memory mem = Memory.wrap(ByteBuffer, ByteOrder byteOrder);
HllSketch sk = HllSketch.wrap(mem);
if (sk.isEmpty()) { ... }
######
Another approach would be examining the binary as you are trying to do. But
this can be fragile since the binary format can change.
It doesn't matter whether the sketch has been compacted or not, the
following will hold true:
Given the following byte names and indices, starting from 0:
- Family = index 2 //for HLL sketches this is == 7
- Flags = index 5
- ListCount = index 6
- Mode = index 7
If an HLL sketch is empty, the following will be true:
- Family == 7
- Flags & 0x4 > 0 // an empty sketch should have the empty bit set
- ListCount == 0 // an empty sketch always has a List Count == 0
- Mode & 0x3 == 0 // an empty sketch is always in LIST mode
An empty sketch always has at least 8 bytes, if the size is < 8 bytes, it
has been truncated outside our sketch code and is not a valid sketch.
One approach would be to examine the first 8 bytes in one operation:
long mask = 0x03_FF_04_00_00_07_00_00L;
if (first8bytes & mask == 0x4_00_00_07_00_00L) { empty } else { not empty}
I hope this helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
