richardstartin commented on code in PR #8796:
URL: https://github.com/apache/pinot/pull/8796#discussion_r884001737
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/BitmapInvertedIndexReader.java:
##########
@@ -40,22 +40,25 @@ public class BitmapInvertedIndexReader implements
InvertedIndexReader<ImmutableR
// Use the offset of the first bitmap to support 2 different format of the
inverted index:
// 1. Offset buffer stores the offsets within the whole data buffer
(including offset buffer)
// 2. Offset buffer stores the offsets within the bitmap buffer
- private final int _firstOffset;
+ private final long _firstOffset;
public BitmapInvertedIndexReader(PinotDataBuffer dataBuffer, int numBitmaps)
{
long offsetBufferEndOffset = (long) (numBitmaps + 1) * Integer.BYTES;
_offsetBuffer = dataBuffer.view(0, offsetBufferEndOffset,
ByteOrder.BIG_ENDIAN);
_bitmapBuffer = dataBuffer.view(offsetBufferEndOffset, dataBuffer.size());
-
- _firstOffset = _offsetBuffer.getInt(0);
+ _firstOffset = getOffset(0);
}
@SuppressWarnings("unchecked")
@Override
public ImmutableRoaringBitmap getDocIds(int dictId) {
- int offset = _offsetBuffer.getInt(dictId * Integer.BYTES);
- int length = _offsetBuffer.getInt((dictId + 1) * Integer.BYTES) - offset;
- return new ImmutableRoaringBitmap(_bitmapBuffer.toDirectByteBuffer(offset
- _firstOffset, length));
+ long offset = getOffset(dictId);
+ long length = getOffset(dictId + 1) - offset;
+ return new ImmutableRoaringBitmap(_bitmapBuffer.toDirectByteBuffer(offset
- _firstOffset, (int) length));
Review Comment:
Note that the maximum size of a `RoaringBitmap` is 2^16 * (8KB + 2B) ~ 512MB
so the cast to int never overflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
