yashmayya commented on PR #19757: URL: https://github.com/apache/pinot/pull/19757#issuecomment-6027224125
@gortiz Thanks! Done in 021374bb0e. IdSets now read through `DataBufferPinotInputStream` over `PinotByteBuffer.slice`, and `Roaring64NavigableMapIdSet` passes the stream to `deserialize(DataInput)` directly, which decodes about 2x faster. One addition: Bloom filter reads go through a `BufferedInputStream`. Guava reads 8 bytes at a time, and `PinotByteBuffer.copyTo` copies reads of up to 10 bytes one byte at a time, so Bloom decoding was about 2x slower without it. With the buffer, a 4.5 MB Bloom set decodes within about 1 ms of the old path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
