J-HowHuang commented on code in PR #18898:
URL: https://github.com/apache/pinot/pull/18898#discussion_r3547375507
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/bloom/OnHeapGuavaBloomFilterCreator.java:
##########
@@ -43,12 +43,23 @@
public class OnHeapGuavaBloomFilterCreator implements BloomFilterCreator {
private static final Logger LOGGER =
LoggerFactory.getLogger(OnHeapGuavaBloomFilterCreator.class);
+ /** Legacy format: {@code [TYPE_VALUE=1 (int)][VERSION (int)][Guava
bytes...]} — no fpp in header. */
public static final int TYPE_VALUE = 1;
+ /**
+ * V2 format: {@code [TYPE_VALUE_V2=2 (int)][VERSION (int)][effective FPP
(double)][Guava bytes...]}.
+ * The effective fpp (after applying any {@code maxSizeInBytes} cap) is
stored at byte offset 8 so that
+ * {@link
org.apache.pinot.segment.local.segment.index.loader.bloomfilter.BloomFilterHandler}
can compare it
+ * directly without depending on Guava's internal serialisation layout.
+ */
+ public static final int TYPE_VALUE_V2 = 2;
Review Comment:
I think if we're using the same `OnHeapGuavaBloomFilterCreator`, then it
makes more sense to keep the same `TYPE_VALUE` and increase a new `VERSION`. In
the factory, read from a different buffer offset based on which version it is.
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/bloom/OnHeapGuavaBloomFilterCreator.java:
##########
@@ -87,8 +99,9 @@ public void add(String value) {
public void seal()
throws IOException {
try (DataOutputStream out = new DataOutputStream(new
FileOutputStream(_bloomFilterFile))) {
- out.writeInt(TYPE_VALUE);
+ out.writeInt(TYPE_VALUE_V2);
out.writeInt(VERSION);
Review Comment:
Is there any case that an original v1 index would be preferred? Or is it
okay to make every newly created index v2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
