krishan1390 commented on code in PR #16517:
URL: https://github.com/apache/pinot/pull/16517#discussion_r2255793554
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java:
##########
@@ -200,6 +200,74 @@ private File getValidDocIdsSnapshotFile() {
V1Constants.VALID_DOC_IDS_SNAPSHOT_FILE_NAME);
}
+ @Nullable
+ public MutableRoaringBitmap loadQueryableDocIdsFromSnapshot() {
+ File queryablDocIdsSnapshotFile = getQueryableDocIdsSnapshotFile();
+ if (queryablDocIdsSnapshotFile.exists()) {
+ try {
+ byte[] bytes =
FileUtils.readFileToByteArray(queryablDocIdsSnapshotFile);
+ MutableRoaringBitmap queryableDocIds =
+ new
ImmutableRoaringBitmap(ByteBuffer.wrap(bytes)).toMutableRoaringBitmap();
+ LOGGER.info("Loaded queryableDocIds for segment: {} with: {} queryable
docs", getSegmentName(),
+ queryableDocIds.getCardinality());
+ return queryableDocIds;
+ } catch (Exception e) {
+ LOGGER.warn("Caught exception while loading queryableDocIds from
snapshot file: {}, ignoring the snapshot",
+ queryablDocIdsSnapshotFile);
+ }
+ }
+ return null;
+ }
+
+ public void persistQueryableDocIdsSnapshot() {
Review Comment:
since a lot of code is shared with valid docs ids , can we create a method
to take Snapshotfile as an argument and call that from
persistQueryableDocIdsSnapshot and persistValidDocIdsSnapshot ?
its applicable for loadQueryableDocIdsFromSnapshot too .
##########
pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java:
##########
@@ -91,8 +91,9 @@ public enum ServerGauge implements AbstractMetrics.Gauge {
LUCENE_INDEXING_DELAY_MS("milliseconds", false),
LUCENE_INDEXING_DELAY_DOCS("documents", false),
// Needed to track if valid doc id snapshots are present for faster restarts
- UPSERT_VALID_DOC_ID_SNAPSHOT_COUNT("upsertValidDocIdSnapshotCount", false),
+ UPSERT_SNAPSHOT_COUNT("upsertSnapshotCount", false),
Review Comment:
can we avoid changing the metric here and in ServerMeter ? it becomes
backward incompatible for users using the current metric
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
