HTHou commented on code in PR #19287:
URL: https://github.com/apache/iotdb/pull/19287#discussion_r4582715757


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/wal/recover/CheckpointRecoverUtils.java:
##########
@@ -42,38 +45,44 @@ public static CheckpointInfo recoverMemTableInfo(File 
logDirectory) {
     }
     // desc sort by version id
     CheckpointFileUtils.descSortByVersionId(checkpointFiles);
-    // find last valid .checkpoint file and load checkpoints from it
-    long maxMemTableId = 0;
-    List<Checkpoint> checkpoints = null;
+    // Read back to a complete snapshot within the newest contiguous run of 
readable checkpoints.
+    // A missing version may have removed mappings that are still present in 
an older snapshot.
+    Deque<CheckpointReader> readers = new ArrayDeque<>();
+    int nextVersion = -1;
     for (File checkpointFile : checkpointFiles) {
+      int version = 
CheckpointFileUtils.parseVersionId(checkpointFile.getName());
+      if (!readers.isEmpty() && version != nextVersion) {
+        break;
+      }
       CheckpointReader reader = new CheckpointReader(checkpointFile);
-      maxMemTableId = reader.getMaxMemTableId();
-      checkpoints = reader.getCheckpoints();
-      if (!checkpoints.isEmpty()) {
+      List<Checkpoint> checkpoints = reader.getCheckpoints();
+      if (checkpoints.isEmpty()) {
+        if (!readers.isEmpty()) {
+          break;
+        }
+        continue;
+      }
+      readers.addFirst(reader);

Review Comment:
   **[P2] Replay all active memtables for the same TsFile.** A retained 
`_0.checkpoint` can contain memtable 1 while the consecutive `_1.checkpoint` 
contains a readable create record for memtable 2 after a legacy snapshot write 
failed. Both memtables can target the same unsealed TsFile while the older one 
is still flushing. Merging the checkpoints then returns both IDs. 
`WALNodeRecoverTask` removes the recovery performer by TsFile path on its first 
lookup, so the second ID has no performer and its WAL entries are skipped. In a 
local test with one forced insert for each ID, this revision returned `SUCCESS` 
and replayed only memtable 1. With the base checkpoint reader, memtable 2 was 
replayed. Please route the valid memtable IDs for each TsFile to one performer 
and run its recovery lifecycle once per file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to