Caideyipi commented on code in PR #19335:
URL: https://github.com/apache/iotdb/pull/19335#discussion_r4638051177


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/LoadTsFileSnapshot.java:
##########
@@ -167,6 +167,10 @@ private static void copyTask(final File taskDir, final 
File targetDir) throws IO
   }
 
   private static void discardUnrecordedStagedFiles(final File snapshotTaskDir) 
throws IOException {
+    // A whole-file task has no chunk records by design: its marker and its 
bytes are the state.
+    if (new File(snapshotTaskDir, 
TsFileWriterManager.WHOLE_FILE_META_NAME).isFile()) {

Review Comment:
   [P2] Catch up whole-file bytes to the snapshot boundary. `copyTask` copies 
staged data before progress logs, then relies on `catchUpStagedFiles` to repair 
concurrent appends from recorded ranges. Whole-file writes do not record 
per-slice ranges, so this early return preserves whatever prefix the copy 
captured. If a snapshot overlaps a PIECE already covered by its consensus 
boundary, the restored staging file can miss that slice tail; later slices then 
fail the offset check in `appendWholeFile`, or PREPARE rejects the file length. 
Please record/catch up a whole-file applied offset or otherwise serialize these 
writes with snapshot copying.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/load/TsFileWriterManager.java:
##########
@@ -830,6 +1189,52 @@ TsFileWriterManager recoverFromDisk() throws IOException {
     return this;
   }
 
+  private TsFileWriterManager recoverWholeFileFromDisk(final File metaFile) 
throws IOException {
+    final WholeFileMeta meta = readWholeFileMeta(metaFile);
+    if (meta == null) {
+      return this;
+    }
+    wholeFile = new File(taskDir, meta.tsFileName);
+    if (!wholeFile.isFile()) {
+      return this;
+    }
+    wholeFileChannel =
+        FileChannel.open(wholeFile.toPath(), StandardOpenOption.READ, 
StandardOpenOption.WRITE);
+    wholeFileSourceLength = meta.sourceLength;
+    wholeFileSealedLength = -1L;
+    wholeFileTimePartitionStart = meta.timePartitionStart;
+    wholeFileTimePartition = new 
TTimePartitionSlot(wholeFileTimePartitionStart);
+    wholeFileGeneratedByPipe = meta.generatedByPipe;
+    wholeFileProgress = new LoadTsFileProgress(wholeFile);
+
+    final File modsV2 = ModificationFile.getExclusiveMods(wholeFile);
+    final File modsV1 = ModificationFileV1.getNormalMods(wholeFile);
+    if (modsV2.isFile()) {
+      wholeFileMods = modsV2;
+      wholeFileModsKind = WholeFileData.Kind.MODS_V2;
+    } else if (modsV1.isFile()) {
+      wholeFileMods = modsV1;
+      wholeFileModsKind = WholeFileData.Kind.MODS_V1;
+    }
+    if (wholeFileMods != null) {
+      wholeFileModsChannel =
+          FileChannel.open(
+              wholeFileMods.toPath(), StandardOpenOption.READ, 
StandardOpenOption.WRITE);
+      wholeFileModsSourceLength = wholeFileMods.length();

Review Comment:
   [P2] Persist the declared mods length for recovery. `ensureWholeFileMods` 
compares this field with each incoming slice sourceLength, but recovery assigns 
the current staged prefix length here. After a restart during a multi-slice 
.mods transfer, the next replayed slice carries the original full source length 
and is rejected, so the transaction cannot resume. Persist the mods kind and 
declared source length separately from the staged file length, and restore 
those values here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to