jt2594838 commented on code in PR #18693:
URL: https://github.com/apache/iotdb/pull/18693#discussion_r4088998236


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/wal/io/WALInputStream.java:
##########
@@ -223,6 +255,28 @@ private void loadNextSegmentV1() throws IOException {
   private void loadNextSegmentV2() throws IOException {
     long position = channel.position();
     SegmentInfo segmentInfo = getNextSegmentInfo();
+    long remainingBytes = fileSize - channel.position();
+    if (recoveringEntries && segmentInfo.compressionType == 
CompressionType.UNCOMPRESSED) {
+      // Complete entries in a partially written uncompressed segment remain 
readable. A compressed
+      // segment needs its full payload before any of its entries can be 
recovered.
+      segmentInfo.dataInDiskSize = (int) Math.min(segmentInfo.dataInDiskSize, 
remainingBytes);
+      segmentInfo.uncompressedSize = segmentInfo.dataInDiskSize;
+    }
+    if (segmentInfo.dataInDiskSize <= 0
+        || segmentInfo.uncompressedSize <= 0
+        || segmentInfo.dataInDiskSize > remainingBytes) {
+      throw new EOFException(StorageEngineMessages.UNEXPECTED_END_OF_FILE);
+    }
+    // Recovery inspects untrusted headers. Bound allocations by the writer's 
configured segment
+    // capacity so a tiny corrupt payload cannot request a huge decompression 
buffer.
+    if (recoveringEntries
+        && (segmentInfo.uncompressedSize > WALBuffer.ONE_THIRD_WAL_BUFFER_SIZE

Review Comment:
   It is too difficult to decide on a safe recovery bound without introducing 
additional OOM risk.
   
   Since this thing is overall a best-effort attempt, I would prefer to leave 
the WAL quarantined and let the DBA decide whether increasing the memory 
setting and restarting are applicable.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to