jt2594838 commented on code in PR #18693:
URL: https://github.com/apache/iotdb/pull/18693#discussion_r4078426134
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/wal/io/WALInputStream.java:
##########
@@ -223,6 +253,18 @@ private void loadNextSegmentV1() throws IOException {
private void loadNextSegmentV2() throws IOException {
long position = channel.position();
SegmentInfo segmentInfo = getNextSegmentInfo();
+ long remainingBytes = fileSize - channel.position();
+ if (recoveringEntries && segmentInfo.compressionType ==
CompressionType.UNCOMPRESSED) {
+ // Complete entries in a partially written uncompressed segment remain
readable. A compressed
+ // segment needs its full payload before any of its entries can be
recovered.
+ segmentInfo.dataInDiskSize = (int) Math.min(segmentInfo.dataInDiskSize,
remainingBytes);
+ segmentInfo.uncompressedSize = segmentInfo.dataInDiskSize;
+ }
+ if (segmentInfo.dataInDiskSize <= 0
+ || segmentInfo.uncompressedSize <= 0
Review Comment:
Addressed in bb74080164e. Recovery now rejects a declared logical segment
size above WALBuffer.ONE_THIRD_WAL_BUFFER_SIZE before allocating direct
buffers, and bounds compressed payloads using the compressor maximum for that
capacity. The 16-byte LZ4 regression includes the declared payload and expects
EOFException from the pre-allocation check. All 14 WALFileTest tests pass, with
Checkstyle and Spotless.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/wal/io/WALMetaData.java:
##########
@@ -218,9 +222,17 @@ public static WALMetaData deserialize(ByteBuffer buffer) {
public static WALMetaData deserialize(ByteBuffer buffer, WALFileVersion
version) {
long firstSearchIndex = buffer.getLong();
int entriesNum = buffer.getInt();
+ // Reject corrupted counts before allocating arrays from an untrusted
footer.
+ if (entriesNum < 0 || entriesNum > buffer.remaining() / Integer.BYTES) {
+ throw new
IllegalArgumentException(StorageEngineMessages.UNEXPECTED_EXCEPTION);
+ }
List<Integer> buffersSize = new ArrayList<>(entriesNum);
for (int i = 0; i < entriesNum; ++i) {
- buffersSize.add(buffer.getInt());
+ int size = buffer.getInt();
+ if (size <= 0) {
Review Comment:
The full entry-size scan has been reverted because it adds a full logical
WAL read/decompression pass to metadata lookup. The footer path remains
unchanged in bb74080164e. This positive-size corruption case remains open; a
cheaper validation approach is needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]