cheeeee commented on code in PR #23445:
URL: https://github.com/apache/kafka/pull/23445#discussion_r4320285538


##########
storage/src/main/java/org/apache/kafka/storage/internals/log/LocalLog.java:
##########
@@ -526,6 +528,48 @@ else if (segment.baseOffset() == 
maxOffsetMetadata.segmentBaseOffset && !maxOffs
         );
     }
 
+    /**
+     * The records of a fetch are read from the segment only when they are 
written to the network, after this log
+     * has returned them, so an IOException from that read cannot be caught 
here. Have the records report a failed
+     * write instead, so that the log directory is marked offline if the 
segment cannot be read, as for any other
+     * IOException in this log.
+     */
+    private FetchDataInfo withTransferFailureHandler(FetchDataInfo 
fetchDataInfo) {
+        if (!(fetchDataInfo.records instanceof FileRecords fileRecords))
+            return fetchDataInfo;
+        return 
fetchDataInfo.withRecords(fileRecords.withTransferFailureHandler(transferFailureHandler));
+    }
+
+    /**
+     * Called on the network thread when writing fetched records failed, which 
may be a socket or a storage failure.
+     * Re-reading the region tells them apart, so it is done on the scheduler 
rather than on the network thread, at
+     * most once at a time per log dir, so that a burst of failed sends or a 
slow failing disk does not take up the
+     * threads of the scheduler, which other tasks of the broker share.
+     */
+    private void checkReadAfterTransferFailure(FileRecords records, long 
position, int count, IOException transferFailure) {

Review Comment:
   Done, without the null check: `scheduleOnce` never returns null. The test 
now also covers an `Error` from `scheduleOnce`.
   



##########
clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java:
##########
@@ -278,17 +315,61 @@ public int truncateTo(int targetSize) throws IOException {
 
     @Override
     public int writeTo(TransferableChannel destChannel, int offset, int 
length) throws IOException {
-        long newSize = Math.min(channel.size(), end) - start;
         int oldSize = sizeInBytes();
-        if (newSize < oldSize)
-            throw new KafkaException(String.format(
-                    "Size of FileRecords %s has been truncated during write: 
old size %d, new size %d",
-                    file.getAbsolutePath(), oldSize, newSize));
-
         long position = start + offset;
         int count = Math.min(length, oldSize - offset);
-        // safe to cast to int since `count` is an int
-        return (int) destChannel.transferFrom(channel, position, count);
+        try {
+            long newSize = Math.min(channel.size(), end) - start;
+            if (newSize < oldSize)
+                throw new KafkaException(String.format(
+                        "Size of FileRecords %s has been truncated during 
write: old size %d, new size %d",
+                        file.getAbsolutePath(), oldSize, newSize));
+            // safe to cast to int since `count` is an int
+            return (int) destChannel.transferFrom(channel, position, count);
+        } catch (IOException e) {
+            // The transfer reads the file and writes the socket in one call, 
so a storage failure and a socket
+            // failure both arrive here as a bare IOException. Let the handler 
find out which one it was. A closed
+            // channel means the file was not read, so there is nothing to 
find out.
+            if (transferFailureHandler != null && !(e instanceof 
ClosedChannelException))
+                transferFailureHandler.onTransferFailure(this, position, 
count, e);
+            throw e;
+        }
+    }
+
+    /**
+     * Read the given region of the backing file, after {@link #writeTo} 
failed on it, to tell a storage failure from
+     * a socket failure. The whole region is read rather than its first byte 
because, depending on the platform, the
+     * transfer may fail anywhere within it without reporting progress. Only a 
failed read of an open channel counts
+     * as a storage failure: the end of the file may be reached because the 
log was truncated concurrently (e.g. when
+     * the replica becomes a follower), and the channel may have been closed 
concurrently, by a deletion or a
+     * shutdown; neither says anything about the medium.
+     *
+     * @param position The position of the region in the file

Review Comment:
   Done, here and in `TransferFailureHandler.onTransferFailure`.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to