cheeeee commented on code in PR #23445:
URL: https://github.com/apache/kafka/pull/23445#discussion_r4317845403


##########
clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java:
##########
@@ -278,17 +315,61 @@ public int truncateTo(int targetSize) throws IOException {
 
     @Override
     public int writeTo(TransferableChannel destChannel, int offset, int 
length) throws IOException {
-        long newSize = Math.min(channel.size(), end) - start;
         int oldSize = sizeInBytes();
-        if (newSize < oldSize)
-            throw new KafkaException(String.format(
-                    "Size of FileRecords %s has been truncated during write: 
old size %d, new size %d",
-                    file.getAbsolutePath(), oldSize, newSize));
-
         long position = start + offset;
         int count = Math.min(length, oldSize - offset);
-        // safe to cast to int since `count` is an int
-        return (int) destChannel.transferFrom(channel, position, count);
+        try {
+            long newSize = Math.min(channel.size(), end) - start;
+            if (newSize < oldSize)
+                throw new KafkaException(String.format(
+                        "Size of FileRecords %s has been truncated during 
write: old size %d, new size %d",
+                        file.getAbsolutePath(), oldSize, newSize));
+            // safe to cast to int since `count` is an int
+            return (int) destChannel.transferFrom(channel, position, count);
+        } catch (IOException e) {
+            // The transfer reads the file and writes the socket in one call, 
so a storage failure and a socket
+            // failure both arrive here as a bare IOException. Let the handler 
find out which one it was. A closed
+            // channel means the file was not read, so there is nothing to 
find out.
+            if (transferFailureHandler != null && !(e instanceof 
ClosedChannelException))
+                transferFailureHandler.onTransferFailure(this, position, 
count, e);
+            throw e;
+        }
+    }
+
+    /**
+     * Read the given region of the backing file, after {@link #writeTo} 
failed on it, to tell a storage failure from

Review Comment:
   Thanks, Jun. I'd rather keep re-reading the whole region. The premise no 
longer holds on TLS: with #23832, `transferFrom` rethrows after partial 
progress, and in the SSL run in the test notes the read failed at position 
2613248 of a transfer of 8388608 bytes from position 0. On macOS, `sendfile` 
reads 64 pages (256 KiB) at a time and drops the progress of the chunk whose 
read fails, so the bad bytes can be up to 256 KiB past `position`.
   
   **Re-reading the whole region does not depend on where each platform fails 
the transfer, so Linux, macOS and Windows are handled the same way**, for any 
read failure the JVM reports within the region, and it is cheap on all of them: 
a disconnect usually comes right after the send, so the region is in the page 
cache. In the test notes we measured 64 KiB, 1 MiB and 8 MiB regions on the 
test host, a VM on RAID 0 SATA SSDs under load: the re-read took 0.1 to 2.3 ms 
from the page cache and 0.9 to 29 ms (median) from disk, on the scheduler and 
at most one at a time per log directory. The numbers depend on the host and 
should not be taken literally, but their spread shows the order of the cost.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to