cheeeee commented on code in PR #23445:
URL: https://github.com/apache/kafka/pull/23445#discussion_r4320285538
##########
storage/src/main/java/org/apache/kafka/storage/internals/log/LocalLog.java:
##########
@@ -526,6 +528,48 @@ else if (segment.baseOffset() ==
maxOffsetMetadata.segmentBaseOffset && !maxOffs
);
}
+ /**
+ * The records of a fetch are read from the segment only when they are
written to the network, after this log
+ * has returned them, so an IOException from that read cannot be caught
here. Have the records report a failed
+ * write instead, so that the log directory is marked offline if the
segment cannot be read, as for any other
+ * IOException in this log.
+ */
+ private FetchDataInfo withTransferFailureHandler(FetchDataInfo
fetchDataInfo) {
+ if (!(fetchDataInfo.records instanceof FileRecords fileRecords))
+ return fetchDataInfo;
+ return
fetchDataInfo.withRecords(fileRecords.withTransferFailureHandler(transferFailureHandler));
+ }
+
+ /**
+ * Called on the network thread when writing fetched records failed, which
may be a socket or a storage failure.
+ * Re-reading the region tells them apart, so it is done on the scheduler
rather than on the network thread, at
+ * most once at a time per log dir, so that a burst of failed sends or a
slow failing disk does not take up the
+ * threads of the scheduler, which other tasks of the broker share.
+ */
+ private void checkReadAfterTransferFailure(FileRecords records, long
position, int count, IOException transferFailure) {
Review Comment:
Done, without the null check: `scheduleOnce` never returns null. The test
now also covers an `Error` from `scheduleOnce`.
##########
clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java:
##########
@@ -278,17 +315,61 @@ public int truncateTo(int targetSize) throws IOException {
@Override
public int writeTo(TransferableChannel destChannel, int offset, int
length) throws IOException {
- long newSize = Math.min(channel.size(), end) - start;
int oldSize = sizeInBytes();
- if (newSize < oldSize)
- throw new KafkaException(String.format(
- "Size of FileRecords %s has been truncated during write:
old size %d, new size %d",
- file.getAbsolutePath(), oldSize, newSize));
-
long position = start + offset;
int count = Math.min(length, oldSize - offset);
- // safe to cast to int since `count` is an int
- return (int) destChannel.transferFrom(channel, position, count);
+ try {
+ long newSize = Math.min(channel.size(), end) - start;
+ if (newSize < oldSize)
+ throw new KafkaException(String.format(
+ "Size of FileRecords %s has been truncated during
write: old size %d, new size %d",
+ file.getAbsolutePath(), oldSize, newSize));
+ // safe to cast to int since `count` is an int
+ return (int) destChannel.transferFrom(channel, position, count);
+ } catch (IOException e) {
+ // The transfer reads the file and writes the socket in one call,
so a storage failure and a socket
+ // failure both arrive here as a bare IOException. Let the handler
find out which one it was. A closed
+ // channel means the file was not read, so there is nothing to
find out.
+ if (transferFailureHandler != null && !(e instanceof
ClosedChannelException))
+ transferFailureHandler.onTransferFailure(this, position,
count, e);
+ throw e;
+ }
+ }
+
+ /**
+ * Read the given region of the backing file, after {@link #writeTo}
failed on it, to tell a storage failure from
+ * a socket failure. The whole region is read rather than its first byte
because, depending on the platform, the
+ * transfer may fail anywhere within it without reporting progress. Only a
failed read of an open channel counts
+ * as a storage failure: the end of the file may be reached because the
log was truncated concurrently (e.g. when
+ * the replica becomes a follower), and the channel may have been closed
concurrently, by a deletion or a
+ * shutdown; neither says anything about the medium.
+ *
+ * @param position The position of the region in the file
Review Comment:
Done, here and in `TransferFailureHandler.onTransferFailure`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]