kevin-wu24 commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4315268557
##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -137,12 +137,21 @@ private MemoryRecords createMemoryRecords(FileRecords
fileRecords) {
MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
Integer firstBatchSize = memoryRecords.firstBatchSize();
- // firstBatchSize is null when fewer than HEADER_SIZE_UP_TO_MAGIC
bytes were read, which only
- // happens at the end of the records. Once all records have been read,
a trailing partial
- // batch can't be completed, so return the records read so far instead
of growing the buffer.
- if (firstBatchSize == null || firstBatchSize <= buffer.remaining() ||
- bytesRead >= fileRecords.sizeInBytes()
- ) {
+ if (firstBatchSize != null && firstBatchSize <= buffer.remaining()) {
+ // The buffer contains at least one complete batch
+ return memoryRecords;
+ } else if (firstBatchSize == null || bytesRead >=
fileRecords.sizeInBytes()) {
+ // No more bytes can be read, but the buffer doesn't contain a
complete batch. firstBatchSize
+ // is null when fewer than HEADER_SIZE_UP_TO_MAGIC bytes are left.
Callers are expected to
+ // pass records that end on a batch boundary, so the records were
cut in the middle of a
+ // batch or are corrupt. The partial batch can't be completed, so
don't grow the buffer.
+ logger.error(
Review Comment:
I believe it is better to throw an exception. The callers of this should all
be at batch aligned offsets, since the HWM, LEO, and any snapshot offset should
be batch aligned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]