junrao commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4273527092
##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,9 +136,11 @@ private MemoryRecords createMemoryRecords(FileRecords
fileRecords) {
MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
- // firstBatchSize() is always non-null because the minimum buffer is
HEADER_SIZE_UP_TO_MAGIC.
- int firstBatchSize = memoryRecords.firstBatchSize();
- if (firstBatchSize <= buffer.remaining()) {
+ Integer firstBatchSize = memoryRecords.firstBatchSize();
+ int bytesAvailable = buffer.remaining() + (fileRecords.sizeInBytes() -
bytesRead);
+ if (firstBatchSize == null || firstBatchSize <= buffer.remaining() ||
+ firstBatchSize > bytesAvailable
Review Comment:
My concern is on the caller logic. The check `if (bytesRead < recordSize)`
used to be always true when the iteration finishes for the records. However,
with this change, this is no longer true. It doesn't cause an issue now since
existing callers don't depend on that. However, this is a bit dangerous since
future callers may not realize this. I think it's better to make the behavior
more consistent. We can either change createMemoryRecords() to drain all bytes
in the records or keep the optimization in createMemoryRecords(), but remove
`if (bytesRead < recordSize)` in nextBatches().
```
private Iterator<MutableRecordBatch> nextBatches() {
int recordSize = records.sizeInBytes();
if (bytesRead < recordSize) {
final MemoryRecords memoryRecords;
if (records instanceof MemoryRecords) {
bytesRead = recordSize;
memoryRecords = (MemoryRecords) records;
} else if (records instanceof FileRecords) {
memoryRecords = createMemoryRecords((FileRecords) records);
} else {
throw new IllegalStateException(String.format("Unexpected
Records type %s", records.getClass()));
}
return memoryRecords.batchIterator();
}
return Collections.emptyIterator();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]