junrao commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4273527092
##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,9 +136,11 @@ private MemoryRecords createMemoryRecords(FileRecords
fileRecords) {
MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
- // firstBatchSize() is always non-null because the minimum buffer is
HEADER_SIZE_UP_TO_MAGIC.
- int firstBatchSize = memoryRecords.firstBatchSize();
- if (firstBatchSize <= buffer.remaining()) {
+ Integer firstBatchSize = memoryRecords.firstBatchSize();
+ int bytesAvailable = buffer.remaining() + (fileRecords.sizeInBytes() -
bytesRead);
+ if (firstBatchSize == null || firstBatchSize <= buffer.remaining() ||
+ firstBatchSize > bytesAvailable
Review Comment:
My concern is on the caller logic. The check `if (bytesRead < recordSize)`
used to always reach false when the iteration finishes for the records.
However, with this change, this is no longer the case. It doesn't cause an
issue now since existing callers don't depend on that. However, this is a bit
dangerous since future callers may not realize this. I think it's better to
make the behavior more consistent. We can either change createMemoryRecords()
to drain all bytes in the records or keep the optimization in
createMemoryRecords(), but remove `if (bytesRead < recordSize)` in
nextBatches().
```
private Iterator<MutableRecordBatch> nextBatches() {
int recordSize = records.sizeInBytes();
if (bytesRead < recordSize) {
final MemoryRecords memoryRecords;
if (records instanceof MemoryRecords) {
bytesRead = recordSize;
memoryRecords = (MemoryRecords) records;
} else if (records instanceof FileRecords) {
memoryRecords = createMemoryRecords((FileRecords) records);
} else {
throw new IllegalStateException(String.format("Unexpected
Records type %s", records.getClass()));
}
return memoryRecords.batchIterator();
}
return Collections.emptyIterator();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]