junrao commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4273527092


##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,9 +136,11 @@ private MemoryRecords createMemoryRecords(FileRecords 
fileRecords) {
 
         MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
 
-        // firstBatchSize() is always non-null because the minimum buffer is 
HEADER_SIZE_UP_TO_MAGIC.
-        int firstBatchSize = memoryRecords.firstBatchSize();
-        if (firstBatchSize <= buffer.remaining()) {
+        Integer firstBatchSize = memoryRecords.firstBatchSize();
+        int bytesAvailable = buffer.remaining() + (fileRecords.sizeInBytes() - 
bytesRead);
+        if (firstBatchSize == null || firstBatchSize <= buffer.remaining() ||
+            firstBatchSize > bytesAvailable

Review Comment:
   My concern is on the caller logic. The check `if (bytesRead < recordSize)` 
used to always reach false when the iteration finishes for the records. 
However, with this change, this is no longer the case. It doesn't cause an 
issue now since existing callers don't depend on that. However, this is a bit 
dangerous since future callers may not realize this. I think it's better to 
make the behavior more consistent. We can either change createMemoryRecords() 
to drain all bytes in the records or keep the optimization in 
createMemoryRecords(), but remove `if (bytesRead < recordSize)` in 
nextBatches().
   
   ```
       private Iterator<MutableRecordBatch> nextBatches() {
           int recordSize = records.sizeInBytes();
           if (bytesRead < recordSize) {
               final MemoryRecords memoryRecords;
               if (records instanceof MemoryRecords) {
                   bytesRead = recordSize;
                   memoryRecords = (MemoryRecords) records;
               } else if (records instanceof FileRecords) {
                   memoryRecords = createMemoryRecords((FileRecords) records);
               } else {
                   throw new IllegalStateException(String.format("Unexpected 
Records type %s", records.getClass()));
               }
   
               return memoryRecords.batchIterator();
           }
   
           return Collections.emptyIterator();
       }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to