junrao commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4273527092


##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,9 +136,11 @@ private MemoryRecords createMemoryRecords(FileRecords 
fileRecords) {
 
         MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
 
-        // firstBatchSize() is always non-null because the minimum buffer is 
HEADER_SIZE_UP_TO_MAGIC.
-        int firstBatchSize = memoryRecords.firstBatchSize();
-        if (firstBatchSize <= buffer.remaining()) {
+        Integer firstBatchSize = memoryRecords.firstBatchSize();
+        int bytesAvailable = buffer.remaining() + (fileRecords.sizeInBytes() - 
bytesRead);
+        if (firstBatchSize == null || firstBatchSize <= buffer.remaining() ||
+            firstBatchSize > bytesAvailable

Review Comment:
   My concern is on the caller logic. The check `if (bytesRead < recordSize)` 
used to be always true when the iteration finishes for the records. However, 
with this change, this is no longer true. It doesn't cause an issue now since 
existing callers don't depend on that. However, this is a bit dangerous since 
future callers may not realize this. I think it's better to make the behavior 
more consistent. We can either change createMemoryRecords() to drain all bytes 
in the records or keep the optimization in createMemoryRecords(), but remove 
`if (bytesRead < recordSize)` in nextBatches().
   
   ```
       private Iterator<MutableRecordBatch> nextBatches() {
           int recordSize = records.sizeInBytes();
           if (bytesRead < recordSize) {
               final MemoryRecords memoryRecords;
               if (records instanceof MemoryRecords) {
                   bytesRead = recordSize;
                   memoryRecords = (MemoryRecords) records;
               } else if (records instanceof FileRecords) {
                   memoryRecords = createMemoryRecords((FileRecords) records);
               } else {
                   throw new IllegalStateException(String.format("Unexpected 
Records type %s", records.getClass()));
               }
   
               return memoryRecords.batchIterator();
           }
   
           return Collections.emptyIterator();
       }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to