kevin-wu24 commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4328082058


##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,18 +136,23 @@ private MemoryRecords createMemoryRecords(FileRecords 
fileRecords) {
 
         MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
 
-        // firstBatchSize() is always non-null because the minimum buffer is 
HEADER_SIZE_UP_TO_MAGIC.
-        int firstBatchSize = memoryRecords.firstBatchSize();
-        if (firstBatchSize <= buffer.remaining()) {
+        Integer firstBatchSize = memoryRecords.firstBatchSize();
+        if (firstBatchSize != null && firstBatchSize <= buffer.remaining()) {
+            // The buffer contains at least one complete batch
             return memoryRecords;
+        } else if (firstBatchSize == null || bytesRead >= 
fileRecords.sizeInBytes()) {
+            // No more bytes can be read, but the buffer doesn't contain a 
complete batch. firstBatchSize
+            // is null when fewer than HEADER_SIZE_UP_TO_MAGIC bytes are left. 
Callers are expected to
+            // pass records that end on a batch boundary, so reaching this 
code is unexpected.
+            throw new IllegalStateException("Unexpected partial batch of 
records");

Review Comment:
   > RecordsIterator supports both MemoryRecords and FileRecords. MemoryRecords 
input still ignores a trailing incomplete batch through batchIterator(), while 
FileRecords now throws. Could we make the behavior consistent and also document 
this in the javadoc?
   
   The `MemoryRecords` branch in `nextBatches` is only hit during unit tests 
because those use `MockLog`. But yeah, I can make the change.
   
   Also moved the validation on batch alignment in `createMemoryRecords` into 
`readFileRecords`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to