kevin-wu24 commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4328631714


##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,18 +136,23 @@ private MemoryRecords createMemoryRecords(FileRecords 
fileRecords) {
 
         MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
 
-        // firstBatchSize() is always non-null because the minimum buffer is 
HEADER_SIZE_UP_TO_MAGIC.
-        int firstBatchSize = memoryRecords.firstBatchSize();
-        if (firstBatchSize <= buffer.remaining()) {
+        Integer firstBatchSize = memoryRecords.firstBatchSize();
+        if (firstBatchSize != null && firstBatchSize <= buffer.remaining()) {
+            // The buffer contains at least one complete batch
             return memoryRecords;
+        } else if (firstBatchSize == null || bytesRead >= 
fileRecords.sizeInBytes()) {
+            // No more bytes can be read, but the buffer doesn't contain a 
complete batch. firstBatchSize
+            // is null when fewer than HEADER_SIZE_UP_TO_MAGIC bytes are left. 
Callers are expected to
+            // pass records that end on a batch boundary, so reaching this 
code is unexpected.
+            throw new IllegalStateException("Unexpected partial batch of 
records");

Review Comment:
   > This can happen when concurrently reading uncommitted data to the LEO and 
another thread appends to the LEO.
   
   This case should never happen in KRaft right? 
`KRaftControlRecordStateMachine#maybeLoadLog` is the only uncommitted reader, 
and it runs on the raft poll/driver thread, which is also responsible for 
appending to the LEO right?
   
   > Or when the slice doesn't end in a batch boundary
   
   I agree that this case leads to a `FileRecords` slice which does not have 
enough bytes to complete a batch. However, the only "correct/valid" 
`FileRecords` slices in the KRaft/metdata code should be ending at the HWM and 
the LEO, which are batch aligned. This is the motivation for throwing, since it 
is not something that is expected.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to