kevin-wu24 commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4328631714
##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,18 +136,23 @@ private MemoryRecords createMemoryRecords(FileRecords
fileRecords) {
MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
- // firstBatchSize() is always non-null because the minimum buffer is
HEADER_SIZE_UP_TO_MAGIC.
- int firstBatchSize = memoryRecords.firstBatchSize();
- if (firstBatchSize <= buffer.remaining()) {
+ Integer firstBatchSize = memoryRecords.firstBatchSize();
+ if (firstBatchSize != null && firstBatchSize <= buffer.remaining()) {
+ // The buffer contains at least one complete batch
return memoryRecords;
+ } else if (firstBatchSize == null || bytesRead >=
fileRecords.sizeInBytes()) {
+ // No more bytes can be read, but the buffer doesn't contain a
complete batch. firstBatchSize
+ // is null when fewer than HEADER_SIZE_UP_TO_MAGIC bytes are left.
Callers are expected to
+ // pass records that end on a batch boundary, so reaching this
code is unexpected.
+ throw new IllegalStateException("Unexpected partial batch of
records");
Review Comment:
> This can happen when concurrently reading uncommitted data to the LEO and
another thread appends to the LEO.
This case should never happen in KRaft right?
`KRaftControlRecordStateMachine#maybeLoadLog` is the only uncommitted reader,
and it runs on the raft poll/driver thread, which is also responsible for
appending to the LEO right?
> Or when the slice doesn't end in a batch boundary
I agree that this case leads to a `FileRecords` slice which does not have
enough bytes to complete a batch. However, the only "correct/valid"
`FileRecords` slices in the KRaft/metdata code should be ending at the HWM and
the LEO, which are batch aligned. This is the motivation for throwing, since it
is not something that is expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]