kevin-wu24 commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4328082058
##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,18 +136,23 @@ private MemoryRecords createMemoryRecords(FileRecords
fileRecords) {
MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
- // firstBatchSize() is always non-null because the minimum buffer is
HEADER_SIZE_UP_TO_MAGIC.
- int firstBatchSize = memoryRecords.firstBatchSize();
- if (firstBatchSize <= buffer.remaining()) {
+ Integer firstBatchSize = memoryRecords.firstBatchSize();
+ if (firstBatchSize != null && firstBatchSize <= buffer.remaining()) {
+ // The buffer contains at least one complete batch
return memoryRecords;
+ } else if (firstBatchSize == null || bytesRead >=
fileRecords.sizeInBytes()) {
+ // No more bytes can be read, but the buffer doesn't contain a
complete batch. firstBatchSize
+ // is null when fewer than HEADER_SIZE_UP_TO_MAGIC bytes are left.
Callers are expected to
+ // pass records that end on a batch boundary, so reaching this
code is unexpected.
+ throw new IllegalStateException("Unexpected partial batch of
records");
Review Comment:
> RecordsIterator supports both MemoryRecords and FileRecords. MemoryRecords
input still ignores a trailing incomplete batch through batchIterator(), while
FileRecords now throws. Could we make the behavior consistent and also document
this in the javadoc?
The `MemoryRecords` branch in `nextBatches` is only hit during unit tests
because those use `MockLog`. But yeah, I can make the change.
Also moved the validation on batch alignment in `createMemoryRecords` into
`readFileRecords`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]