jsancio commented on code in PR #23737:
URL: https://github.com/apache/kafka/pull/23737#discussion_r4327614303


##########
clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java:
##########
@@ -128,15 +126,20 @@ public FileChannel channel() {
     }
 
     /**
-     * Read log batches into the given buffer until there are no bytes 
remaining in the buffer or the end of the file
-     * is reached.
+     * Read log batches into the given buffer until there are no bytes 
remaining in the buffer or the end of
+     * this FileRecords object is reached.
      *
      * @param buffer The buffer to write the batches to
-     * @param position Position in the buffer to read from
+     * @param position Position to start reading from, relative to the start 
of this FileRecords object
      * @throws IOException If an I/O error occurs, see {@link 
FileChannel#read(ByteBuffer, long)} for details on the
-     * possible exceptions
+     * possible exceptions.
+     * @throws IllegalArgumentException If position is negative or greater 
than {@link #sizeInBytes()}
      */
     public void readInto(ByteBuffer buffer, int position) throws IOException {
+        int currentSize = sizeInBytes();
+        if (position < 0 || position > currentSize)
+            throw new IllegalArgumentException("Read position " + position + " 
is outside " + this);

Review Comment:
   Can we enclose this statement using `{ }`:
   ```java
           if (position < 0 || position > currentSize) {
               throw new IllegalArgumentException("Read position " + position + 
" is outside " + this);
           }
   ```



##########
raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java:
##########
@@ -136,18 +136,23 @@ private MemoryRecords createMemoryRecords(FileRecords 
fileRecords) {
 
         MemoryRecords memoryRecords = readFileRecords(fileRecords, buffer);
 
-        // firstBatchSize() is always non-null because the minimum buffer is 
HEADER_SIZE_UP_TO_MAGIC.
-        int firstBatchSize = memoryRecords.firstBatchSize();
-        if (firstBatchSize <= buffer.remaining()) {
+        Integer firstBatchSize = memoryRecords.firstBatchSize();
+        if (firstBatchSize != null && firstBatchSize <= buffer.remaining()) {
+            // The buffer contains at least one complete batch
             return memoryRecords;
+        } else if (firstBatchSize == null || bytesRead >= 
fileRecords.sizeInBytes()) {
+            // No more bytes can be read, but the buffer doesn't contain a 
complete batch. firstBatchSize
+            // is null when fewer than HEADER_SIZE_UP_TO_MAGIC bytes are left. 
Callers are expected to
+            // pass records that end on a batch boundary, so reaching this 
code is unexpected.
+            throw new IllegalStateException("Unexpected partial batch of 
records");

Review Comment:
   I am not sure that we should throw an exception if FileRecords doesn't have 
enough bytes to complete a batch. This can happen when concurrently reading 
uncommitted data to the LEO and another thread appends to the LEO. Or when the 
slice doesn't end in a batch boundary.
   
   I think we need to distinguish the two cases:
   1. The buffer is not big enough to read the entire batch - create a new 
batch that is big enough to read the batch from the file records.
   2. The file record (slice or not) doesn't contain an entire batch - return 
to the user that there are no more batches.
   
   I guess the issue is that if there are concurrent read and write from the 
LEO `RecordsIterator::hasNext` may return false in one call to later return 
true. Is this what we are trying to fix? I took a look at the Java 
documentation for Iterator and the documentation does forbid this. I would 
assume that concurrent iterator need to be able to return false in one call and 
true in a future call.
   
   What do we think?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to