[GitHub] [kafka] ijuma commented on a change in pull request #11080: fix NPE when record==null in append
ijuma commented on a change in pull request #11080:
URL: https://github.com/apache/kafka/pull/11080#discussion_r673207693
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
Yes, I understand you're talking about the producer case. I am talking
about the fetch case. As I said, I think we may not need that special logic
anymore, but @hachikuji would know for sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #11080: fix NPE when record==null in append
ijuma commented on a change in pull request #11080:
URL: https://github.com/apache/kafka/pull/11080#discussion_r672610426
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
Is this really an exceptional case? Don't we do reads where we don't
know exactly where the read ends and hence will trigger this path?
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
I think the intent here was to cover the case where an incomplete record
is returned by the broker. However, we have broker logic to try and avoid this
case since KIP-74:
```java
} else if (!hardMaxBytesLimit && readInfo.fetchedData.firstEntryIncomplete) {
// For FetchRequest version 3, we replace incomplete message
sets with an empty one as consumers can make
// progress in such cases and don't need to report a
`RecordTooLargeException`
FetchDataInfo(readInfo.fetchedData.fetchOffsetMetadata,
MemoryRecords.EMPTY)
```
@hachikuji Do you remember if there is still a reason to return `null` here
instead of the exception @ccding is proposing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #11080: fix NPE when record==null in append
ijuma commented on a change in pull request #11080:
URL: https://github.com/apache/kafka/pull/11080#discussion_r672610426
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
Is this really an exceptional case? Don't we do reads where we don't
know exactly where the read ends and hence will trigger this path?
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
I think the intent here was to cover the case where an incomplete record
is returned by the broker. However, we have broker logic to try and avoid this
case since KIP-74:
```java
} else if (!hardMaxBytesLimit && readInfo.fetchedData.firstEntryIncomplete) {
// For FetchRequest version 3, we replace incomplete message
sets with an empty one as consumers can make
// progress in such cases and don't need to report a
`RecordTooLargeException`
FetchDataInfo(readInfo.fetchedData.fetchOffsetMetadata,
MemoryRecords.EMPTY)
```
@hachikuji Do you remember if there is still a reason to return `null` here
instead of the exception @ccding is proposing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #11080: fix NPE when record==null in append
ijuma commented on a change in pull request #11080:
URL: https://github.com/apache/kafka/pull/11080#discussion_r672784612
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
I think the intent here was to cover the case where an incomplete record
is returned by the broker. However, we have broker logic to try and avoid this
case since KIP-74:
```java
} else if (!hardMaxBytesLimit && readInfo.fetchedData.firstEntryIncomplete) {
// For FetchRequest version 3, we replace incomplete message
sets with an empty one as consumers can make
// progress in such cases and don't need to report a
`RecordTooLargeException`
FetchDataInfo(readInfo.fetchedData.fetchOffsetMetadata,
MemoryRecords.EMPTY)
```
@hachikuji Do you remember if there is still a reason to return `null` here
instead of the exception @ccding is proposing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #11080: fix NPE when record==null in append
ijuma commented on a change in pull request #11080:
URL: https://github.com/apache/kafka/pull/11080#discussion_r672610426
##
File path:
clients/src/main/java/org/apache/kafka/common/record/DefaultRecord.java
##
@@ -293,7 +293,9 @@ public static DefaultRecord readFrom(ByteBuffer buffer,
Long logAppendTime) {
int sizeOfBodyInBytes = ByteUtils.readVarint(buffer);
if (buffer.remaining() < sizeOfBodyInBytes)
-return null;
+throw new InvalidRecordException("Invalid record size: expected "
+ sizeOfBodyInBytes +
+" bytes in record payload, but instead the buffer has only " +
buffer.remaining() +
+" remaining bytes.");
Review comment:
Is this really an exceptional case? Don't we do reads where we don't
know exactly where the read ends and hence will trigger this path?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
