Philip Nee created KAFKA-17825:
----------------------------------
Summary: ByteBufferDeserializaer's array size can be inconsistent
with the older version
Key: KAFKA-17825
URL: https://issues.apache.org/jira/browse/KAFKA-17825
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 3.7.1, 3.8.0, 3.6.2, 3.6.1, 3.7.0, 3.6.0, 3.9.0, 3.8.1,
3.9.1
Reporter: Philip Nee
Assignee: LinShunkang
We've noticed that using the ByteBufferDeserializer can yield a different byte
array length compare to the deserializer from 3.5.2. This is attributed by
KIP-863, in particular, the old deserializer truncated the byte array starting
from
`buffer.position() + buffer.arrayOffset() + offset` using `Utils.toArray`
Whereas the current implementation is a passthrough.
This can be reproduced using the
{code:java}
KafkaConsumerProducerDemo.java{code}
by changing the type to <Integer, ByteBuffer> and perform a print after poll.
For example, the producer produces a record [0, test0] (key is an int, "test0"
is a 5 bytes long string, converted to byte buffer using
{code:java}
ByteBuffer.wrap(value.getBytes()){code}
Prior to KIP-863 we see the following after polling the record from consumer:
3.5.2: test0
3.6.0:
{code:java}
?$���y�NNޅ�-p�=�����NAc�����8D���8D���������������
test0{code}
And if you analyze the ByteBuffer post 3.6.0, we can see the current position
is at 140 with array length of 149.
[~LSK] - since you wrote the kip and did the implementation, can you address
this ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
