[
https://issues.apache.org/jira/browse/KAFKA-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17778727#comment-17778727
]
Matthias J. Sax edited comment on KAFKA-4852 at 1/9/25 9:04 PM:
----------------------------------------------------------------
As reported on https://issues.apache.org/jira/browse/KAFKA-15602, this ticket
introduces a couple of breaking changes and bugs.
# Semantics changes like this, need to be backed up by a KIP
# The code does not achieve what it aims to do
# There is lack of proper unit testing
Given that such a change requires a KIP and the current code is broken, we
propose to revert this change for the time being, and do a proper fix via a
KIP. I prepare a PR for this: [https://github.com/apache/kafka/pull/14617]
\cc [~guozhang]
was (Author: mjsax):
As reported on https://issues.apache.org/jira/browse/KAFKA-15602, this ticket
introduces a couple of breaking changes and bugs.
# Semantics changes like this, need to be backed up by a KIP
# The code does not achieve what it aims to do
# The is lack of proper unit testing
Given that such a change requires a KIP and the current code is broken, we
propose to revert this change for the time being, and do a proper fix via a
KIP. I prepare a PR for this: [https://github.com/apache/kafka/pull/14617]
\cc [~guozhang]
> ByteBufferSerializer not compatible with offsets
> ------------------------------------------------
>
> Key: KAFKA-4852
> URL: https://issues.apache.org/jira/browse/KAFKA-4852
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.1.1
> Environment: all
> Reporter: Werner Daehn
> Priority: Minor
> Labels: needs-kip, serializers
>
> Quick intro: A ByteBuffer.rewind() resets the position to zero. What if the
> ByteBuffer was created with an offset? new ByteBuffer(data, 3, 10)? The
> ByteBufferSerializer will send from pos=0 and not from pos=3 onwards.
> Solution: No rewind() but flip() for reading a ByteBuffer. That's what the
> flip is meant for.
> Story:
> Imagine the incoming data comes from a byte[], e.g. a network stream
> containing topicname, partition, key, value, ... and you want to create a new
> ProducerRecord for that. As the constructor of ProducerRecord requires
> (topic, partition, key, value) you have to copy from above byte[] the key and
> value. That means there is a memcopy taking place. Since the payload can be
> potentially large, that introduces a lot of overhead. Twice the memory.
> A nice solution to this problem is to simply wrap the network byte[] into new
> ByteBuffers:
> ByteBuffer key = ByteBuffer.wrap(data, keystart, keylength);
> ByteBuffer value = ByteBuffer.wrap(data, valuestart, valuelength);
> and then use the ByteBufferSerializer instead of the ByteArraySerializer.
> But that does not work as the ByteBufferSerializer does a rewind(), hence
> both, key and value, will start at position=0 of the data[].
> public class ByteBufferSerializer implements Serializer<ByteBuffer> {
> public byte[] serialize(String topic, ByteBuffer data) {
> data.rewind();
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
