[
https://issues.apache.org/jira/browse/KAFKA-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Chen updated KAFKA-5966:
-----------------------------
Description:
context: https://github.com/apache/kafka/pull/3027/files#r117378317
Currently Kafka Streams only supports serialization using byte arrays. This
means we generate a lot of garbage and spend unnecessary time copying bytes,
especially when working with windowed state stores that rely on composite keys.
In many places in the code we have extract parts of the composite key to
deserialize the either the timestamp or the message key from the state store
key (e.g. the methods in WindowStoreUtils)
Having support for serde into/from ByteBuffers would allow us to reuse the
underlying bytearrays and just pass around slices of the underlying Buffers to
avoid the unnecessary copying.
was:
Currently Kafka Streams only supports serialization using byte arrays. This
means we generate a lot of garbage and spend unnecessary time copying bytes,
especially when working with windowed state stores that rely on composite keys.
In many places in the code we have extract parts of the composite key to
deserialize the either the timestamp or the message key from the state store
key (e.g. the methods in WindowStoreUtils)
Having support for serde into/from ByteBuffers would allow us to reuse the
underlying bytearrays and just pass around slices of the underlying Buffers to
avoid the unnecessary copying.
> Support ByteBuffer serialization in Kafka Streams
> -------------------------------------------------
>
> Key: KAFKA-5966
> URL: https://issues.apache.org/jira/browse/KAFKA-5966
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Xavier Léauté
> Assignee: Luke Chen
> Priority: Major
>
> context: https://github.com/apache/kafka/pull/3027/files#r117378317
>
> Currently Kafka Streams only supports serialization using byte arrays. This
> means we generate a lot of garbage and spend unnecessary time copying bytes,
> especially when working with windowed state stores that rely on composite
> keys. In many places in the code we have extract parts of the composite key
> to deserialize the either the timestamp or the message key from the state
> store key (e.g. the methods in WindowStoreUtils)
> Having support for serde into/from ByteBuffers would allow us to reuse the
> underlying bytearrays and just pass around slices of the underlying Buffers
> to avoid the unnecessary copying.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
