[
https://issues.apache.org/jira/browse/KAFKA-7777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749214#comment-16749214
]
Matthias J. Sax commented on KAFKA-7777:
----------------------------------------
This is an interesting request. Note, that in state restore, Kafka Streams
currently only copies the key/value bytes from the changelog topic unmodified
into the state store. This change would required to deserialize the data using
the topic serde, and serialize them again using the state store serde on
restore. Thus, fail-over time might increase.
> Decouple topic serdes from materialized serdes
> ----------------------------------------------
>
> Key: KAFKA-7777
> URL: https://issues.apache.org/jira/browse/KAFKA-7777
> Project: Kafka
> Issue Type: Wish
> Components: streams
> Reporter: Maarten
> Priority: Trivial
>
> It would be valuable to us to have the the encoding format in a Kafka topic
> decoupled from the encoding format used to cache the data locally in a kafka
> streams app.
> We would like to use the `range()` function in the interactive queries API to
> look up a series of results, but can't with our encoding scheme due to our
> keys being variable length.
> We use protobuf, but based on what I've read Avro, Flatbuffers and Cap'n
> proto have similar problems.
> Currently we use the following code to work around this problem:
> {code}
> builder
> .stream("input-topic", Consumed.with(inputKeySerde, inputValueSerde))
> .to("intermediate-topic", Produced.with(intermediateKeySerde,
> intermediateValueSerde));
> t1 = builder
> .table("intermediate-topic", Consumed.with(intermediateKeySerde,
> intermediateValueSerde), t1Materialized);
> {code}
> With the encoding formats decoupled, the code above could be reduced to a
> single step, not requiring an intermediate topic.
> Based on feedback on my [SO
> question|https://stackoverflow.com/questions/53913571/is-there-a-way-to-separate-kafka-topic-serdes-from-materialized-serdes]
> a change that introduces this would impact state restoration when using an
> input topic for recovery.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
