[
https://issues.apache.org/jira/browse/KAFKA-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated KAFKA-6273:
----------------------------
Description:
Currently, the internal name of a windowing join is generated at runtime using
a counter to ensure uniqueness within a running topology.
If a topology is changed and redeployed, different names can be generated for
the same window. This can result in lost windowing state without intervention
modifying offsets to re-consume the input topics.
https://kafka.apache.org/10/javadoc/org/apache/kafka/streams/kstream/Joined.html
The proposed change would be to add an optional configuration parameter
"joinName" to Joined. If provided, this would be used by KStreamImpl to
generate internal names when building a join. If not provided the existing name
generation would be used. (Since this is an opt-in, optional parameter there
would be no impact on existing code.)
Let me know if this should be a KIP instead. The change would be something
along [these lines|https://github.com/FundingCircle/kafka/pull/3/files].
was:
Currently, the internal name of a windowing join is generated at runtime using
a counter to ensure uniqueness within a running topology.
If a topology is changed and redeployed, different names can be generated for
the same window. This can result in lost windowing state without intervention
modifying offsets to re-consume the input topics.
https://kafka.apache.org/10/javadoc/org/apache/kafka/streams/kstream/Joined.html
The proposed change would be to add an optional configuration parameter
"joinName" to Joined. If provided, this would be used by KStreamImpl to
generate internal names when building a join. If not provided the existing name
generation would be used. (Since this is an opt-in, optional parameter there
would be no impact on existing code.)
> Allow names for windowing joins
> -------------------------------
>
> Key: KAFKA-6273
> URL: https://issues.apache.org/jira/browse/KAFKA-6273
> Project: Kafka
> Issue Type: Improvement
> Reporter: Matthias
>
> Currently, the internal name of a windowing join is generated at runtime
> using a counter to ensure uniqueness within a running topology.
> If a topology is changed and redeployed, different names can be generated for
> the same window. This can result in lost windowing state without intervention
> modifying offsets to re-consume the input topics.
> https://kafka.apache.org/10/javadoc/org/apache/kafka/streams/kstream/Joined.html
> The proposed change would be to add an optional configuration parameter
> "joinName" to Joined. If provided, this would be used by KStreamImpl to
> generate internal names when building a join. If not provided the existing
> name generation would be used. (Since this is an opt-in, optional parameter
> there would be no impact on existing code.)
> Let me know if this should be a KIP instead. The change would be something
> along [these lines|https://github.com/FundingCircle/kafka/pull/3/files].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
