[
https://issues.apache.org/jira/browse/KAFKA-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708021#comment-16708021
]
John Roesler commented on KAFKA-7591:
-------------------------------------
I looked into the code, and for reference, the reason the config doesn't get
propagated is that in
`org.apache.kafka.streams.processor.internals.InternalTopicManager#makeReady`,
we first fetch the number of partitions for each internal topic in the
topology. If the topic exists and has the right number of partitions, then we
do nothing.
There's no idempotent create operation, so we'd either have to idempotently
"alter configs" on every topic or fetch the configs and only alter the ones
that don't match the configs that come from the topology.
I don't think it would be practical, without changing the code structure, to do
this only for window stores, and it might be nice to have changes from other
internal topic configs propagated.
> Changelog retention period doesn't synchronise with window-store size
> ---------------------------------------------------------------------
>
> Key: KAFKA-7591
> URL: https://issues.apache.org/jira/browse/KAFKA-7591
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Jon Bates
> Priority: Major
>
> When a new windowed state store is created, the associated changelog topic's
> `retention.ms` value is set to `window-size +
> CHANGELOG_ADDITIONAL_RETENTION_MS`
> h3. Expected Behaviour
> If the window-size is updated, the changelog topic's `retention.ms` config
> should be updated to reflect the new size
> h3. Actual Behaviour
> The changelog-topic's `retention.ms` setting is not amended, resulting in
> possible loss of data upon application restart
>
> n.b. Although it is easy to update changelog topic config, I logged this as
> `major` due to the potential for data-loss for any user of Kafka-Streams who
> may not be intimately aware of the relationship between a windowed store and
> the changelog config
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
