Sophie Blee-Goldman commented on KAFKA-7591:

Shouldn't you need to reset the app if the window size changes? On the other 
hand, it might be a good idea to at least verify the topic configs (for all 
internal topics), and log a warning or even throw an exception if they don't 

On a related note, users may want to increase the retention period to allow 
querying the state for longer – in that case it does seem reasonable for 
Streams to alter the changelog's retention. But it wouldn't be able to 
distinguish a change in retention from a change in window size, thus I think 
it's still better to just detect the discrepancy and alert the user so they can 
consider the best course of action (reset app or alter topic config)

> Changelog retention period doesn't synchronise with window-store size
> ---------------------------------------------------------------------
>                 Key: KAFKA-7591
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7591
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Jon Bates
>            Priority: Major
> When a new windowed state store is created, the associated changelog topic's 
> `retention.ms` value is set to `window-size + 
> h3. Expected Behaviour
> If the window-size is updated, the changelog topic's `retention.ms` config 
> should be updated to reflect the new size
> h3. Actual Behaviour
> The changelog-topic's `retention.ms` setting is not amended, resulting in 
> possible loss of data upon application restart
> n.b. Although it is easy to update changelog topic config, I logged this as 
> `major` due to the potential for data-loss for any user of Kafka-Streams who 
> may not be intimately aware of the relationship between a windowed store and 
> the changelog config

This message was sent by Atlassian Jira

Reply via email to