The retention for the joins is as specified above. With until set to 240 and StreamsConfig.WINDOW_STORE_CHANGE_LOG_ADDITIONAL_RETENTION_MS_CONFIG set to 1 day (the default) this would be 100800000
For plain key value stores, there should be no retention period as the topics are compacted only. On Mon, 30 Oct 2017 at 15:48 Sameer Kumar <sam.kum.w...@gmail.com> wrote: > Actually I am using Key Value store, I do use join as part of my DAG(until > for the same has been set at 240 mins). The sink processor is key-value, is > there any option to control it. > > -Sameer. > > On Mon, Oct 30, 2017 at 6:33 PM, Damian Guy <damian....@gmail.com> wrote: > > > The topics in question are both changelogs for window stores. The > retention > > period for them is calculated as the Window retention period, which is > the > > value that is passed to `JoinWindows.until(...)` (default is 1 day) plus > > the value of the config > > StreamsConfig.WINDOW_STORE_CHANGE_LOG_ADDITIONAL_RETENTION_MS_CONFIG ( > > default is 1 day) > > > > > > > > On Mon, 30 Oct 2017 at 12:49 Sameer Kumar <sam.kum.w...@gmail.com> > wrote: > > > > > Hi, > > > > > > I have configured my settings to be the following:- > > > > > > log.retention.hours=3 > > > delete.topic.enable=true > > > delete.retention.ms=10800000 > > > min.cleanable.dirty.ratio=0.20 > > > segment.ms=180000 > > > > > > Howsoever, the changelog topic created as part of stream has the > > > rentention.ms to be 100800000, the source topic has it to be 3 hours. > > > > > > [root@dmpkafka6591 kafka-11-single]# bin/kafka-topics.sh --describe > > > --zookeeper 172.29.65.91:2181 --topic > > > c-7-aq7-KSTREAM-JOINOTHER-0000000019-store-changelog > > > Topic:c-7-aq7-KSTREAM-JOINOTHER-0000000019-store-changelog > > > PartitionCount:60 ReplicationFactor:1 Configs:retention.ms= > > > 100800000,cleanup.policy=delete,compact > > > > > > Can someone please explain this behavior. > > > > > > -Sameer. > > > > > >
