Hi Samza devs and users

This is my first try with KeyValueStore and I am really excited!

I glanced through TaskStorageManager source code, it looks creates
consumers for stores and I am wondering that how kafka cleanup will be
propagated to KeyValueStore.

My KeyValueStore usage is a little bit different from usual cases because
 I have to cache all unique ids for the past six hours, which can be
configured for the retention usage. Unique ids won't be repeated such as
timestamp. In this case, log.cleanup.policy=compact will keep growing the
KeyValueStore size, right?

Can I use Samza KeyValueStore for the topics
with log.cleanup.policy=delete? If not, what's your recommended way for
state management of non-changelog Kafka topic? If it's possible, how does
Kafka cleanup remove outdated records in KeyValueStore?

Thank you
Best, Jae

Reply via email to