Hi - I have a question which is mostly to clarify some conceptions regarding state management and restore functionality using Kafka Streams ..
When I have multiple instances of the same application running (same application id for each of the instances), are the following assumptions correct ? 1. each instance has a separate state store (local) 2. all instances are backed up by a *single* changelog topic Now the question arises, how does restore work in the above case when we have 1 changelog topic backing up multiple state stores ? Each instance of the application ingests data from specific partitions of the topic. And there can be multiple topics too. e.g. if we have m topics with n partitions in each, and p instances of the application, then all the (m x n) partitions are distributed across the p instances of the application. Is this true ? If so, then does the changelog topic also has (m x n) partitions, so that Kafka knows which state to restore in which store in case of a restore operation ? And finally, if we decide to merge topics / partitions in between without complete reset of the application, will (a) it work ? and (b) the changelog topic gets updated accordingly and (c) is this recommended ? regards. -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg
