Guozhang Wang commented on KAFKA-6555:

I would say yes; it changes the state protocol of a Streams application, hence 
worth being discussed inside the community.

> Making state store queryable during restoration
> -----------------------------------------------
>                 Key: KAFKA-6555
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6555
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Ashish Surana
>            Priority: Major
> State store in Kafka streams are currently only queryable when StreamTask is 
> in RUNNING state. The idea is to make it queryable even in the RESTORATION 
> (PARTITION_ASSIGNED) state as the time spend on restoration can be huge and 
> making the data inaccessible during this time could be downtime not suitable 
> for many applications.
> When the active partition goes down then one of the following occurs:
>  # One of the standby replica partition gets promoted to active: Replica task 
> has to restore the remaining state from the changelog topic before it can 
> become RUNNING. The time taken for this depends on how much the replica is 
> lagging behind. During this restoration time the state store for that 
> partition is currently not queryable giving making the partition down. We can 
> make the state store partition queryable for the data already present in the 
> state store.
>  # When there is no replica or standby task, then active task will be started 
> in one of the existing node. That node has to build the entire state from the 
> changelog topic which can take lot of time depending on how big is the 
> changelog topic, and keeping state store not queryable during this time is 
> the downtime for the parition.
> It's very important improvement as it could simply improve the availability 
> of microservices developed using kafka streams.
> I am working on a patch for this change. Any feedback or comments are welcome.

This message was sent by Atlassian JIRA

Reply via email to