Ashish Surana updated KAFKA-6555:
    Flags: Important

> Making state store queryable during restoration
> -----------------------------------------------
>                 Key: KAFKA-6555
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6555
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Ashish Surana
>            Priority: Major
> State store in Kafka streams are currently only queryable when StreamTask is 
> in RUNNING state. The idea is to make it queryable even in the RESTORATION 
> (PARTITION_ASSIGNED) state as the time spend on restoration can be huge and 
> making the data inaccessible during this time could be downtime not suitable 
> for many applications.
> When the active partition goes down then one of the following occurs:
>  # One of the standby replica partition gets promoted to active: Replica 
> restores the remaining state from the changelog topic. The time taken for 
> this depends on how much the replica is lagging behind in consuming changelog 
> topic. During this restoration time the state store for that partition is not 
> queryable giving us the downtime for the entire parition. We can make it 
> queryable for the data already present in the state store.
>  # When there is no replica or standby task, then active task will be started 
> in one of the existing node. That node has to build the entire state from the 
> changelog topic which can take lot of time depending on how big is the 
> changelog topic, and keeping state store not queryable during this time is 
> the downtime for the parition.
> It's very important improvement as it could simply improve the availability 
> of microservices developed using kafka streams. Any feedback or comments are 
> welcome.

This message was sent by Atlassian JIRA

Reply via email to