Ashish Surana created KAFKA-6555:
------------------------------------
Summary: Making state store queryable during restoration
Key: KAFKA-6555
URL: https://issues.apache.org/jira/browse/KAFKA-6555
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Ashish Surana
State store in Kafka streams are currently only queryable when StreamTask is in
RUNNING state. The idea is to make it queryable even in the RESTORATION
(PARTITION_ASSIGNED) state as the time spend on restoration can be huge and
making the data inaccessible during this time could be downtime not suitable
for many applications.
When the active partition goes down then one of the following occurs:
# One of the standby replica partition gets promoted to active: Replica
restores the remaining state from the changelog topic. The time taken for this
depends on how much the replica is lagging behind in consuming changelog topic.
During this restoration time the state store for that partition is not
queryable giving us the downtime for the entire parition. We can make it
queryable for the data already present in the state store.
# When there is no replica or standby task, then active task will be started
in one of the existing node. That node has to build the entire state from the
changelog topic which can take lot of time depending on how big is the
changelog topic, and keeping state store not queryable during this time is the
downtime for the parition.
It's very important improvement as it could simply improve the availability of
microservices developed using kafka streams. Any feedback or comments are
welcome.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
