Matthias J. Sax created KAFKA-9501:
--------------------------------------
Summary: Promote Standby tasks to active tasks without closing them
Key: KAFKA-9501
URL: https://issues.apache.org/jira/browse/KAFKA-9501
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: Matthias J. Sax
The purpose of StandbyTasks in Kafka Streams is fast failover. However, atm we
close StandbyTasks (and create a new active task) if they are promoted to
active.
While this works ok for persistent state stores, it renders hot standbys for
in-memory state stores useless, because we drop the in-memory state when we
close the StandbyTask and thus the new active tasks needs to reread the
changelog topic to recreate the in-memory state.
Hence, we should promote StandbyTasks to active tasks without closing them.
This will not fix the issue for in-memory stores, but will make rebalancing
faster for persistent state stores, too, because closing and reopening RocksDB
has significant overhead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
