[
https://issues.apache.org/jira/browse/KAFKA-9501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053897#comment-17053897
]
ASF GitHub Bot commented on KAFKA-9501:
---------------------------------------
ableegoldman commented on pull request #8248: KAFKA-9501: convert between
active and standby without closing stores
URL: https://github.com/apache/kafka/pull/8248
WIP -- need to add tests
Add methods to the active/standby task creators that reuse the state manager
to avoid closing the stores (and losing all data in the case of in-memory
stores)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Promote Standby tasks to active tasks without closing them
> ----------------------------------------------------------
>
> Key: KAFKA-9501
> URL: https://issues.apache.org/jira/browse/KAFKA-9501
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Sophie Blee-Goldman
> Priority: Critical
>
> The purpose of StandbyTasks in Kafka Streams is fast failover. However, atm
> we close StandbyTasks (and create a new active task) if they are promoted to
> active.
> While this works ok for persistent state stores, it renders hot standbys for
> in-memory state stores useless, because we drop the in-memory state when we
> close the StandbyTask and thus the new active tasks needs to reread the
> changelog topic to recreate the in-memory state.
> Hence, we should promote StandbyTasks to active tasks without closing them.
> This will not only fix the issue for in-memory stores, but will make
> rebalancing faster for persistent state stores, too, because closing and
> reopening RocksDB has significant overhead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
