[
https://issues.apache.org/jira/browse/KAFKA-21312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129022#comment-18129022
]
sanghyeok An commented on KAFKA-21312:
--------------------------------------
[~squah-confluent]
Ah, thank you for letting me know!
Let me first share the implementation approach I had in mind.
I was considering extending {{{}CoordinatorTimerImpl{}}}. So far, I've only
thought through the overall flow, and I was planning to validate its
feasibility through a small PoC before working out the implementation details.
# I was planning to add snapshot-related fields to
{{{}CoordinatorTimerImpl{}}}. Basically, the object to be snapshotted would
contain only a deadline value.
# When the Coordinator calls a method such as {{snapshotWithCancel()}} on
{{{}CoordinatorTimerImpl{}}}, the timer currently scheduled in the Timing Wheel
would be canceled, and its deadline would be stored in a snapshot object
registered with the {{{}SnapshotRegistry{}}}.
# If the Coordinator successfully persists the records, it would proceed as
usual. However, if record persistence fails, I was thinking of calling
something like a {{revert()}} method on the timer to restore its previous
snapshot, similar to how {{TimelineHashMap}} handles rollback. (I don't
remember the exact method name, so this is just a conceptual description.)
# At this point, I expect that rescheduling the {{TimerTask}} restored to the
previous epoch in the Timing Wheel with its original deadline would allow the
subsequent processing to follow a flow similar to the existing one.
This is roughly the approach I have in mind.
What do you think? Thanks for reading this!
> Snapshottable tasks in the coordinator runtime
> ----------------------------------------------
>
> Key: KAFKA-21312
> URL: https://issues.apache.org/jira/browse/KAFKA-21312
> Project: Kafka
> Issue Type: Improvement
> Components: group-coordinator
> Reporter: Sean Quah
> Assignee: Sean Quah
> Priority: Major
>
> In a couple of places, the coordinator runtime and group coordinator create
> and destroy tasks, eg. for regex resolution and (soon) asynchronous
> assignment calculations.
> When coordinator state is rolled back, these tasks are not reverted and, when
> they complete, we have to think carefully about how we can discard their
> results if they no longer apply. It would be nice to have these tasks
> participate in the snapshot mechanism.
> Analogous to KAFKA-19023 but for the CoordinatorExecutor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
