[
https://issues.apache.org/jira/browse/KAFKA-21396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax reassigned KAFKA-21396:
---------------------------------------
Assignee: Matthias J. Sax
> Kafka Streams should report active task lag when restore status changes
> -----------------------------------------------------------------------
>
> Key: KAFKA-21396
> URL: https://issues.apache.org/jira/browse/KAFKA-21396
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.4.0
> Reporter: Matthias J. Sax
> Assignee: Matthias J. Sax
> Priority: Minor
> Fix For: 4.5.0, 4.4.1
>
>
> Kafka Streams using "streams" protocol reports task offsets for active
> restoring tasks as well as standby/warmup tasks. When an active task is
> RUNNING, no offsets are reported. This allows the
> broker/group-coordinator/refiner to distinguish between active-restoring and
> active-running tasks.
> However, when an active-restoring task transits from RESTORING to RUNNING, no
> immediate task-offset-push is triggered, but it only happens on the next
> reporting interval (task.offset.interval.ms, default 60 sec). If a rebalance
> happens during this time, the GC might make an incorrect decision base on
> stale data).
> For warmup-task, the client also reports task offsets on
> `task.offset.interval.ms` during regular restore, but if a warmup task
> becomes "hot" it report on every heartbeat. We should do the same thing for
> active-restoring task, triggering a task-offset report when the
> active-restoring task becomes "hot" and additionally, when is transits to
> RUNNING.
> While we cannot close the race condition entirely, as we are still subject to
> the heartbeat interval (default: 5sec), it's still a significant improvement.
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
