Matthias J. Sax created KAFKA-21488:
---------------------------------------

             Summary: StandbyUpdateListener should be called when state-updated 
goes into active-restoring mode
                 Key: KAFKA-21488
                 URL: https://issues.apache.org/jira/browse/KAFKA-21488
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Matthias J. Sax


The state-updater thread operates in two different modes: active-restoring and 
standby-updating. When there is at least one active task that needs to get 
restored, the active-restoring mode ensures that we get full bandwidth to get 
the active tasks into RUNNING state first, before we spent bandwidth to 
maintain standbys. Only if there is no active task that needs restoring, we use 
standby-updating state to maintain standby tasks.

After a rebalance, we may transit from standby-updating back to 
active-restoring, effectively pausing standby tasks for a potentially longer 
period of time. However, the standby-update-listener is not invoked for this 
case. The user would only know about this indirectly, if they have a restore 
listener; as a side effect, `onBatchLoaded()` would just not be called for a 
longer period of time.

Even worse, if we are in active-restoring mode and a rebalance assigns us a new 
standby task, we would invoke `onUpdateStart()` right away when initializing 
the standby task, even if we put the standby into "paused" mode right away, and 
would only effectively start maintaining it, when entering standby-updating 
mode.

We should consider to improve this, to give users better visibility, when 
standby tasks are effectively maintained vs paused. We might also need a new 
"suspend reason" code for the "pause" state. Not sure if we need a "resume" 
callback, or if the first `onBatchLoaded()` would be sufficient?

 



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to