Richard Yu updated KAFKA-4696:
    External issue URL:   (was: 

> Streams standby task assignment should be state-store aware
> -----------------------------------------------------------
>                 Key: KAFKA-4696
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4696
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>    Affects Versions:,
>            Reporter: Damian Guy
>            Priority: Major
> Task Assignment is currently not aware of which tasks have State Stores. This 
> can result in uneven balance of standby task assignment as all tasks are 
> assigned, but only those tasks with state-stores are ever created by 
> {{StreamThread}}. So what seems like an optimal strategy during assignment 
> time could be sub-optimal post-assignment.
> For example, lets say we have 4 tasks (2 with state-stores), 2 clients, 
> numStandbyReplicas = 1. Each client would get 2 active and 2 standby tasks.  
> One of the clients may end up with both state-store tasks, while the other 
> has none.
> Further to this, standby task configuration is currently "all or nothing". It 
> might make sense to allow more fine grained configuration, i.e., the ability 
> to specify the number of standby replicas individually for each stateful 
> operator.

This message was sent by Atlassian JIRA

Reply via email to