Giridhar Addepalli updated KAFKA-6645:
    Issue Type: New Feature  (was: Bug)

> Sticky Partition Assignment across Kafka Streams application restarts
> ---------------------------------------------------------------------
>                 Key: KAFKA-6645
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6645
>             Project: Kafka
>          Issue Type: New Feature
>          Components: streams
>            Reporter: Giridhar Addepalli
>            Priority: Major
> Since Kafka Streams applications have lot of state in the stores in the 
> general, it would be good to remember the assignment of partitions to 
> machines. So that when whole application is restarted for whatever reason, 
> there is a way to use past assignment of partitions to machines and there 
> won't be need to build up state by reading off of changelog kafka topic and 
> would result in faster start-up.
> Samza has support for Host Affinity 
> (https://samza.apache.org/learn/documentation/0.14/yarn/yarn-host-affinity.html)
> KIP-54 
> ([https://cwiki.apache.org/confluence/display/KAFKA/KIP-54+-+Sticky+Partition+Assignment+Strategy)]
>  , handles cases where some members of consumer group goes down / comes up, 
> and KIP-54 ensures there is minimal diff between assignments before and after 
> rebalance. 
> But to handle whole restart use case, we need to remember past assignment 
> somewhere, and use it after restart.
> Please let us know if this is already solved problem / some cleaner way of 
> achieving this objective

This message was sent by Atlassian JIRA

Reply via email to