[
https://issues.apache.org/jira/browse/KAFKA-6645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridhar Addepalli updated KAFKA-6645:
--------------------------------------
Summary: Host Affinity to facilitate faster restarts of kafka streams
applications (was: Sticky Partition Assignment to facilitate faster restarts
of kafka streams applications)
> Host Affinity to facilitate faster restarts of kafka streams applications
> -------------------------------------------------------------------------
>
> Key: KAFKA-6645
> URL: https://issues.apache.org/jira/browse/KAFKA-6645
> Project: Kafka
> Issue Type: New Feature
> Components: streams
> Reporter: Giridhar Addepalli
> Priority: Major
>
> Since Kafka Streams applications have lot of state in the stores in general,
> it would be good to remember the assignment of partitions to machines. So
> that when whole application is restarted for whatever reason, there is a way
> to use past assignment of partitions to machines and there won't be need to
> build up state by reading off of changelog kafka topic and would result in
> faster start-up.
> Samza has support for Host Affinity
> ([https://samza.apache.org/learn/documentation/0.14/yarn/yarn-host-affinity.html])
> KIP-54
> ([https://cwiki.apache.org/confluence/display/KAFKA/KIP-54+-+Sticky+Partition+Assignment+Strategy)]
> , handles cases where some members of consumer group goes down / comes up,
> and KIP-54 ensures there is minimal diff between assignments before and after
> rebalance.
> But to handle whole restart use case, we need to remember past assignment
> somewhere, and use it after restart.
> Please let us know if this is already solved problem / some cleaner way of
> achieving this objective
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
