[
https://issues.apache.org/jira/browse/KAFKA-21060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ziyun Fu reassigned KAFKA-21060:
--------------------------------
Assignee: Ziyun Fu
> Make StickyAssignor aware of stateless vs stateful tasks
> --------------------------------------------------------
>
> Key: KAFKA-21060
> URL: https://issues.apache.org/jira/browse/KAFKA-21060
> Project: Kafka
> Issue Type: Improvement
> Components: group-coordinator, streams
> Reporter: Matthias J. Sax
> Assignee: Ziyun Fu
> Priority: Major
>
> Kafka Streams currently ships three implementation of sticky task assignor:
> * LegacyStickyTaskAssignor (internal only – classic protocol)
> *
> o.a.k.streams.processor.assignment.assignors.StickyTaskAssignor (base on
> public API – classic protocol)
> *
> o.a.k.coordinator.group.streams.assignor.StickyTaskAssignor (streams protocol)
> All three implementation basically round-robin across all task, and they do
> not distinguish between stateless and stateful tasks.
> It might be a good improvement, to actually have a "task bucket" for each
> flavor (stateless and stateful), and round-robin over each bucket
> independently, to ensure that we don't end up with two stateless tasks
> assigned to one member, and two stateful tasks assigned to a different member.
> We should also double check how we compute load, and might want to have two
> load metrics, one for stateless load and one for stateful load.
> Not sure if we want to make this change in all three assignors, or maybe only
> for the new streams protocol one, to support adoption of KIP-1071. Note, for
> "classic" sticky assignor is not the default, while for "streams" it is...
> As this is only a high level idea, we might also want to do some socialized
> testing to see if such an change is really beneficial or not? Not sure. Maybe
> we can also assess intellectually that it's a good idea, w/o specific
> "benchmarking"?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
