A. Sophie Blee-Goldman created KAFKA-16867:
----------------------------------------------
Summary: Streams should run tag-based standby assignment based on
rack ids
Key: KAFKA-16867
URL: https://issues.apache.org/jira/browse/KAFKA-16867
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: A. Sophie Blee-Goldman
In KIP-708, we introduced a tag-based standby task assignment algorithm that
runs if the user has configured their clients with "rack aware assignment
tags". If no tags are configured, the default load-based standby task
assignment algorithm is run instead.
In KIP-924 we introduced a different kind of rack-aware assignment logic which
is based on the "rack.id" of the consumers and topic partitions. While this did
not replace the tag-based rack-aware assignment of KIP-708 which had different
(and opposing) goals, we realized that Streams could leverage the rack.ids to
run the tag-based standby task assignment algorithm even if clients were not
configured with assignment tags.
Unfortunately, during implementation of KIP-924, a bug in the logic meant that
the tag-based algorithm was never actually being run based on the rack ids.
This bug is present to this day and carried over (intentionally) during the
task assignor refactoring of KIP-924.
We should still fix this bug so that users can benefit from the resiliency of
KIP-708 based on consumer rack ids, even if they did not explicitly opt-in by
configuring clients with assignment tags, since KIP-708 is a net benefit with
no downside
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
