Chris Egerton created KAFKA-15693:
-------------------------------------
Summary: Disabling scheduled rebalance delay in Connect can lead
to indefinitely unassigned connectors and tasks
Key: KAFKA-15693
URL: https://issues.apache.org/jira/browse/KAFKA-15693
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Affects Versions: 3.5.1, 3.6.0, 3.4.1, 3.5.0, 3.3.2, 3.3.1, 3.2.3, 3.2.2,
3.4.0, 3.2.1, 3.1.2, 3.0.2, 3.3.0, 3.1.1, 3.2.0, 2.8.2, 3.0.1, 3.0.0, 2.8.1,
2.7.2, 2.6.3, 3.1.0, 2.6.2, 2.7.1, 2.8.0, 2.6.1, 2.7.0, 2.5.1, 2.6.0, 2.4.1,
2.5.0, 2.3.1, 2.4.0, 2.3.0, 3.7.0
Reporter: Chris Egerton
Assignee: Chris Egerton
Kafka Connect supports deferred resolution of imbalances when using the
incremental rebalancing algorithm introduced in
[KIP-415|https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect].
When enabled, this feature introduces a configurable delay period between when
"lost" assignments (i.e., connectors and tasks that were assigned to a worker
in the previous round of rebalance but are not assigned to a worker during the
current round of rebalance) are detected and when they are reassigned to a
worker. The delay can be configured with the
{{scheduled.rebalance.max.delay.ms}} property.
If this property is set to 0, then there should be no delay between when lost
assignments are detected and when they are reassigned. Instead, however, this
configuration can cause lost assignments to be withheld during a rebalance,
remaining unassigned until the next rebalance, which, because scheduled delays
are disabled, will not happen on its own and will only take place when
unrelated conditions warrant it (such as the creation or deletion of a
connector, a worker joining or leaving the cluster, new task configs being
generated for a connector, etc.).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
