ASF GitHub Bot commented on KAFKA-9472:

C0urante commented on pull request #8118: KAFKA-9472: Remove deleted tasks from 
status store
URL: https://github.com/apache/kafka/pull/8118
   Although the statuses for tasks are removed from the status store when their 
_connector_ is deleted, their status are not removed when only the task is 
deleted, which happens in the case that the number of tasks for a connector is 
   These changes add logic for deleting the statuses for those tasks from the 
status store whenever a rebalance has completed and the leader of a distributed 
cluster has detected that there are recently-deleted tasks. Standalone is also 
updated to accomplish this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

> Reducing number of tasks for connector causes deleted tasks to show as 
> ---------------------------------------------------------------------------------
>                 Key: KAFKA-9472
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9472
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 
> 2.4.0, 2.3.1
>            Reporter: Chris Egerton
>            Assignee: Chris Egerton
>            Priority: Major
> If a connector is successfully created with {{t1}} running tasks and then 
> reconfigured to use {{t1 - n}} tasks (where {{t1}} and {{n}} are both whole 
> numbers and {{n}} is strictly less than {{t1}}), the connector should then 
> list {{t1 - n}} total tasks in its status (which can be queried via the 
> {{/connectors/:name:/status}} endpoint or the {{/connectors}} endpoint with 
> the {{expand}} URL query parameter set to {{status}}).
> However, the connector will instead continue to list {{t1}} total tasks in 
> its status, with {{n}} of them being listed as {{UNASSIGNED}} and the 
> remaining {{t1 - n}} of them being listed as {{STARTED}}.
> This is because the only time a task status is removed from the status 
> backing store (as opposed to simply being updated to {{UNASSIGNED}}) is when 
> its connector is deleted. See relevant code snippets from the 
> [AbstractHerder|https://github.com/apache/kafka/blob/df13fc93d0aebfe0ecc40dd4af3c5fb19b35f710/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java#L187-L192]
>  and 
> [DistributedHerder|https://github.com/apache/kafka/blob/df13fc93d0aebfe0ecc40dd4af3c5fb19b35f710/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L1511-L1520]
>  classes.

This message was sent by Atlassian Jira

Reply via email to