[
https://issues.apache.org/jira/browse/KAFKA-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773988#comment-17773988
]
Mickael Maison commented on KAFKA-15575:
----------------------------------------
Thanks for opening this item. Regardless of KIP-987, I think this should be
fixed. I think as you said in the thread on the mailing list, I can't think of
a case where tasks.max was not correctly used in taskConfigs() and it was not a
bug.
> Prevent Connectors from exceeding tasks.max configuration
> ---------------------------------------------------------
>
> Key: KAFKA-15575
> URL: https://issues.apache.org/jira/browse/KAFKA-15575
> Project: Kafka
> Issue Type: Task
> Components: KafkaConnect
> Reporter: Greg Harris
> Priority: Minor
>
> The Connector::taskConfigs(int maxTasks) function is used by Connectors to
> enumerate tasks configurations. This takes an argument which comes from the
> tasks.max connector config. This is the Javadoc for that method:
> {noformat}
> /**
> * Returns a set of configurations for Tasks based on the current
> configuration,
> * producing at most {@code maxTasks} configurations.
> *
> * @param maxTasks maximum number of configurations to generate
> * @return configurations for Tasks
> */
> public abstract List<Map<String, String>> taskConfigs(int maxTasks);
> {noformat}
> This includes the constraint that the number of tasks is at most maxTasks,
> but this constraint is not enforced by the framework.
>
> We should begin enforcing this constraint by dropping configs that exceed the
> limit, and logging a warning. For sink connectors this should harmlessly
> rebalance the consumer subscriptions onto the remaining tasks. For source
> connectors that distribute their work via task configs, this may result in an
> interruption in data transfer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
