[ https://issues.apache.org/jira/browse/KAFKA-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Egerton reassigned KAFKA-15575: ------------------------------------- Assignee: Chris Egerton > Prevent Connectors from exceeding tasks.max configuration > --------------------------------------------------------- > > Key: KAFKA-15575 > URL: https://issues.apache.org/jira/browse/KAFKA-15575 > Project: Kafka > Issue Type: Task > Components: KafkaConnect > Reporter: Greg Harris > Assignee: Chris Egerton > Priority: Minor > > The Connector::taskConfigs(int maxTasks) function is used by Connectors to > enumerate tasks configurations. This takes an argument which comes from the > tasks.max connector config. This is the Javadoc for that method: > {noformat} > /** > * Returns a set of configurations for Tasks based on the current > configuration, > * producing at most {@code maxTasks} configurations. > * > * @param maxTasks maximum number of configurations to generate > * @return configurations for Tasks > */ > public abstract List<Map<String, String>> taskConfigs(int maxTasks); > {noformat} > This includes the constraint that the number of tasks is at most maxTasks, > but this constraint is not enforced by the framework. > > We should begin enforcing this constraint by dropping configs that exceed the > limit, and logging a warning. For sink connectors this should harmlessly > rebalance the consumer subscriptions onto the remaining tasks. For source > connectors that distribute their work via task configs, this may result in an > interruption in data transfer. -- This message was sent by Atlassian Jira (v8.20.10#820010)