[
https://issues.apache.org/jira/browse/KAFKA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766107#comment-17766107
]
Yash Mayya commented on KAFKA-15470:
------------------------------------
https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+Allow+creating+connectors+in+a+stopped+state
> Allow creating connectors in a stopped state
> --------------------------------------------
>
> Key: KAFKA-15470
> URL: https://issues.apache.org/jira/browse/KAFKA-15470
> Project: Kafka
> Issue Type: New Feature
> Components: connect, KafkaConnect
> Reporter: Yash Mayya
> Assignee: Yash Mayya
> Priority: Major
> Labels: connect, kafka-connect, kip-required
> Fix For: 3.7.0
>
>
> [KIP-875: First-class offsets support in Kafka
> Connect|https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect]
> introduced a new {{STOPPED}} state for connectors along with some REST API
> endpoints to retrieve and modify offsets for connectors. Currently, only
> connectors that already exist can be stopped and any newly created connector
> will always be in the {{RUNNING}} state initially. Allowing the creation of
> connectors in a {{STOPPED}} state will facilitate multiple new use cases. One
> interesting use case would be to migrate connectors from one Kafka Connect
> cluster to another. Individual connector migration would be useful in a
> number of scenarios such as breaking a large cluster into multiple smaller
> clusters (or vice versa), moving a connector from a cluster running in one
> data center to another etc. A connector migration could be achieved by using
> the following sequence of steps :-
> # Stop the running connector on the original Kafka Connect cluster
> # Retrieve the offsets for the connector via the {{GET
> /connectors/\{connector}/offsets}} endpoint
> # Create the connector in a stopped state using the same configuration on
> the new Kafka Connect cluster
> # Alter the offsets for the connector on the new cluster via the {{PATCH
> /connectors/\{connector}/offsets}} endpoint (using the offsets obtained from
> the original cluster)
> # Resume the connector on the new cluster and delete it on the original
> cluster
> Another use case for creating connectors in a stopped state could be
> deploying connectors as a part of a larger data pipeline before the source /
> sink data system has been created or is ready for data transfer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
