kaikai.hou created KAFKA-9070:
---------------------------------
Summary: Kafka Connect - Get the wrong offset value comes from
Kafka Connect after increase the number of offset storage topic partition to 3
Key: KAFKA-9070
URL: https://issues.apache.org/jira/browse/KAFKA-9070
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Affects Versions: 2.3.0
Environment: debezium/connect: 0.10
mysql: 5.6
kafka: 2.3.0
Reporter: kaikai.hou
I'm using the *Debezium* project with distributed mode.
*Problem:*
I found a problem : Get the wrong offset value comes from Kafka Connect after
increase the number of offset storage topic partition to 3.
1. Cluster mode, tow node (container);
2. Offset storage topic only have 1 partition;
3. Create 3 mysql connector ;
4. Change data, all connector have offset recored in partition 0.
4. *Increase offset storage topic partition number to 3*.
5. Change data, and some connector store their offset records to partition 1 or
partition 2.
6.* Restart all connect service, then all connector will read offset records
from partition 0*.
7. Then, the connector that stored offset records to partition 1 or partition 2
will get *repeat data*. (the offset records in partition 0 are* too old* for
this connectors)
*Debug*
The Debezium developer checked their code found that the partition handling is
delegated to Kafka Connect. [their
reply|https://issues.jboss.org/browse/DBZ-1551?focusedCommentId=13800286&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13800286]
Then, after they reproduced the problem, they found that [the problem is that
the incorrect offset is returned by Kafka
Connect|https://issues.jboss.org/browse/DBZ-1551?focusedCommentId=13801400&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13801400]
.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
