CC-Hsu opened a new issue, #20656: URL: https://github.com/apache/pulsar/issues/20656
### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation Currently the Pulsar I/O DBZ connector is v1.9.7. https://github.com/apache/pulsar/blob/v3.0.0/pom.xml#L194 https://github.com/apache/pulsar/blob/master/pom.xml#L195 Previously there was an update from DBZ v1.0. https://github.com/apache/pulsar/issues/12418 Recently there are many notable new features for the recent releases, and I list several big features of them. * [DBZ v2.3](https://debezium.io/releases/2.3/release-notes) - Support exactly-once semantic for streaming phase from Postgres connector [DBZ-6547](https://issues.redhat.com/browse/DBZ-6547) - PostgreSQL: Set Replica Identity when the connector starts [DBZ-6112](https://issues.redhat.com/browse/DBZ-6112) - Support multiple tasks when streaming shard list [DBZ-6365](https://issues.redhat.com/browse/DBZ-6365) * [DBZ v2.2](https://debezium.io/releases/2.2/release-notes) - Capture evenents in order across mongodb shards [DBZ-5590](https://issues.redhat.com/browse/DBZ-5590) - Debezium JDBC Sink Connector [DBZ-3647](https://issues.redhat.com/browse/DBZ-3647) * [DBZ v2.1](https://debezium.io/releases/2.1/release-notes) - Implement support for JSON_TABLE in MySQL parser [DBZ-3575](https://issues.redhat.com/browse/DBZ-3575) - Support for Postgres 15 [DBZ-5370](https://issues.redhat.com/browse/DBZ-5370) - MySQL Connector capture TRUNCATE command as message in table topic [DBZ-5610](https://issues.redhat.com/browse/DBZ-5610) * [DBZ v2.0](https://debezium.io/releases/2.0/release-notes) - Improve LogMiner query performance by reducing REGEXP_LIKE disjunctions [DBZ-5648](https://issues.redhat.com/browse/DBZ-5648) - Added support for Mongo pre-image in change stream [DBZ-5628](https://issues.redhat.com/browse/DBZ-5628) - Support binlog compression for MySQL [DBZ-2663](https://issues.redhat.com/browse/DBZ-2663) - support mongodb connection string as configuration option [DBZ-4733](https://issues.redhat.com/browse/DBZ-4733) - Mysql Commit Timestamp [DBZ-5170](https://issues.redhat.com/browse/DBZ-5170) And there is [a whole new DBZ JDBC sink connector available](https://debezium.io/documentation/reference/2.3/connectors/jdbc.html). It would be great if the Pulsar I/O DBZ connector can also be updated and including these new exciting features if possible. ### Solution 1. Update Pulsar IO DBZ connector to at lease v2.2 (update to the latest one is more appreciated), which contains the DBZ JDBC sink connector, and MongoDB sharding support. 2. Adding [the DBZ JDBC sink connector](https://debezium.io/documentation/reference/2.3/connectors/jdbc.html) to Pulsar IO CDC connector ### Alternatives _No response_ ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
