dennisylyung commented on issue #6034: [Pulsar IO][Issue 5633]Support avro schema for debezium connector URL: https://github.com/apache/pulsar/pull/6034#issuecomment-613278588 > @dennisylyung Hi, could you provide some config info about the error logs, such as the pulsar source config and the reproduce steps, thanks. > > This is my test steps to follow the demo http://pulsar.apache.org/docs/en/io-debezium-source/#example-of-mysql, but I didn't reproduce your problem. > > ``` > # start debezium > docker run -it --rm \ > --name mysql \ > -p 3306:3306 \ > -e MYSQL_ROOT_PASSWORD=debezium \ > -e MYSQL_USER=mysqluser \ > -e MYSQL_PASSWORD=mysqlpw debezium/example-mysql:0.8 > > # start pulsar > bin/pulsar standalone > > # start source connector yaml > bin/pulsar-admin source localrun \ > --source-config-file debezium-mysql-source-config.yaml > > # subscribe the topic > bin/pulsar-client consume -s "sub-products" public/default/dbserver1.inventory.products -n 0 > > # start mysql > docker run -it --rm \ > --name mysqlterm \ > --link mysql \ > --rm mysql:5.7 sh \ > -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"' > ``` This is the config I was using ``` tenant: "my-tenant" namespace: "debezium" name: "debezium-mysql-source" topicName: "my-tenant/debezium/debezium-mysql-topic" archive: "connectors/pulsar-io-debezium-mysql-2.6.0-SNAPSHOT.nar" parallelism: 1 resources: cpu: 0.2 ram: 1073741824 configs: ## sourceTask task.class: "io.debezium.connector.mysql.MySqlConnectorTask" ## config for mysql, docker image: debezium/example-mysql:0.8 database.hostname: "db.hostname" database.port: "3306" database.user: "kafka_connect" database.password: "password" database.server.id: "89447" database.server.name: "myname" database.whitelist: "myschema" table.whitelist: "myschema.mytable" database.history: "org.apache.pulsar.io.debezium.PulsarDatabaseHistory" database.history.pulsar.topic: "my-tenant/debezium/history-topic" database.history.pulsar.service.url: "pulsar://pulsar-broker.dev-pulsar.svc.cluster.local:6650" ## KEY_CONVERTER_CLASS_CONFIG, VALUE_CONVERTER_CLASS_CONFIG key.converter: "org.apache.pulsar.kafka.shade.io.confluent.connect.avro.AvroConverter" value.converter: "org.apache.pulsar.kafka.shade.io.confluent.connect.avro.AvroConverter" schema.registry.url: "http://schema-registry.kafka.internal.data.mydomain.com:8081"; ## PULSAR_SERVICE_URL_CONFIG pulsar.service.url: "pulsar://pulsar-broker.dev-pulsar.svc.cluster.local:6650" ## OFFSET_STORAGE_TOPIC_CONFIG offset.storage.topic: "my-tenant/debezium/offset-topic" ## Extra settings # database.history.store.only.monitored.tables.ddl: "true" database.serverTimezone: "Asia/Hong_Kong" ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
