tuteng opened a new pull request #3268: Optimization of binlog to pulsar use canal URL: https://github.com/apache/pulsar/pull/3268 ### Motivation Optimization of mysql binlog to pulsar use canal ### Modifications Add MessageUtils for parse columns Support query of presto sql ### Result #### INSERT mysql: ``` MariaDB [aaa]> insert into users321(name, extra) values('xxxddxxxmm', 'ddd'); ``` python consumer: ``` {u'timestamp': u'2018-12-31 00:31:24', u'message': u'[{"data":null,"database":"","es":1546234283000,"id":58,"isDdl":false,"mysqlType":null,"old":null,"sql":"insert into users321(name, extra) values(\'xxxddxxxmm\', \'ddd\')","sqlType":null,"table":"","ts":1546234284669,"type":"QUERY"},{"data":[{"isKey":"1","isNull":"0","index":"0","mysqlType":"int(11)","columnName":"id","columnValue":"34","updated":"1"},{"isKey":"0","isNull":"0","index":"1","mysqlType":"varchar(50)","columnName":"name","columnValue":"xxxddxxxmm","updated":"1"},{"isKey":"0","isNull":"0","index":"2","mysqlType":"varchar(50)","columnName":"extra","columnValue":"ddd","updated":"1"}],"database":"aaa","es":1546234283000,"id":58,"isDdl":false,"mysqlType":null,"old":null,"sql":"","sqlType":null,"table":"users321","ts":1546234284670,"type":"INSERT"}]', u'id': 58} ``` presto sql: ``` presto> select id, timestamp, message from pulsar."public/default".my_topic_test where id=58; id | timestamp | ----+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 58 | 2018-12-31 00:31:24 | [{"data":null,"database":"","es":1546234283000,"id":58,"isDdl":false,"mysqlType":null,"old":null,"sql":"insert into users321(name, extra) values('xxxddxxxmm', 'ddd')","sqlType" ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
