odbozhou commented on code in PR #302: URL: https://github.com/apache/rocketmq-connect/pull/302#discussion_r973883935
########## connectors/rocketmq-connect-doris/README.txt: ########## @@ -0,0 +1,173 @@ +# rocketmq-connect-doris + +## rocketmq-connect-doirs packaging + +``` +mvn clean package -Dmaven.test.skip=true +``` + +## rocketmq-connect-doris +Supporting for database languages: +| Language | Support | +|----------------|------------| +| DDL | NO | +| DML | YES | +| DCL | NO | +| TCL | NO | +DDL: +Currently, sink-connect-doris load data to Doris via Stream Load which does not support DDL, so DDL + is also not supported by sink-connect-doris. +DML: +Sink-connect-doris only support insert mode, and entries should have increment filed. +DCL: +Not support now. +TCL: +Transaction concept not different in Mysql and Doris, so TCL is not supported. + +## How to use +0. Doris cluster and Rocketmq Cluster should be prepared. +1. build rocketmq connect (https://github.com/apache/rocketmq-connect). +2. package rocketmq-connect-jdbc and rocketmq-connect-doris. +3. run rocketmq connect runtime. +4. create source-connect-mysql. +5. create sourcr-connect-doris. + +``` + +* **doris-sink-connector** Start + +``` +POST http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-sink-connector-name} +{ + "connector.class":"org.apache.rocketmq.connect.doris.connector.DorisSinkConnector", + "max-task":"1", + "table.whitelist":"sink_test.doris_test_sink", + "connect.topicname":"doris_test_sink", + "connect.topicnames":"doris_test_sink", + "host":"xx.xx.xx.xx", + "port":"xxxx", + "user":"xx", + "passwd":"****", + "database":"database", + "insert.mode":"INSERT", + "db.timezone":"UTC", + "table.types":"TABLE", + "source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter", + "key.converter":"org.apache.rocketmq.connect.runtime.converter.record.json.JsonConverter", + "value.converter":"org.apache.rocketmq.connect.runtime.converter.record.json.JsonConverter"}' +} +``` + +> **Note: +1.rocketmq-jdbc-connect depends on the startup of the `rocketmq-connect-runtime` project. You will need to put all the `jar` packages into the path configured by `pluginPaths` in the `runtime` project and re-execute the above startup request. This value is configured in the `connect.conf` file under the `runtime` project. +2.The data pulled by the source connector is written to the topic automatically created with the table name. If you need to write to a specific topic, you need to specify the "connect-topicname" parameter +3.The table name used by sink-connect-doris is specified by schema in message, which is the same with original mysql table name, so doris table name created should be the same with mysql table name. + +## rocketmq-connect-jdbc Stop + +``` +http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-doris-connector-name}/stop +``` + +* **jdbc-sink-connector 参数说明** + +| KEY | TYPE | Must be filled | Description | Example +|------------------------|----|---------|---------------|-------------------| +|host | String | YES |doris host | 192.168.0.1 | +|port | String | YES |doris http port | 8030 | +|user | String | YES |doris user name | root | +|passwd | String | YES |doris passwd | passwd | +|max-task | Integer | NO |task number | 2 | Review Comment: Refer to the above modification ########## connectors/rocketmq-connect-doris/README.txt: ########## @@ -0,0 +1,173 @@ +# rocketmq-connect-doris + +## rocketmq-connect-doirs packaging + +``` +mvn clean package -Dmaven.test.skip=true +``` + +## rocketmq-connect-doris +Supporting for database languages: +| Language | Support | +|----------------|------------| +| DDL | NO | +| DML | YES | +| DCL | NO | +| TCL | NO | +DDL: +Currently, sink-connect-doris load data to Doris via Stream Load which does not support DDL, so DDL + is also not supported by sink-connect-doris. +DML: +Sink-connect-doris only support insert mode, and entries should have increment filed. +DCL: +Not support now. +TCL: +Transaction concept not different in Mysql and Doris, so TCL is not supported. + +## How to use +0. Doris cluster and Rocketmq Cluster should be prepared. +1. build rocketmq connect (https://github.com/apache/rocketmq-connect). +2. package rocketmq-connect-jdbc and rocketmq-connect-doris. +3. run rocketmq connect runtime. +4. create source-connect-mysql. +5. create sourcr-connect-doris. + +``` + +* **doris-sink-connector** Start + +``` +POST http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-sink-connector-name} +{ + "connector.class":"org.apache.rocketmq.connect.doris.connector.DorisSinkConnector", + "max-task":"1", + "table.whitelist":"sink_test.doris_test_sink", + "connect.topicname":"doris_test_sink", + "connect.topicnames":"doris_test_sink", + "host":"xx.xx.xx.xx", + "port":"xxxx", + "user":"xx", + "passwd":"****", + "database":"database", + "insert.mode":"INSERT", + "db.timezone":"UTC", + "table.types":"TABLE", + "source-record-converter":"org.apache.rocketmq.connect.runtime.converter.JsonConverter", Review Comment: deprecated configuration ########## connectors/rocketmq-connect-doris/README.txt: ########## @@ -0,0 +1,173 @@ +# rocketmq-connect-doris + +## rocketmq-connect-doirs packaging + +``` +mvn clean package -Dmaven.test.skip=true +``` + +## rocketmq-connect-doris +Supporting for database languages: +| Language | Support | +|----------------|------------| +| DDL | NO | +| DML | YES | +| DCL | NO | +| TCL | NO | +DDL: +Currently, sink-connect-doris load data to Doris via Stream Load which does not support DDL, so DDL + is also not supported by sink-connect-doris. +DML: +Sink-connect-doris only support insert mode, and entries should have increment filed. +DCL: +Not support now. +TCL: +Transaction concept not different in Mysql and Doris, so TCL is not supported. + +## How to use +0. Doris cluster and Rocketmq Cluster should be prepared. +1. build rocketmq connect (https://github.com/apache/rocketmq-connect). +2. package rocketmq-connect-jdbc and rocketmq-connect-doris. +3. run rocketmq connect runtime. +4. create source-connect-mysql. +5. create sourcr-connect-doris. + +``` + +* **doris-sink-connector** Start + +``` +POST http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-sink-connector-name} +{ + "connector.class":"org.apache.rocketmq.connect.doris.connector.DorisSinkConnector", + "max-task":"1", Review Comment: Please use max.task ########## connectors/rocketmq-connect-doris/README.txt: ########## @@ -0,0 +1,173 @@ +# rocketmq-connect-doris + +## rocketmq-connect-doirs packaging + +``` +mvn clean package -Dmaven.test.skip=true +``` + +## rocketmq-connect-doris +Supporting for database languages: +| Language | Support | +|----------------|------------| +| DDL | NO | +| DML | YES | +| DCL | NO | +| TCL | NO | +DDL: +Currently, sink-connect-doris load data to Doris via Stream Load which does not support DDL, so DDL + is also not supported by sink-connect-doris. +DML: +Sink-connect-doris only support insert mode, and entries should have increment filed. +DCL: +Not support now. +TCL: +Transaction concept not different in Mysql and Doris, so TCL is not supported. + +## How to use +0. Doris cluster and Rocketmq Cluster should be prepared. +1. build rocketmq connect (https://github.com/apache/rocketmq-connect). +2. package rocketmq-connect-jdbc and rocketmq-connect-doris. +3. run rocketmq connect runtime. +4. create source-connect-mysql. +5. create sourcr-connect-doris. + +``` + +* **doris-sink-connector** Start + +``` +POST http://${runtime-ip}:${runtime-port}/connectors/${rocketmq-jdbc-sink-connector-name} +{ + "connector.class":"org.apache.rocketmq.connect.doris.connector.DorisSinkConnector", + "max-task":"1", + "table.whitelist":"sink_test.doris_test_sink", + "connect.topicname":"doris_test_sink", Review Comment: Just configure connect.topicnames, no need for connect.topicname -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
