tuteng commented on a change in pull request #5715: [Issues 5692]Add document
for mongodb connector
URL: https://github.com/apache/pulsar/pull/5715#discussion_r349391287
##########
File path: site2/docs/io-cdc-debezium.md
##########
@@ -305,3 +312,114 @@ This example shows how to change the data of a
PostgreSQL table using the Pulsar
----- got message -----
{"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":107}}�{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":false,"field":"db"},{"type":"int64","optional":true,"field":"ts_usec"},{"type":"int64","optional":true,"field":"txId"},{"type":"int64","optional":true,"field":"lsn"},{"type":"string","optional":true,"field":"schema"},{"type":"string","optional":true,"field":"table"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"boolean","optional":true,"field":"last_snapshot_record"}],"optional":false,"name":"io.debezium.connector.postgresql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":{"id":107,"name":"rocks","description":"box
of assorted
rocks","weight":5.3},"after":{"id":107,"name":"1111111111","description":"box
of assorted
rocks","weight":5.3},"source":{"version":"0.9.2.Final","connector":"postgresql","name":"dbserver1","db":"postgres","ts_usec":1559208957661080,"txId":577,"lsn":23862872,"schema":"inventory","table":"products","snapshot":false,"last_snapshot_record":null},"op":"u","ts_ms":1559208957692}}
```
+## Example of MongoDB
+
+You need to create a configuration file before using the Pulsar Debezium
connector.
+
+* JSON
+
+ ```json
+ {
+ "mongodb.hosts": "rs0/mongodb:27017",
+ "mongodb.name": "dbserver1",
+ "mongodb.user": "debezium",
+ "mongodb.password": "dbz",
+ "mongodb.task.id": "1",
+ "schema.whitelist": "inventory",
+ "pulsar.service.url": "pulsar://127.0.0.1:6650"
+ }
+ ```
+
+* YAML
+
+ You can create a `debezium-postgres-source-config.yaml` file and copy the
[contents](https://github.com/apache/pulsar/blob/master/pulsar-io/debezium/postgres/src/main/resources/debezium-postgres-source-config.yaml)
below to the `debezium-postgres-source-config.yaml` file.
+
+ ```yaml
+ tenant: "public"
+ namespace: "default"
+ name: "debezium-mongodb-source"
+ topicName: "debezium-mongodb-topic"
+ archive: "connectors/pulsar-io-debezium-mongodb-{{pulsar:version}}.nar"
+ parallelism: 1
+
+ configs:
+
+ ## config for pg, docker image: debezium/example-postgress:0.8
+ "mongodb.hosts": "rs0/mongodb:27017",
+ "mongodb.name": "dbserver1",
+ "mongodb.user": "debezium",
+ "mongodb.password": "dbz",
+ "mongodb.task.id": "1",
+ "schema.whitelist": "inventory",
+
+ ## PULSAR_SERVICE_URL_CONFIG
+ pulsar.service.url: "pulsar://127.0.0.1:6650"
+ ```
+
+### Usage
+
+This example shows how to change the data of a MongoDB table using the Pulsar
Debezium connector.
+
+
+1. Start a PostgreSQL server with a database from which Debezium can capture
changes.
+
+ ```bash
+ $ docker pull debezium/example-mongodb:0.10
+ $ docker run -d -it --rm --name pulsar-mongodb -e MONGODB_USER=mongodb -e
MONGODB_PASSWORD=mongodb -p 27017:27017 debezium/example-mongodb:0.10
+ ```
+
+2. Start a Pulsar service locally in standalone mode.
+
+ ```bash
+ $ bin/pulsar standalone
+ ```
+
+3. Start the Pulsar Debezium connector in local run mode using one of the
following methods.
+
+ * Use the **JSON** configuration file as shown previously.
+
+ Make sure the nar file is available at
`connectors/pulsar-io-mongodb-postgres-{{pulsar:version}}.nar`.
+
+ ```bash
+ $ bin/pulsar-admin source localrun \
+ --archive
connectors/pulsar-io-debezium-postgres-{{pulsar:version}}.nar \
+ --name debezium-mongodb-source \
+ --destination-topic-name debezium-mongodb-topic \
+ --tenant public \
+ --namespace default \
+ --source-config '{"mongodb.hosts": "rs0/mongodb:27017","mongodb.name":
"dbserver1","mongodb.user": "debezium","mongodb.password":
"dbz","mongodb.task.id": "1","schema.whitelist":
"inventory","pulsar.service.url": "pulsar://127.0.0.1:6650"}'
+ ```
+
+ * Use the **YAML** configuration file as shown previously.
+
+ ```bash
+ $ bin/pulsar-admin source localrun \
+ --source-config-file debezium-mongodb-source-config.yaml
+ ```
+
+4. Subscribe the topic _sub-products_ for the _inventory.products_ table.
+
+ ```
+ $ bin/pulsar-client consume -s "sub-products"
public/default/dbserver1.inventory.products -n 0
Review comment:
To verify whether the data is synchronized to the pulsar, you can test it by
adding, deleting and updating the data in MongoDB.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
