This is an automated email from the ASF dual-hosted git repository.
junma pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new e1e3baf1eef [improve][doc] Add doc of reader config to
`pulsar-io-debezium` (#16891)
e1e3baf1eef is described below
commit e1e3baf1eef37567d84eb4a3947a9305c63324e7
Author: Rui Fu <[email protected]>
AuthorDate: Thu Sep 29 10:46:55 2022 +0800
[improve][doc] Add doc of reader config to `pulsar-io-debezium` (#16891)
* add reader config doc
* update to the versioned doc
* Update site2/docs/io-debezium-source.md
Co-authored-by: momo-jun <[email protected]>
* Update site2/docs/io-debezium-source.md
Co-authored-by: momo-jun <[email protected]>
* revert changes to 2.10.1 and 2.9.3
Co-authored-by: momo-jun <[email protected]>
---
site2/docs/io-debezium-source.md | 29 ++++++++++++++++++++++++++++-
1 file changed, 28 insertions(+), 1 deletion(-)
diff --git a/site2/docs/io-debezium-source.md b/site2/docs/io-debezium-source.md
index 0f7dd19b5d7..c4ce2c2f63e 100644
--- a/site2/docs/io-debezium-source.md
+++ b/site2/docs/io-debezium-source.md
@@ -26,7 +26,9 @@ The configuration of the Debezium source connector has the
following properties.
| `database.history.pulsar.topic` | true | null | The name of the database
history topic where the connector writes and recovers DDL statements. <br /><br
/>**Note: this topic is for internal use only and should not be used by
consumers.** |
| `database.history.pulsar.service.url` | true | null | Pulsar cluster service
URL for history topic. |
| `offset.storage.topic` | true | null | Record the last committed offsets
that the connector successfully completes. |
-| `json-with-envelope` | false | false | Present the message only consist of
payload.
+| `json-with-envelope` | false | false | Present the message only consist of
payload. |
+| `database.history.pulsar.reader.config` | false | null | The configs of the
reader for the database schema history topic, in the form of a JSON string with
key-value pairs. |
+| `offset.storage.reader.config` | false | null | The configs of the reader
for the kafka connector offsets topic, in the form of a JSON string with
key-value pairs. |
### Converter Options
@@ -50,7 +52,32 @@ Schema.AUTO_CONSUME(), KeyValueEncodingType.SEPARATED)`, and
the message consist
| `mongodb.password` | true | null | Password to be used when connecting to
MongoDB. This is required only when MongoDB is configured to use
authentication. |
| `mongodb.task.id` | true | null | The taskId of the MongoDB connector that
attempts to use a separate task for each replica set. |
+### Customize the Reader config for the metadata topics
+The Debezium Connector exposes `database.history.pulsar.reader.config` and
`offset.storage.reader.config` to configure the reader of database schema
history topic and the Kafka connector offsets topic. For example, it can be
used to configure the subscription name and other reader configurations. You
can find the available configurations at
[ReaderConfigurationData](https://github.com/apache/pulsar/blob/master/pulsar-client/src/main/java/org/apache/pulsar/client/impl/conf/ReaderConfigura
[...]
+
+For example, to configure the subscription name for both Readers, you can add
the following configuration:
+
+* JSON
+
+ ```json
+ {
+ "configs": {
+ "database.history.pulsar.reader.config":
"{\"subscriptionName\":\"history-reader\"}",
+ "offset.storage.reader.config":
"{\"subscriptionName\":\"offset-reader\"}",
+ }
+ }
+ ```
+
+* YAML
+
+ ```yaml
+
+ configs:
+ database.history.pulsar.reader.config:
"{\"subscriptionName\":\"history-reader\"}"
+ offset.storage.reader.config: "{\"subscriptionName\":\"offset-reader\"}"
+
+ ```
## Example of MySQL
