[GitHub] [pulsar] tuteng commented on a change in pull request #5197: [Doc] Update *Debezium Connector Guide*
tuteng commented on a change in pull request #5197: [Doc] Update *Debezium Connector Guide* URL: https://github.com/apache/pulsar/pull/5197#discussion_r325004903 ## File path: site2/docs/io-cdc-debezium.md ## @@ -1,227 +1,281 @@ --- id: io-cdc-debezium -title: CDC Debezium Connector -sidebar_label: CDC Debezium Connector +title: Debezium Connector +sidebar_label: Debezium Connector --- -### Source Configuration Options +The Debezium source connector pulls messages from MySQL or PostgreSQL to Pulsar topics. -The Configuration is mostly related to Debezium task config, besides this we should provides the service URL of Pulsar cluster, and topic names that used to store offset and history. +This guide explains how to congifure and use Debezium source connector. + +## Configuration + +The configuration of Debezium source connector has the following parameters. | Name | Required | Default | Description | |--|--|-|-| -| `task.class` | `true` | `null` | A source task class that implemented in Debezium. | -| `database.hostname` | `true` | `null` | The address of the Database server. | -| `database.port` | `true` | `null` | The port number of the Database server.. | -| `database.user` | `true` | `null` | The name of the Database user that has the required privileges. | -| `database.password` | `true` | `null` | The password for the Database user that has the required privileges. | -| `database.server.id` | `true` | `null` | The connector’s identifier that must be unique within the Database cluster and similar to Database’s server-id configuration property. | -| `database.server.name` | `true` | `null` | The logical name of the Database server/cluster, which forms a namespace and is used in all the names of the Kafka topics to which the connector writes, the Kafka Connect schema names, and the namespaces of the corresponding Avro schema when the Avro Connector is used. | -| `database.whitelist` | `false` | `null` | A list of all databases hosted by this server that this connector will monitor. This is optional, and there are other properties for listing the databases and tables to include or exclude from monitoring. | -| `key.converter` | `true` | `null` | The converter provided by Kafka Connect to convert record key. | -| `value.converter` | `true` | `null` | The converter provided by Kafka Connect to convert record value. | -| `database.history` | `true` | `null` | The name of the database history class name. | -| `database.history.pulsar.topic` | `true` | `null` | The name of the database history topic where the connector will write and recover DDL statements. This topic is for internal use only and should not be used by consumers. | -| `database.history.pulsar.service.url` | `true` | `null` | Pulsar cluster service url for history topic. | -| `pulsar.service.url` | `true` | `null` | Pulsar cluster service url. | -| `offset.storage.topic` | `true` | `null` | Record the last committed offsets that the connector successfully completed. | +| `task.class` | true | null | A source task class that implemented in Debezium. | +| `database.hostname` | true | null | The address of a database server. | +| `database.port` | true | null | The port number of a database server.| +| `database.user` | true | null | The name of a database user that has the required privileges. | +| `database.password` | true | null | The password for a database user that has the required privileges. | +| `database.server.id` | true | null | The connector’s identifier that must be unique within a database cluster and similar to the database’s server-id configuration property. | +| `database.server.name` | true | null | The logical name of a database server/cluster, which forms a namespace and it is used in all the names of Kafka topics to which the connector writes, the Kafka Connect schema names, and the namespaces of the corresponding Avro schema when the Avro Connector is used. | +| `database.whitelist` | false | null | A list of all databases hosted by this server which is monitored by the connector. This is optional, and there are other properties for listing databases and tables to include or exclude from monitoring. | +| `key.converter` | true | null | The converter provided by Kafka Connect to convert record key. | +| `value.converter` | true | null | The converter provided by Kafka Connect to convert record value. | +| `database.history` | true | null | The name of the database history class. | +| `database.history.pulsar.topic` | true | null | The name of the database history topic where the connector writes and recovers DDL statements. **Note: this topic is for internal use only and should not be used by consumers.** | +| `database.history.pulsar.service.url` | true | null | Pulsar cluster service URL for history topic. | +| `pulsar.service.url` | true | null | Pulsar cluster service URL. | +| `offset.storage.topic` | true |
[GitHub] [pulsar] tuteng commented on a change in pull request #5197: [Doc] Update *Debezium Connector Guide*
tuteng commented on a change in pull request #5197: [Doc] Update *Debezium Connector Guide* URL: https://github.com/apache/pulsar/pull/5197#discussion_r325004765 ## File path: site2/docs/io-cdc-debezium.md ## @@ -1,227 +1,281 @@ --- id: io-cdc-debezium -title: CDC Debezium Connector -sidebar_label: CDC Debezium Connector +title: Debezium Connector +sidebar_label: Debezium Connector --- -### Source Configuration Options +The Debezium source connector pulls messages from MySQL or PostgreSQL to Pulsar topics. -The Configuration is mostly related to Debezium task config, besides this we should provides the service URL of Pulsar cluster, and topic names that used to store offset and history. +This guide explains how to congifure and use Debezium source connector. + +## Configuration + +The configuration of Debezium source connector has the following parameters. | Name | Required | Default | Description | |--|--|-|-| -| `task.class` | `true` | `null` | A source task class that implemented in Debezium. | -| `database.hostname` | `true` | `null` | The address of the Database server. | -| `database.port` | `true` | `null` | The port number of the Database server.. | -| `database.user` | `true` | `null` | The name of the Database user that has the required privileges. | -| `database.password` | `true` | `null` | The password for the Database user that has the required privileges. | -| `database.server.id` | `true` | `null` | The connector’s identifier that must be unique within the Database cluster and similar to Database’s server-id configuration property. | -| `database.server.name` | `true` | `null` | The logical name of the Database server/cluster, which forms a namespace and is used in all the names of the Kafka topics to which the connector writes, the Kafka Connect schema names, and the namespaces of the corresponding Avro schema when the Avro Connector is used. | -| `database.whitelist` | `false` | `null` | A list of all databases hosted by this server that this connector will monitor. This is optional, and there are other properties for listing the databases and tables to include or exclude from monitoring. | -| `key.converter` | `true` | `null` | The converter provided by Kafka Connect to convert record key. | -| `value.converter` | `true` | `null` | The converter provided by Kafka Connect to convert record value. | -| `database.history` | `true` | `null` | The name of the database history class name. | -| `database.history.pulsar.topic` | `true` | `null` | The name of the database history topic where the connector will write and recover DDL statements. This topic is for internal use only and should not be used by consumers. | -| `database.history.pulsar.service.url` | `true` | `null` | Pulsar cluster service url for history topic. | -| `pulsar.service.url` | `true` | `null` | Pulsar cluster service url. | -| `offset.storage.topic` | `true` | `null` | Record the last committed offsets that the connector successfully completed. | +| `task.class` | true | null | A source task class that implemented in Debezium. | +| `database.hostname` | true | null | The address of a database server. | +| `database.port` | true | null | The port number of a database server.| +| `database.user` | true | null | The name of a database user that has the required privileges. | +| `database.password` | true | null | The password for a database user that has the required privileges. | +| `database.server.id` | true | null | The connector’s identifier that must be unique within a database cluster and similar to the database’s server-id configuration property. | +| `database.server.name` | true | null | The logical name of a database server/cluster, which forms a namespace and it is used in all the names of Kafka topics to which the connector writes, the Kafka Connect schema names, and the namespaces of the corresponding Avro schema when the Avro Connector is used. | +| `database.whitelist` | false | null | A list of all databases hosted by this server which is monitored by the connector. This is optional, and there are other properties for listing databases and tables to include or exclude from monitoring. | +| `key.converter` | true | null | The converter provided by Kafka Connect to convert record key. | +| `value.converter` | true | null | The converter provided by Kafka Connect to convert record value. | +| `database.history` | true | null | The name of the database history class. | +| `database.history.pulsar.topic` | true | null | The name of the database history topic where the connector writes and recovers DDL statements. **Note: this topic is for internal use only and should not be used by consumers.** | +| `database.history.pulsar.service.url` | true | null | Pulsar cluster service URL for history topic. | +| `pulsar.service.url` | true | null | Pulsar cluster service URL. | +| `offset.storage.topic` | true |
