yunqingmoswu commented on code in PR #418: URL: https://github.com/apache/incubator-inlong-website/pull/418#discussion_r899712806
########## i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md: ########## @@ -1,4 +1,192 @@ --- title: MongoDB-CDC sidebar_position: 7 ---- \ No newline at end of file +--- + +## MongoDB-CDC Extract 节点 + +MongoDB CDC 连接器允许从 MongoDB 读取快照数据和增量数据。本文档介绍如何设置 MongoDB CDC 连接器以对 MongoDB 运行 SQL 查询。 + +## 支持的版本 +| Extract 节点 | 版本 | +| ------------------------------- | --------------------------------------------- | +| [mongodb-cdc](./mongodb-cdc.md) | [MongoDB](https://www.mongodb.com/): `>=` 3.6 | Review Comment: [mongodb-cdc] -> [MongoDB-CDC] ########## i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md: ########## @@ -1,4 +1,192 @@ --- title: MongoDB-CDC sidebar_position: 7 ---- \ No newline at end of file +--- + +## MongoDB-CDC Extract 节点 + +MongoDB CDC 连接器允许从 MongoDB 读取快照数据和增量数据。本文档介绍如何设置 MongoDB CDC 连接器以对 MongoDB 运行 SQL 查询。 + +## 支持的版本 +| Extract 节点 | 版本 | +| ------------------------------- | --------------------------------------------- | +| [mongodb-cdc](./mongodb-cdc.md) | [MongoDB](https://www.mongodb.com/): `>=` 3.6 | + +## 依赖项 + +I.为了设置 MongoDB CDC 连接器，下表提供了使用构建自动化工具（例如 Maven 或 SBT）的依赖关系信息 + +### Maven依赖 + +```xml +<dependency> + <groupId>org.apache.inlong</groupId> + <artifactId>sort-connector-mongodb-cdc</artifactId> + <!-- 选择你使用的 inlong 的版本 --> + <version>inlong_version</version> +</dependency> +``` + +## 设置 MongoDB + +### 可用性 + +- MongoDB版本 + + MongoDB 版本 \>= 3.6 + 我们使用 [更改流](https://docs.mongodb.com/manual/changeStreams/)功能（3.6 版中的新功能）来捕获更改数据。 + +- 集群部署 + + 需要 [副本集](https://docs.mongodb.com/manual/replication/)或 [分片集群](https://docs.mongodb.com/manual/sharding/)。 + +- 存储引擎 + + 需要 [WiredTiger](https://docs.mongodb.com/manual/core/wiredtiger/#std-label-storage-wiredtiger)存储引擎。 + +- [副本集协议版本](https://docs.mongodb.com/manual/reference/replica-configuration/#mongodb-rsconf-rsconf.protocolVersion) + + 需要副本集协议版本 1 [(pv1)](https://docs.mongodb.com/manual/reference/replica-configuration/#mongodb-rsconf-rsconf.protocolVersion)。 + 从版本 4.0 开始，MongoDB 仅支持 pv1。pv1 是使用 MongoDB 3.2 或更高版本创建的所有新副本集的默认值。 + +- 特权 + + `changeStream` MongoDB Kafka 连接器 `read` 需要权限。 + + 您可以使用以下示例进行简单授权。 + 更详细的授权请参考 [MongoDB 数据库用户角色](https://docs.mongodb.com/manual/reference/built-in-roles/#database-user-roles)。 + + ```shell + use admin; + db.createUser({ + user: "flinkuser", + pwd: "flinkpw", + roles: [ + { role: "read", db: "admin" }, // read role includes changeStream privilege + { role: "readAnyDatabase", db: "admin" } // for snapshot reading + ] + }); + ``` + +## 如何创建 MongoDB Extract 节点 + +### SQL API 用法 + +这个例子展示了如何使用 `Flink SQL` 创建一个 MongoDB Extract 节点: + +```sql +-- Set checkpoint every 3000 milliseconds +Flink SQL> SET 'execution.checkpointing.interval' = '3s'; + +-- Create a MySQL table 'mongodb_extract_node' in Flink SQL +Flink SQL> CREATE TABLE mongodb_extract_node ( + _id STRING, // must be declared + name STRING, + weight DECIMAL(10,3), + tags ARRAY<STRING>, -- array + price ROW<amount DECIMAL(10,2), currency STRING>, -- embedded document + suppliers ARRAY<ROW<name STRING, address STRING>>, -- embedded documents + PRIMARY KEY(_id) NOT ENFORCED +) WITH ( + 'connector' = 'mongodb-cdc', + 'hosts' = 'localhost:27017,localhost:27018,localhost:27019', + 'username' = 'flinkuser', + 'password' = 'flinkpw', + 'database' = 'inventory', + 'collection' = 'mongodb_extract_node' +); + +-- Read snapshot and binlogs from mongodb_extract_node +Flink SQL> SELECT * FROM mongodb_extract_node; +``` + +**注意** + +MongoDB 的更改事件记录在消息之前没有更新。所以，我们只能将其转换为 Flink 的 UPSERT 变更日志流。UPSERT 流需要唯一键，因此我们必须声明 `_id` 为主键。我们不能将其他列声明为主键，因为删除操作不包含除 `_id` 和 `sharding key` 之外的键和值。 + +### InLong Dashboard 用法 + +TODO: 未来会支持 + +### InLong Manager 用法 + +TODO: 未来会支持 + +## MongoDB Extract 节点选项 Review Comment: 选项 -> 参数 ########## i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md: ########## @@ -1,4 +1,192 @@ --- title: MongoDB-CDC sidebar_position: 7 ---- \ No newline at end of file +--- + +## MongoDB-CDC Extract节点 + +MongoDB CDC 连接器允许从 MongoDB 读取快照数据和增量数据。本文档介绍如何设置 MongoDB CDC 连接器以对 MongoDB 运行 SQL 查询。 + +## 支持的版本 +| Extract节点 | 版本 | +| ------------------------------- | -------------------------------------------- | +| [mongodb-cdc](./mongodb-cdc.md) | [MongoDB](https://www.mongodb.com/): \>= 3.6 | + +## 依赖项 + +I.为了设置 MongoDB CDC 连接器，下表提供了使用构建自动化工具（例如 Maven 或 SBT）的依赖关系信息 + +### Maven依赖 + +```xml +<dependency> + <groupId>com.ververica</groupId> + <artifactId>flink-connector-mongodb-cdc</artifactId> + <!-- the dependency is available only for stable releases. --> + <version>2.1.1</version> +</dependency> +``` + +## 设置 MongoDB + +### 可用性 + +- MongoDB版本 + + MongoDB 版本 \>= 3.6 Review Comment: \> -> `> ` ########## docs/data_node/extract_node/mongodb-cdc.md: ########## @@ -1,4 +1,197 @@ --- title: MongoDB-CDC sidebar_position: 7 ---- \ No newline at end of file +--- + +## MongoDB-CDC Extract Node + +The MongoDB CDC connector allows for reading snapshot data and incremental data from MongoDB. This document describes how to setup the MongoDB CDC connector to run SQL queries against MongoDB. + +## Supported Version +| Extract Node | Version | +| ------------------------------- | -------------------------------------------- | +| [mongodb-cdc](./mongodb-cdc.md) | [MongoDB](https://www.mongodb.com/): `>=` 3.6 | + +## Dependencies + +In order to setup the MongoDB CDC connector, the following table provides dependency information for both projects using a build automation tool (such as Maven or SBT) and SQL Client with SQL JAR bundles. + +### Maven dependency + +```xml +<dependency> + <groupId>org.apache.inlong</groupId> + <artifactId>sort-connector-mongodb-cdc</artifactId> + <!-- select inlong version --> + <version>inlong_version</version> +</dependency> +``` + +## Setup MongoDB + +### Availability + +- MongoDB version + + MongoDB version \>= 3.6 + We use [change streams](https://docs.mongodb.com/manual/changeStreams/) feature (new in version 3.6) to capture change data. + +- Cluster Deployment + + [replica sets](https://docs.mongodb.com/manual/replication/) or [sharded clusters](https://docs.mongodb.com/manual/sharding/) is required. + +- Storage Engine + + [WiredTiger](https://docs.mongodb.com/manual/core/wiredtiger/#std-label-storage-wiredtiger) storage engine is required. + +- [Replica set protocol version](https://docs.mongodb.com/manual/reference/replica-configuration/#mongodb-rsconf-rsconf.protocolVersion) + + Replica set protocol version 1 [(pv1)](https://docs.mongodb.com/manual/reference/replica-configuration/#mongodb-rsconf-rsconf.protocolVersion) is required. + Starting in version 4.0, MongoDB only supports pv1. pv1 is the default for all new replica sets created with MongoDB 3.2 or later. + +- Privileges + + `changeStream` and `read` privileges are required by MongoDB Kafka Connector. + + You can use the following example for simple authorization. + For more detailed authorization, please refer to [MongoDB Database User Roles](https://docs.mongodb.com/manual/reference/built-in-roles/#database-user-roles). + + ```shell + use admin; + db.createUser({ + user: "flinkuser", + pwd: "flinkpw", + roles: [ + { role: "read", db: "admin" }, // read role includes changeStream privilege + { role: "readAnyDatabase", db: "admin" } // for snapshot reading + ] + }); + ``` + +## How to create a MongoDB Extract Node + +### Usage for SQL API + +The example below shows how to create an MongoDB Extract Node with `Flink SQL` : + +```sql +-- Set checkpoint every 3000 milliseconds +Flink SQL> SET 'execution.checkpointing.interval' = '3s'; + +-- Create a MySQL table 'mongodb_extract_node' in Flink SQL +Flink SQL> CREATE TABLE mongodb_extract_node ( + _id STRING, // must be declared + name STRING, + weight DECIMAL(10,3), + tags ARRAY<STRING>, -- array + price ROW<amount DECIMAL(10,2), currency STRING>, -- embedded document + suppliers ARRAY<ROW<name STRING, address STRING>>, -- embedded documents + PRIMARY KEY(_id) NOT ENFORCED +) WITH ( + 'connector' = 'mongodb-cdc', + 'hosts' = 'localhost:27017,localhost:27018,localhost:27019', + 'username' = 'flinkuser', + 'password' = 'flinkpw', + 'database' = 'inventory', + 'collection' = 'mongodb_extract_node' +); + +-- Read snapshot and binlogs from mongodb_extract_node +Flink SQL> SELECT * FROM mongodb_extract_node; +``` + +**Note that** Review Comment: Note that -> Note ########## docs/data_node/extract_node/mongodb-cdc.md: ########## @@ -1,4 +1,197 @@ --- title: MongoDB-CDC sidebar_position: 7 ---- \ No newline at end of file +--- + +## MongoDB-CDC Extract Node + +The MongoDB CDC connector allows for reading snapshot data and incremental data from MongoDB. This document describes how to setup the MongoDB CDC connector to run SQL queries against MongoDB. + +## Supported Version +| Extract Node | Version | +| ------------------------------- | -------------------------------------------- | +| [mongodb-cdc](./mongodb-cdc.md) | [MongoDB](https://www.mongodb.com/): `>=` 3.6 | Review Comment: `>=` -> `>=` ########## i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/mongodb-cdc.md: ########## @@ -1,4 +1,192 @@ --- title: MongoDB-CDC sidebar_position: 7 ---- \ No newline at end of file +--- + +## MongoDB-CDC Extract 节点 + +MongoDB CDC 连接器允许从 MongoDB 读取快照数据和增量数据。本文档介绍如何设置 MongoDB CDC 连接器以对 MongoDB 运行 SQL 查询。 + +## 支持的版本 +| Extract 节点 | 版本 | +| ------------------------------- | --------------------------------------------- | +| [mongodb-cdc](./mongodb-cdc.md) | [MongoDB](https://www.mongodb.com/): `>=` 3.6 | + +## 依赖项 + +I.为了设置 MongoDB CDC 连接器，下表提供了使用构建自动化工具（例如 Maven 或 SBT）的依赖关系信息 + +### Maven依赖 + +```xml +<dependency> + <groupId>org.apache.inlong</groupId> + <artifactId>sort-connector-mongodb-cdc</artifactId> + <!-- 选择你使用的 inlong 的版本 --> + <version>inlong_version</version> +</dependency> +``` + +## 设置 MongoDB + +### 可用性 + +- MongoDB版本 Review Comment: MongoDB后面加个空格 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
