This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 2a5aed46f7 add CloudCanal en doc (#22731)
2a5aed46f7 is described below
commit 2a5aed46f72259baf7fd2c5004c466822c09c992
Author: Meta Luo <[email protected]>
AuthorDate: Tue Aug 22 10:49:54 2023 +0800
add CloudCanal en doc (#22731)
---
docs/en/docs/ecosystem/cloudcanal.md | 79 +++++++++++++++++++++++++++++--
docs/images/cloudcanal/cloudcanal-1.png | Bin 0 -> 489343 bytes
docs/images/cloudcanal/cloudcanal-11.png | Bin 0 -> 488805 bytes
docs/images/cloudcanal/cloudcanal-12.png | Bin 0 -> 582329 bytes
docs/images/cloudcanal/cloudcanal-13.png | Bin 0 -> 465703 bytes
docs/images/cloudcanal/cloudcanal-14.png | Bin 0 -> 629850 bytes
docs/images/cloudcanal/cloudcanal-15.png | Bin 0 -> 705494 bytes
docs/images/cloudcanal/cloudcanal-16.png | Bin 0 -> 473013 bytes
docs/images/cloudcanal/cloudcanal-17.png | Bin 0 -> 85983 bytes
docs/images/cloudcanal/cloudcanal-2.png | Bin 0 -> 603103 bytes
docs/images/cloudcanal/cloudcanal-3.png | Bin 0 -> 463608 bytes
docs/zh-CN/docs/ecosystem/cloudcanal.md | 16 +++----
12 files changed, 84 insertions(+), 11 deletions(-)
diff --git a/docs/en/docs/ecosystem/cloudcanal.md
b/docs/en/docs/ecosystem/cloudcanal.md
index 3fb7518057..0444d69bc9 100644
--- a/docs/en/docs/ecosystem/cloudcanal.md
+++ b/docs/en/docs/ecosystem/cloudcanal.md
@@ -28,8 +28,81 @@ under the License.
## Introduction
-CloudCanal Community Edition is a free data migration and synchronization
platform published by [ClouGence Company](https://www.clougence.com) that
integrates structure migration, full data migration/check/correction, and
incremental real-time synchronization. Product contains complete
-Its productization capabilities help enterprises break data silos, complete
data integration and interoperability, and make better use of data.
+CloudCanal Community Edition is a free data migration and synchronization
platform published by [ClouGence Co., Ltd](https://www.cloudcanalx.com) that
integrates Schema Migration, Full Data Migration, verification, Correction, and
real-time Incremental Synchronization.
+CloudCanal help users build a modern data stack in a simple way.
-> There is no English version of this document, please switch to the Chinese
version.
+## Download
+
+[CloudCanal Download Link](https://www.cloudcanalx.com)
+
+## Function Description
+
+- It is highly recommended to utilize CloudCanal version 2.2.5.0 or higher for
efficient data import into Doris.
+- It is advisable to exercise control over the ingestion frequency when using
CloudCanal to import **incremental data** into Doris. The default import
frequency for writing data from CloudCanal to Doris can be adjusted using the
`realFlushPauseSec` parameter, which is set to 10 seconds by default.
+- In the current community edition with a maximum memory configuration of 2GB,
if DataJobs encounter OOM exceptions or significant GC pauses, it is
recommended to reduce the batch size to minimize memory usage. For full
DataTask, you can adjust the `fullBatchSize` and `fullRingBufferSize`
parameters. For incremental DataTask, the `increBatchSize` and
`increRingBufferSize` parameters can be adjusted accordingly.
+- Supported Source endpoints and features：
+
+ | Source Endpoints \ Feature | Schema Migration | Full Data | Incremental |
Verification |
+ | --- | --- | --- | --- | --- |
+ | Oracle | Yes | Yes | Yes | Yes |
+ | PostgreSQL | Yes | Yes | Yes | Yes |
+ | Greenplum | Yes | Yes | No | Yes |
+ | MySQL | Yes | Yes | Yes | Yes |
+
+## Instructions for Use
+
+CloudCanal offers a comprehensive productized capability where users can
seamlessly add DataSources and create DataJobs through a visual interface. This
enables automated schema migration, full data migration, and real-time
incremental synchronization. The following example demonstrates how to migrate
and synchronize data from a MySQL to the target Doris. Similar procedures can
be applied when synchronizing other source endpoints with Doris.
+
+### Prerequisites
+
+First, refer to the [CloudCanal Quick
Start](https://www.cloudcanalx.com/us/cc-doc/quick/quick_start) to complete the
installation and deployment of the CloudCanal Community Edition.
+
+### Add DataSource
+
+- Log in to the CloudCanal platform
+- Go to **DataSource Management** -> **Add DataSource**
+- Select **Doris** from the options for self-built databases
+
+
+
+> Tips:
+>
+> - Client Address: The address of the Doris server's MySQL client service
port. CloudCanal primarily uses this address to query metadata information of
the database tables.
+>
+> - HTTP Address: The HTTP address is mainly used to receive data import
requests from CloudCanal.
+
+### Create DataJob
+
+Once the DataSource has been added successfully, you can follow these steps to
create data migration and synchronization DataJob.
+
+- Go to **DataJob Management** -> **Create DataJob** in the CloudCanal
+- Select the source and target databases for the DataJob
+- Click Next Step
+
+
+
+- Choose **Incremental** and enable **Full Data**
+- Select DDL Sync
+- Click Next Step
+
+
+
+- Select the tables you want to subscribe to. Please note that **the tables
automatically created during structural migration follow the primary key model,
so tables without a primary key are not currently supported**
+- Click Next Step
+
+
+
+- Configure the column mapping
+- Click Next Step
+
+
+
+- Create DataJob
+
+
+
+- Check the status of DataJob. The DataJob will automatically go through the
stages of Schema Migration, Full Data, and Incremental after it has been created
+
+
+
diff --git a/docs/images/cloudcanal/cloudcanal-1.png
b/docs/images/cloudcanal/cloudcanal-1.png
new file mode 100644
index 0000000000..bb5a2bf7f7
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-1.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-11.png
b/docs/images/cloudcanal/cloudcanal-11.png
new file mode 100644
index 0000000000..8087737d46
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-11.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-12.png
b/docs/images/cloudcanal/cloudcanal-12.png
new file mode 100644
index 0000000000..e526bb4f5a
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-12.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-13.png
b/docs/images/cloudcanal/cloudcanal-13.png
new file mode 100644
index 0000000000..4220b9bf72
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-13.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-14.png
b/docs/images/cloudcanal/cloudcanal-14.png
new file mode 100644
index 0000000000..3774df4f9a
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-14.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-15.png
b/docs/images/cloudcanal/cloudcanal-15.png
new file mode 100644
index 0000000000..c8852fcaa3
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-15.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-16.png
b/docs/images/cloudcanal/cloudcanal-16.png
new file mode 100644
index 0000000000..708e4b00b5
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-16.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-17.png
b/docs/images/cloudcanal/cloudcanal-17.png
new file mode 100644
index 0000000000..d84706a17e
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-17.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-2.png
b/docs/images/cloudcanal/cloudcanal-2.png
new file mode 100644
index 0000000000..b3ee1b8c48
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-2.png differ
diff --git a/docs/images/cloudcanal/cloudcanal-3.png
b/docs/images/cloudcanal/cloudcanal-3.png
new file mode 100644
index 0000000000..7475d53d44
Binary files /dev/null and b/docs/images/cloudcanal/cloudcanal-3.png differ
diff --git a/docs/zh-CN/docs/ecosystem/cloudcanal.md
b/docs/zh-CN/docs/ecosystem/cloudcanal.md
index 279e00cb79..1e6eba3e7b 100644
--- a/docs/zh-CN/docs/ecosystem/cloudcanal.md
+++ b/docs/zh-CN/docs/ecosystem/cloudcanal.md
@@ -33,7 +33,7 @@ CloudCanal 社区版是一款由 [ClouGence 公司](https://www.clougence.com)
## 下载安装
-[CloudCanal 最新版下载地址](https://clougence.com)
+[CloudCanal 最新版下载地址](https://www.clougence.com/)
## 功能说明
@@ -55,7 +55,7 @@ CloudCanal 提供了完整的产品化能力，用户在可视化界面完成数
### 前置条件
-首先参考 [CloudCanal 安装部署参考文档](https://www.askcug.com/topic/75) 完成 CloudCanal
社区版的安装和部署。
+首先参考 [CloudCanal 快速开始](https://www.clougence.com/cc-doc/quick/quick_start) 完成
CloudCanal 社区版的安装和部署。
### 添加数据源
@@ -63,7 +63,7 @@ CloudCanal 提供了完整的产品化能力，用户在可视化界面完成数
- 数据源管理-> 新增数据源
- 选择自建数据库中 Doris
-
+
> Tips:
>
@@ -79,13 +79,13 @@ CloudCanal 提供了完整的产品化能力，用户在可视化界面完成数
- 选择 **源** 和 **目标** 数据库
- 点击 下一步
-
+
- 选择 **增量同步**，并且启用 **全量数据初始化**
-- 不勾选 DDL 同步(暂不支持)
+- 勾选 DDL 同步
- 点击下一步
-
+
- 选择订阅的表，**结构迁移自动创建的表为主键模型的表，因此暂不支持无主键表**
- 点击下一步
@@ -109,6 +109,6 @@ CloudCanal 提供了完整的产品化能力，用户在可视化界面完成数
更多关于 CloudCanal 同步 Doris 的资料，可以查看
-- [5 分钟搞定 MySQL/PostgreSQL/Oracle 到 Doris 数据迁移同步-CloudCanal
实战](https://www.clougence.com/cc-doc/blog/postgresql_doris_sync/)
+- [5 分钟搞定 PostgreSQL 到 Doris 数据迁移同步-CloudCanal
实战](https://www.clougence.com/cc-doc/blog/postgresql_doris_sync/)
-- [CloudCanal 官方社区](https://www.askcug.com/)
+- [CloudCanal 官网](https://www.clougence.com/)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
