This is an automated email from the ASF dual-hosted git repository.
vernedeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 07f12c3e7fd [INLONG-1050][Doc] Add example of pulsar to clickhouse
(#1051)
07f12c3e7fd is described below
commit 07f12c3e7fd93b2a62b3c2270e66a84c22a6d72d
Author: vernedeng <[email protected]>
AuthorDate: Mon Oct 14 13:06:52 2024 +0800
[INLONG-1050][Doc] Add example of pulsar to clickhouse (#1051)
Co-authored-by: vernedeng <[email protected]>
---
.../data_sync/img/mysql_clickhouse/cluster_tag.png | Bin 28397 -> 0 bytes
.../data_sync/img/mysql_clickhouse/pulsar.png | Bin 26969 -> 0 bytes
.../img/mysql_iceberg/create_cluster_tag.png | Bin 28397 -> 0 bytes
.../img/mysql_iceberg/create_pulsar_cluster.png | Bin 26969 -> 0 bytes
.../img/mysql_starrocks/create_cluster_tag.png | Bin 28397 -> 0 bytes
.../img/mysql_starrocks/create_pulsar_cluster.png | Bin 26969 -> 0 bytes
.../data_sync/img/pulsar_clickhouse/approve.png | Bin 0 -> 23656 bytes
.../img/pulsar_clickhouse/clickhouse_datanode.png | Bin 0 -> 20194 bytes
.../img/pulsar_clickhouse/group_stream.png | Bin 0 -> 22784 bytes
.../data_sync/img/pulsar_clickhouse/sink.png | Bin 0 -> 35250 bytes
.../data_sync/img/pulsar_clickhouse/sink_data.png | Bin 0 -> 16167 bytes
.../img/pulsar_clickhouse/sink_fields.png | Bin 0 -> 22308 bytes
.../data_sync/img/pulsar_clickhouse/source.png | Bin 0 -> 51486 bytes
.../data_sync/mysql_clickhouse_example.md | 20 ------
.../quick_start/data_sync/mysql_iceberg_example.md | 18 -----
.../data_sync/mysql_starrocks_example.md | 19 -----
...use_example.md => pulsar_clickhouse_example.md} | 77 ++++++++-------------
.../data_sync/img/mysql_clickhouse/cluster_tag.png | Bin 14019 -> 0 bytes
.../data_sync/img/mysql_clickhouse/pulsar.png | Bin 28881 -> 0 bytes
.../img/mysql_iceberg/create_cluster_tag.png | Bin 14019 -> 0 bytes
.../img/mysql_iceberg/create_pulsar_cluster.png | Bin 28881 -> 0 bytes
.../img/mysql_starrocks/create_cluster_tag.png | Bin 14019 -> 0 bytes
.../img/mysql_starrocks/create_pulsar_cluster.png | Bin 28881 -> 0 bytes
.../data_sync/img/pulsar_clickhouse/approve.png | Bin 0 -> 24000 bytes
.../img/pulsar_clickhouse/clickhouse_datanode.png | Bin 0 -> 19768 bytes
.../img/pulsar_clickhouse/group_stream.png | Bin 0 -> 23988 bytes
.../data_sync/img/pulsar_clickhouse/sink.png | Bin 0 -> 21930 bytes
.../data_sync/img/pulsar_clickhouse/sink_data.png | Bin 0 -> 16167 bytes
.../img/pulsar_clickhouse/sink_fields.png | Bin 0 -> 21498 bytes
.../data_sync/img/pulsar_clickhouse/source.png | Bin 0 -> 50523 bytes
.../data_sync/mysql_clickhouse_example.md | 12 ----
.../quick_start/data_sync/mysql_iceberg_example.md | 16 -----
.../data_sync/mysql_starrocks_example.md | 16 -----
...use_example.md => pulsar_clickhouse_example.md} | 74 ++++++++------------
34 files changed, 57 insertions(+), 195 deletions(-)
diff --git a/docs/quick_start/data_sync/img/mysql_clickhouse/cluster_tag.png
b/docs/quick_start/data_sync/img/mysql_clickhouse/cluster_tag.png
deleted file mode 100644
index bc1fc8a16d3..00000000000
Binary files a/docs/quick_start/data_sync/img/mysql_clickhouse/cluster_tag.png
and /dev/null differ
diff --git a/docs/quick_start/data_sync/img/mysql_clickhouse/pulsar.png
b/docs/quick_start/data_sync/img/mysql_clickhouse/pulsar.png
deleted file mode 100644
index eb725498e7c..00000000000
Binary files a/docs/quick_start/data_sync/img/mysql_clickhouse/pulsar.png and
/dev/null differ
diff --git
a/docs/quick_start/data_sync/img/mysql_iceberg/create_cluster_tag.png
b/docs/quick_start/data_sync/img/mysql_iceberg/create_cluster_tag.png
deleted file mode 100644
index bc1fc8a16d3..00000000000
Binary files
a/docs/quick_start/data_sync/img/mysql_iceberg/create_cluster_tag.png and
/dev/null differ
diff --git
a/docs/quick_start/data_sync/img/mysql_iceberg/create_pulsar_cluster.png
b/docs/quick_start/data_sync/img/mysql_iceberg/create_pulsar_cluster.png
deleted file mode 100644
index eb725498e7c..00000000000
Binary files
a/docs/quick_start/data_sync/img/mysql_iceberg/create_pulsar_cluster.png and
/dev/null differ
diff --git
a/docs/quick_start/data_sync/img/mysql_starrocks/create_cluster_tag.png
b/docs/quick_start/data_sync/img/mysql_starrocks/create_cluster_tag.png
deleted file mode 100644
index bc1fc8a16d3..00000000000
Binary files
a/docs/quick_start/data_sync/img/mysql_starrocks/create_cluster_tag.png and
/dev/null differ
diff --git
a/docs/quick_start/data_sync/img/mysql_starrocks/create_pulsar_cluster.png
b/docs/quick_start/data_sync/img/mysql_starrocks/create_pulsar_cluster.png
deleted file mode 100644
index eb725498e7c..00000000000
Binary files
a/docs/quick_start/data_sync/img/mysql_starrocks/create_pulsar_cluster.png and
/dev/null differ
diff --git a/docs/quick_start/data_sync/img/pulsar_clickhouse/approve.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/approve.png
new file mode 100644
index 00000000000..c4b06972252
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/approve.png differ
diff --git
a/docs/quick_start/data_sync/img/pulsar_clickhouse/clickhouse_datanode.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/clickhouse_datanode.png
new file mode 100644
index 00000000000..3047553494b
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/clickhouse_datanode.png
differ
diff --git a/docs/quick_start/data_sync/img/pulsar_clickhouse/group_stream.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/group_stream.png
new file mode 100644
index 00000000000..467c55a2445
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/group_stream.png differ
diff --git a/docs/quick_start/data_sync/img/pulsar_clickhouse/sink.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/sink.png
new file mode 100644
index 00000000000..468e72b3527
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/sink.png differ
diff --git a/docs/quick_start/data_sync/img/pulsar_clickhouse/sink_data.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/sink_data.png
new file mode 100644
index 00000000000..a5d2db1a315
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/sink_data.png differ
diff --git a/docs/quick_start/data_sync/img/pulsar_clickhouse/sink_fields.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/sink_fields.png
new file mode 100644
index 00000000000..c80e5e27727
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/sink_fields.png differ
diff --git a/docs/quick_start/data_sync/img/pulsar_clickhouse/source.png
b/docs/quick_start/data_sync/img/pulsar_clickhouse/source.png
new file mode 100644
index 00000000000..c483d245845
Binary files /dev/null and
b/docs/quick_start/data_sync/img/pulsar_clickhouse/source.png differ
diff --git a/docs/quick_start/data_sync/mysql_clickhouse_example.md
b/docs/quick_start/data_sync/mysql_clickhouse_example.md
index 04df530de32..903cabfe62e 100644
--- a/docs/quick_start/data_sync/mysql_clickhouse_example.md
+++ b/docs/quick_start/data_sync/mysql_clickhouse_example.md
@@ -28,26 +28,6 @@ User: admin
Password: inlong
```
-### Create Cluster Tag
-Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the
cluster label name and person in charge.
-
-
-
-:::caution
-`default_cluster` is the default ClusterTags reported by each component. If
you decide to use a different name, make sure to update the corresponding tag
configuration accordingly.
-:::
-
-### Register Pulsar Cluster
-Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar
Cluster.
-
-
-
-:::note
-The ClusterTags selects the newly created `default_cluster`, the Pulsar
cluster deployed by docker:
-
-Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`.
-:::
-
### Register ClickHouse DataNodes
Click [DataNodes] -> [Create] on the page to register ClickHouse DataNodes.
diff --git a/docs/quick_start/data_sync/mysql_iceberg_example.md
b/docs/quick_start/data_sync/mysql_iceberg_example.md
index 4841d8857da..5544428c98a 100644
--- a/docs/quick_start/data_sync/mysql_iceberg_example.md
+++ b/docs/quick_start/data_sync/mysql_iceberg_example.md
@@ -25,24 +25,6 @@ User: admin
Password: inlong
```
-### Create Cluster Tag
-Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the
cluster label name and responsible person.
-
-
-:::caution
-`default_cluster` is the default ClusterTags reported by each component. If
you decide to use a different name, make sure to update the corresponding tag
configuration accordingly.
-:::
-
-### Register Pulsar Cluster
-Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar
Cluster.
-
-
-:::note
-The ClusterTags selects the newly created `default_cluster`, the Pulsar
cluster deployed by docker:
-
-Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`.
-:::
-
### Register Iceberg DataNodes
Click [DataNodes] -> [Create] on the page to register Iceberg DataNodes.

diff --git a/docs/quick_start/data_sync/mysql_starrocks_example.md
b/docs/quick_start/data_sync/mysql_starrocks_example.md
index 405e8b77f0c..83d968fce63 100644
--- a/docs/quick_start/data_sync/mysql_starrocks_example.md
+++ b/docs/quick_start/data_sync/mysql_starrocks_example.md
@@ -26,25 +26,6 @@ User: admin
Password: inlong
```
-### Create Cluster Tag
-Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the
cluster label name and responsible person.
-
-
-:::caution
-`default_cluster` is the default ClusterTags reported by each component. If
you decide to use a different name, make sure to update the corresponding tag
configuration accordingly.
-:::
-
-### Register Pulsar Cluster
-Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar
Cluster.
-
-
-
-:::note
-The ClusterTags selects the newly created `default_cluster`, the Pulsar
cluster deployed by docker:
-
-Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`.
-:::
-
### Register StarRocks DataNodes
Click [DataNodes] -> [Create] on the page to register StarRocks DataNodes.

diff --git a/docs/quick_start/data_sync/mysql_clickhouse_example.md
b/docs/quick_start/data_sync/pulsar_clickhouse_example.md
similarity index 50%
copy from docs/quick_start/data_sync/mysql_clickhouse_example.md
copy to docs/quick_start/data_sync/pulsar_clickhouse_example.md
index 04df530de32..a879040cfcc 100644
--- a/docs/quick_start/data_sync/mysql_clickhouse_example.md
+++ b/docs/quick_start/data_sync/pulsar_clickhouse_example.md
@@ -1,9 +1,9 @@
---
-title: MySQL to ClickHouse Example
-sidebar_position: 1
+title: Pulsar to ClickHouse Example
+sidebar_position: 4
---
-Here we use an example to introduce how to use Apache InLong creating `MySQL
-> ClickHouse` data synchronization.
+Here we use an example to introduce how to use Apache InLong creating `Pulsar
-> ClickHouse` data synchronization.
## Deployment
### Install InLong
@@ -28,80 +28,65 @@ User: admin
Password: inlong
```
-### Create Cluster Tag
-Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the
cluster label name and person in charge.
-
-
-
-:::caution
-`default_cluster` is the default ClusterTags reported by each component. If
you decide to use a different name, make sure to update the corresponding tag
configuration accordingly.
-:::
-
-### Register Pulsar Cluster
-Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar
Cluster.
-
-
-
-:::note
-The ClusterTags selects the newly created `default_cluster`, the Pulsar
cluster deployed by docker:
-
-Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`.
-:::
-
### Register ClickHouse DataNodes
Click [DataNodes] -> [Create] on the page to register ClickHouse DataNodes.
-
+
## Create Task
### Create Data Streams Group
Click [Synchronization] → [Create] on the page and input the Group ID and
Steam ID:
-
+
### Create Data Source
-In the data source, click [New] → [MySQL] to configure the source name,
address, databases and tables information.
+In the data source, click [New] → [Pulsar] to configure the source name,
pulsar tenant, namespace, topic and other information.
-
+
:::note
-- Please create the test.source_table database table in advance, the schema
is: CREATE TABLE test.source_table (id INT PRIMARY KEY, name VARCHAR(50));
+- Please create the pulsar tenant, namespace and topic in advance, you can do
it by
[Pulsar-admin](https://pulsar.apache.org/docs/2.10.x/pulsar-admin/#create-3)
:::
### Create Data Sink
In the data target, click [New] → [ClickHouse] to configure the name, DB name,
table name, and created ck data node.
-
+
### Configuration Fields
Configure fields mapping in [Source Field] and [Target Field] respectively,
and click [Submit] after completion.
-
+
### Approve Data Stream
Click [Approval] -> [MyApproval] -> [Approval] -> [Ok].
-
+
Back to [Synchronization] page, wait for [success].
## Test Data
### Send Data
+Enter the pulsar container
+
+```shell
+docker exec -it pulsar /bin/bash
+```
+
+Insert 1000 pieces of data in total
```shell
#!/bin/bash
-# MySQL info
-DB_HOST="mysql"
-DB_USER="root"
-DB_PASS="inlong"
-DB_NAME="test"
-DB_TABLE="source_table"
+# Pulsar info
+TENANT="public"
+NAMESPACE="default"
+TOPIC="test"
# Insert data in a loop
for ((i=1; i<=1000; i++))
@@ -110,27 +95,19 @@ do
id=$i
name="name_$i"
- # Build an insert SQL
- query="INSERT INTO $DB_TABLE (id, name) VALUES ($id, '$name');"
+ # Build one message
+ message="$id|$name"
- # Execute insert SQL
- mysql -h $DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME -e "$query"
+ # Produce message to pulsar
+ bin/pulsar-client produce persistent://$TENANT/$NAMESPACE/$TOPIC
--messages $message
done
```
-Insert 1000 pieces of data in total:
-
-
-
### Verify Data
Then enter the ClickHouse container and view the source table data:
-
-
-You can also view audit data on the page:
-
-
+
## FAQ
ClickHouse fails to write data, you can view the error on the Flink page and
check the permissions of the user and table engine used.
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_clickhouse/cluster_tag.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_clickhouse/cluster_tag.png
deleted file mode 100644
index aa4b45fdc89..00000000000
Binary files
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_clickhouse/cluster_tag.png
and /dev/null differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_clickhouse/pulsar.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_clickhouse/pulsar.png
deleted file mode 100644
index fb75c178aa3..00000000000
Binary files
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_clickhouse/pulsar.png
and /dev/null differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_iceberg/create_cluster_tag.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_iceberg/create_cluster_tag.png
deleted file mode 100644
index aa4b45fdc89..00000000000
Binary files
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_iceberg/create_cluster_tag.png
and /dev/null differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_iceberg/create_pulsar_cluster.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_iceberg/create_pulsar_cluster.png
deleted file mode 100644
index fb75c178aa3..00000000000
Binary files
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_iceberg/create_pulsar_cluster.png
and /dev/null differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_starrocks/create_cluster_tag.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_starrocks/create_cluster_tag.png
deleted file mode 100644
index aa4b45fdc89..00000000000
Binary files
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_starrocks/create_cluster_tag.png
and /dev/null differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_starrocks/create_pulsar_cluster.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_starrocks/create_pulsar_cluster.png
deleted file mode 100644
index fb75c178aa3..00000000000
Binary files
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/mysql_starrocks/create_pulsar_cluster.png
and /dev/null differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/approve.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/approve.png
new file mode 100644
index 00000000000..c1376e2f274
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/approve.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/clickhouse_datanode.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/clickhouse_datanode.png
new file mode 100644
index 00000000000..34dc55c9788
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/clickhouse_datanode.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/group_stream.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/group_stream.png
new file mode 100644
index 00000000000..c0abc716d4a
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/group_stream.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink.png
new file mode 100644
index 00000000000..0813cd9f7c3
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink_data.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink_data.png
new file mode 100644
index 00000000000..a5d2db1a315
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink_data.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink_fields.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink_fields.png
new file mode 100644
index 00000000000..0e7f69daec9
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/sink_fields.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/source.png
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/source.png
new file mode 100644
index 00000000000..db2435d86f4
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/img/pulsar_clickhouse/source.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
index 72c12ed001d..08e0c3b5a28 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
@@ -27,18 +27,6 @@ docker run -d --rm --net=host --name clickhouse -e
CLICKHOUSE_USER=admin -e CLIC
User: admin
Password: inlong
```
-### 创建集群标签
-页面点击 【集群管理】→【标签管理】→【新建】
-
-
-
-**注意:default_cluster 是各个组件默认上报集群标签,如果使用其它名称,确认对应标签配置已修改。**
-
-### 注册 Pulsar 集群
-
-
-
-**可以参考截图信息填写内部,包括集群名称、所属标签、Pulsar 集群地址等。**
### 注册 ClickHouse 数据节点
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_iceberg_example.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_iceberg_example.md
index bcd4007d822..1c80e65338c 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_iceberg_example.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_iceberg_example.md
@@ -25,22 +25,6 @@ User: admin
Password: inlong
```
-### 创建集群标签
-页面点击 【集群管理】->【标签管理】->【新建】,指定集群标签名称和负责人:
-
-
-**注:default_cluster 是各个组件默认上报集群标签,如果使用其它名称,确认对应标签配置已修改。**
-
-### 注册 Pulsar 集群
-页面点击 [集群管理] -> [集群管理] -> [新建集群],注册 Pulsar 集群:
-
-
-:::note
-集群标签选择刚创建的 `default_cluster`,配置 Docker 部署的 Pulsar 集群:
-
-Service URL 为 `pulsar://pulsar:6650`, Admin URL 为 `http://pulsar:8080`.
-:::
-
### 注册 Iceberg 数据节点
页面点击 [数据节点] -> [创建] ,新增 Iceberg 数据节点.

diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_starrocks_example.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_starrocks_example.md
index 266872a45c9..665c7923578 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_starrocks_example.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_starrocks_example.md
@@ -26,22 +26,6 @@ User: admin
Password: inlong
```
-### 创建集群标签
-页面点击 【集群管理】->【标签管理】->【新建】,指定集群标签名称和负责人:
-
-
-**注:default_cluster 是各个组件默认上报集群标签,如果使用其它名称,确认对应标签配置已修改。**
-
-### 注册 Pulsar 集群
-页面点击 【集群管理】 -> 【集群管理】 -> 【新建集群】,注册 Pulsar 集群:
-
-
-:::note
-集群标签选择刚创建的 `default_cluster`,配置 Docker 部署的 Pulsar 集群:
-
-Service URL 为 `pulsar://pulsar:6650`, Admin URL 为 `http://pulsar:8080`.
-:::
-
### 注册 StarRocks 数据节点
页面点击 【数据节点】 -> 【创建】 ,新增 StarRocks 数据节点.

diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/pulsar_clickhouse_example.md
similarity index 52%
copy from
i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
copy to
i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/pulsar_clickhouse_example.md
index 72c12ed001d..9bb36ab1815 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/mysql_clickhouse_example.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/quick_start/data_sync/pulsar_clickhouse_example.md
@@ -1,9 +1,9 @@
---
-title: MySQL 到 ClickHouse 示例
-sidebar_position: 1
+title: Pulsar 到 ClickHouse 示例
+sidebar_position: 4
---
-在下面的内容中,我们将通过一个完整的示例介绍如何使用 Apache InLong 创建 MySQL -> ClickHouse 数据同步。
+在下面的内容中,我们将通过一个完整的示例介绍如何使用 Apache InLong 创建 Pulsar -> ClickHouse 数据同步。
## 环境部署
### 安装 InLong
@@ -27,101 +27,87 @@ docker run -d --rm --net=host --name clickhouse -e
CLICKHOUSE_USER=admin -e CLIC
User: admin
Password: inlong
```
-### 创建集群标签
-页面点击 【集群管理】→【标签管理】→【新建】
-
-
-
-**注意:default_cluster 是各个组件默认上报集群标签,如果使用其它名称,确认对应标签配置已修改。**
-
-### 注册 Pulsar 集群
-
-
-
-**可以参考截图信息填写内部,包括集群名称、所属标签、Pulsar 集群地址等。**
### 注册 ClickHouse 数据节点
页面点击 【数据节点】→【新建】,新增 ClickHouse 数据节点:
-
+
## 任务创建
### 新建数据流组
页面点击【数据同步】→【新建数据同步】,填写 数据流组 ID 和 SteamID:
-
+
### 新建数据源
-数据来源中 点击 【新建】→【MySQL】,配置数据源名称、地址、用户密码以及库表(test.source_table)信息等。
+数据来源中 点击 【新建】→【Pulsar】,配置数据源名称、地址、用户密码以及库表(test.source_table)信息等。
+
+
+
-
+- 注:请预先创建 Pulsar tenant,namespace 和 topic。
-注:请预先创建 test.source_table 库表,schema 为: CREATE TABLE test.source_table ( id INT
PRIMARY KEY, name VARCHAR(50) );
### 新建数据目标
数据目标中 点击 【新建】→【ClickHouse】,配置名称、库表、已创建的 ck 数据节点。数据同步不支持自动创建数据目标表,所以需要手动创建。
-
+
### 配置字段信息
分别在 【源字段】 和 【目标字段】中配置 Schema 映射信息,完成后点击 【提交审批】
-
+
### 审批数据流
页面点击【审批管理】->【我的审批】->【详情】->【通过】
-
+
返回 【数据同步】页面,等待任务配置成功
## 测试数据
### 发送数据
+进入 pulsar 容器
+```shell
+docker exec -it pulsar /bin/bash
+```
+
+插入 1000 条数据
```shell
#!/bin/bash
-# MySQL数据库连接信息
-DB_HOST="mysql"
-DB_USER="root"
-DB_PASS="inlong"
-DB_NAME="test"
-DB_TABLE="source_table"
+# Pulsar info
+TENANT="public"
+NAMESPACE="default"
+TOPIC="test"
-# 循环插入数据
+# Insert data in a loop
for ((i=1; i<=1000; i++))
do
-# 生成要插入的数据
+# Generate data
id=$i
name="name_$i"
- # 构建插入语句
- query="INSERT INTO $DB_TABLE (id, name) VALUES ($id, '$name');"
+ # Build one message
+ message="$id|$name"
- # 执行插入语句
- mysql -h $DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME -e "$query"
+ # Produce message to pulsar
+ bin/pulsar-client produce persistent://$TENANT/$NAMESPACE/$TOPIC
--messages $message
done
```
-累计插入 1000 条数据:
-
-
-
### 数据验证
然后进入 ClickHouse 容器,查看库表数据:
-
-
-也可以在页面查看审计数据:
-
-
+
## 常见问题
ClickHouse 写入数据失败,可以在 Flink 页面查看报错,并检查所使用的用户的权限以及表引擎
\ No newline at end of file