This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 756cd0db6d [INLONG-8413][Doc] Update the description about data
Ingestion and Sy… (#795)
756cd0db6d is described below
commit 756cd0db6d7d4d93782f315e4c9d869ed010314b
Author: Charles Zhang <[email protected]>
AuthorDate: Tue Jul 4 19:30:42 2023 +0800
[INLONG-8413][Doc] Update the description about data Ingestion and Sy…
(#795)
---
docs/deployment/bare_metal.md | 4 +-
docs/design_and_concept/basic_concept.md | 33 ++++++-----
docs/introduction.md | 62 ++++++++++-----------
.../current/deployment/bare_metal.md | 4 +-
.../current/design_and_concept/basic_concept.md | 7 ++-
.../current/introduction.md | 64 +++++++++++-----------
6 files changed, 90 insertions(+), 84 deletions(-)
diff --git a/docs/deployment/bare_metal.md b/docs/deployment/bare_metal.md
index 584ad36152..9447412610 100644
--- a/docs/deployment/bare_metal.md
+++ b/docs/deployment/bare_metal.md
@@ -17,7 +17,7 @@ InLong Support the following Message Queue services now, you
can choose one of t
You can get binary package from [Download
Page](https://inlong.apache.org/download) ,or you can build the InLong refer to
[How to Build](quick_start/how_to_build.md).。
## Standard Architecture
-You need deploy all InLong components for Standard Architecture.
+You need deploy all InLong components for Standard Architecture, which
supports `Data Ingestion`, `Data Synchronization` and `Data Subscription` at
the same time.
### Deploy
| order | component | dependencies | deploy guide
| description |
@@ -34,5 +34,5 @@ After the InLong cluster deployed successfully, you can
create a data stream ref
## Lightweight Architecture
### Deploy
-Lightweight Architecture only needs InLong Sort component, it is simple and
flexible, suitable for small-scale data.
+Lightweight Architecture only support `Data Synchronization`, you need to
deploy InLong Sort component, it is simple and flexible, suitable for
small-scale data.
You can deploy and use it refer [the deployment
guide](modules/sort/quick_start.md).
\ No newline at end of file
diff --git a/docs/design_and_concept/basic_concept.md
b/docs/design_and_concept/basic_concept.md
index cf41f67583..0158ae370f 100644
--- a/docs/design_and_concept/basic_concept.md
+++ b/docs/design_and_concept/basic_concept.md
@@ -3,18 +3,21 @@ title: Basic Concept
sidebar_position: 1
---
-| Name | Description
| Other
|
-|-------------|----------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
-| Standard | Standard Architecture, contains all InLong components such as
InLong Agent, Manager, MQ, Sort, Dashboard | Suitable for
massive data and large-scale production environments |
-| Lightweight | Lightweight Architecture, contains only one component of
InLong Sort, which also can be used with Manager, Dashboard | The
lightweight architecture is simple and flexible, suitable for small-scale data |
-| Group | Data Streams Group, it contains multiple data streams, and one
Group represents one data business unit. | Group has
attributes such as ID and Name. |
-| Stream | Data Stream, a stream has a specific data source, data format
and data sink. | Stream has
attributes such as ID, Name, and data fields. |
-| Node | Data Node, including `Extract Node` and `Load Node`, stands
for the data source and sink types separately. |
|
-| InLongMsg | InLong data format, if you consume message directly from the
message queue, you need to perform `InLongMsg` parsing first. |
|
-| Cluster | Each component can form a single cluster.
| Contains cluster
name, label, necessary information for each component, etc. |
-| Tag | Clusters of different components can use the same tag to
represent a set of data stream execution units. | Currently
tag are only available for clusters |
-| Agent | The standard architecture uses Agent for data collection, and
Agent represents different types of collection capabilities. | It contains File
Agent, SQL Agent, Binlog Agent, etc. |
-| DataProxy | Forward received data to different message queues.
| Supports data
transmission blocking, placing retransmission. |
-| Sort | Data stream sorting.
| Sort-flink based
on Flink, sort-standalone for local sorting. |
-| TubeMQ | InLong's self-developed message queuing service
| It can also be
called Tube, with low-cost, high-performance features. |
-| Pulsar | [Apache Pulsar](https://pulsar.apache.org/), a
high-performance, high-consistency message queue service |
|
\ No newline at end of file
+| Name | Description
| Other
|
+|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------|
+| Standard | Standard Architecture, contains all InLong components
such as InLong Agent, Manager, MQ, Sort, Dashboard
| Support `Data Ingestion`, `Data Synchronization` and
`Data Subscription` at the same time |
+| Lightweight | Lightweight Architecture, contains only one component
of InLong Sort, which also can be used with Manager, Dashboard
| The lightweight architecture is simple and flexible,
only support `Data Synchronization` |
+| Data Ingestion | Data ingestion is the process of moving data from a
source into a landing area or an object store where it can be used for ad hoc
queries and analytics |
|
+| Data Synchronization | Data synchronization is the process of establishing
consistency between source and target data stores, and the continuous
harmonization of the data over time |
|
+| Data Subscription | Data Subscription provides subscribers bulk data
feeds of the data they are entitled to access
|
|
+| Group | Data Streams Group, it contains multiple data
streams, and one Group represents one data business unit.
| Group has attributes such as ID and Name.
|
+| Stream | Data Stream, a stream has a specific data source,
data format and data sink.
| Stream has attributes such as ID, Name, and data
fields. |
+| Node | Data Node, including `Extract Node` and `Load Node`,
stands for the data source and sink types separately.
|
|
+| InLongMsg | InLong data format, if you consume message directly
from the message queue, you need to perform `InLongMsg` parsing first.
|
|
+| Cluster | Each component can form a single cluster.
| Contains cluster name, label, necessary information
for each component, etc. |
+| Tag | Clusters of different components can use the same tag
to represent a set of data stream execution units.
| Currently tag are only available for clusters
|
+| Agent | The standard architecture uses Agent for data
collection, and Agent represents different types of collection capabilities.
| It contains File Agent, SQL Agent, Binlog
Agent, etc. |
+| DataProxy | Forward received data to different message queues.
| Supports data transmission blocking, placing
retransmission. |
+| Sort | Data stream sorting.
| Sort-flink based on Flink, sort-standalone for local
sorting. |
+| TubeMQ | InLong's self-developed message queuing service
| It can also be called Tube, with low-cost,
high-performance features. |
+| Pulsar | [Apache Pulsar](https://pulsar.apache.org/), a
high-performance, high-consistency message queue service
|
|
\ No newline at end of file
diff --git a/docs/introduction.md b/docs/introduction.md
index e8d79bbc0a..6aa0f60ce5 100644
--- a/docs/introduction.md
+++ b/docs/introduction.md
@@ -45,10 +45,10 @@ values={[
{label: 'Standard', value: 'standard'},
{label: 'Lightweight', value: 'lightweight'},
]}>
-<TabItem value="standard"> Standard Architecture: contains all InLong
components such as InLong Agent, Manager, MQ, Sort, Dashboard, which is
suitable for massive data and large-scale production environments.
+<TabItem value="standard"> Standard Architecture: contains all InLong
components such as InLong Agent, Manager, MQ, Sort, Dashboard, which supports
`Data Ingestion`, `Data Synchronization` and `Data Subscription` at the same
time.
<img src="/img/inlong-structure-en.png" align="center" alt="Apache InLong"/>
</TabItem>
-<TabItem value="lightweight"> Lightweight Architecture: contains only one
component of InLong Sort, which also can be used with Manager, Dashboard, and
it is simple and flexible, suitable for small-scale data.
+<TabItem value="lightweight"> Lightweight Architecture: contains only one
component of InLong Sort, which also can be used with Manager, Dashboard, and
it is simple and flexible, only support `Data Synchronization`.
<img src="/img/inlong-structure-light.png" align="center" style={{objectFit:
'cover', height: '460px', width: '860px'}} alt="Apache InLong"/>
</TabItem>
</Tabs>
@@ -65,32 +65,32 @@ Apache InLong serves the entire life cycle from data
collection to landing, and
- **inlong-audit**, performs real-time audit and reconciliation on the
incoming and outgoing traffic of the Agent, DataProxy, and Sort modules of the
InLong system.
## Supported Data Nodes (Updating)
-| Type | Name | Version |
Architecture |
-|--------------|-------------------|------------------------------|-----------------------|
-| Extract Node | Auto Push | None | Standard
|
-| | File | None | Standard
|
-| | Kafka | 2.x |
Lightweight, Standard |
-| | MySQL | 5.6, 5.7, 8.0.x |
Lightweight, Standard |
-| | MongoDB | >= 3.6 |
Lightweight, Standard |
-| | MQTT | >= 3.1 | Standard
|
-| | Oracle | 11,12,19 |
Lightweight |
-| | PostgreSQL | 9.6, 10, 11, 12 |
Lightweight, Standard |
-| | Pulsar | 2.8.x |
Lightweight |
-| | Redis | 2.6.x | Standard
|
-| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
Lightweight, Standard |
-| Load Node | Auto Consumption | None | Standard
|
-| | Hive | 1.x, 2.x, 3.x |
Lightweight, Standard |
-| | Iceberg | 0.12.x |
Lightweight, Standard |
-| | ClickHouse | 20.7+ |
Lightweight, Standard |
-| | Kafka | 2.x |
Lightweight, Standard |
-| | HBase | 2.2.x |
Lightweight, Standard |
-| | PostgreSQL | 9.6, 10, 11, 12 |
Lightweight, Standard |
-| | Oracle | 11, 12, 19 |
Lightweight, Standard |
-| | MySQL | 5.6, 5.7, 8.0.x |
Lightweight, Standard |
-| | TDSQL-PostgreSQL | 10.17 |
Lightweight, Standard |
-| | Greenplum | 4.x, 5.x, 6.x |
Lightweight, Standard |
-| | Elasticsearch | 6.x, 7.x |
Lightweight, Standard |
-| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
Lightweight, Standard |
-| | Doris | >= 0.13 |
Lightweight, Standard |
-| | StarRocks | >= 2.0 |
Lightweight, Standard |
-| | HDFS | 2.x, 3.x |
Lightweight, Standard |
+| Type | Name | Version |
+|--------------|-------------------|------------------------------|
+| Extract Node | Auto Push | None |
+| | File | None |
+| | Kafka | 2.x |
+| | MySQL | 5.6, 5.7, 8.0.x |
+| | MongoDB | >= 3.6 |
+| | MQTT | >= 3.1 |
+| | Oracle | 11,12,19 |
+| | PostgreSQL | 9.6, 10, 11, 12 |
+| | Pulsar | 2.8.x |
+| | Redis | 2.6.x |
+| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
+| Load Node | Auto Consumption | None |
+| | Hive | 1.x, 2.x, 3.x |
+| | Iceberg | 0.12.x |
+| | ClickHouse | 20.7+ |
+| | Kafka | 2.x |
+| | HBase | 2.2.x |
+| | PostgreSQL | 9.6, 10, 11, 12 |
+| | Oracle | 11, 12, 19 |
+| | MySQL | 5.6, 5.7, 8.0.x |
+| | TDSQL-PostgreSQL | 10.17 |
+| | Greenplum | 4.x, 5.x, 6.x |
+| | Elasticsearch | 6.x, 7.x |
+| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
+| | Doris | >= 0.13 |
+| | StarRocks | >= 2.0 |
+| | HDFS | 2.x, 3.x |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/bare_metal.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/bare_metal.md
index 9dbc039399..e676b29bc6 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/bare_metal.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/bare_metal.md
@@ -16,7 +16,7 @@ InLong 当前支持以下消息队列，根据使用情况**选择其一**即可
可以从 [下载页面](https://inlong.apache.org/download) 获取二进制包，或参考
[如何编译](quick_start/how_to_build.md) 编译需要的版本。
## 标准架构
-标准架构下，你需要部署所有 InLong 组件。
+标准架构下，你需要部署所有 InLong 组件，同时支持`数据接入`、`数据同步`和`数据订阅`。
### 安装
| 顺序 | 组件 | 第三方依赖 | 部署指引
| 说明 |
@@ -33,4 +33,4 @@ InLong 集群部署成功后，你可以参考[Dashboard 使用指引](user_guid
## 轻量化架构
### 安装
-轻量化架构只需要 InLong Sort 组件,
拥有简单、灵活，适合小规模数据的特点。你可以参考[部署指引](modules/sort/quick_start.md)进行安装和使用
+轻量化架构只支持`数据同步`，需要部署 InLong Sort 组件,
拥有简单、灵活，适合小规模数据的特点。你可以参考[部署指引](modules/sort/quick_start.md)进行安装和使用
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
index 2f7e1733d2..494b7c7621 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
@@ -5,8 +5,11 @@ sidebar_position: 1
| Name | Description |
Other |
|-------------|--------------------------------------------------------------|-----------------------------------------------|
-| Standard | 标准架构，包含 InLong Agent、Manager、MQ、Sort、Dashboard 等所有 InLong 组件 |
适合海量数据、大规模生产环境 |
-| Lightweight | 轻量化架构，只包含 InLong Sort 一个组件，也可以搭配 Manager，Dashboard 一起使用 |
轻量化架构简单、灵活，适合小规模数据 |
+| Standard | 标准架构，包含 InLong Agent、Manager、MQ、Sort、Dashboard 等所有 InLong 组件 |
同时支持`数据接入`、`数据同步`和`数据订阅` |
+| Lightweight | 轻量化架构，只包含 InLong Sort 一个组件，也可以搭配 Manager，Dashboard 一起使用 |
轻量化架构简单、灵活，只支持`数据同步` |
+| 数据接入 | 数据接入是将数据从数据源汇聚到同一个存储服务的过程，可用于进一步数据查询和分析 |
|
+| 数据同步 | 数据同步是建立数据源和目标数据存储之间一致性的过程，可随着时间持续协调数据 |
|
+| 数据订阅 | 数据订阅为订阅者提供他们有权访问的数据 |
|
| Group | 数据流组，包含多个数据流，一个 Group 代表一个数据业务单元 |
Group 有 ID、Name 等属性 |
| Stream | 数据流，一个数据流有具体的数据源、数据格式和数据流向 |
Stream 有 ID、Name、数据字段等属性 |
| Node | 数据节点，包括 `Extract Node` 和 `Load Node`，分别代表数据源类型和数据流向目标类型 |
|
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
index fc7572acfd..8f2778055c 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
@@ -42,10 +42,10 @@ values={[
{label: '标准', value: 'standard'},
{label: '轻量化', value: 'lightweight'},
]}>
-<TabItem value="standard"> 标准架构：包含 InLong Agent、Manager、MQ、Sort、Dashboard 等所有
InLong 组件，适合海量数据、大规模生产环境。
+<TabItem value="standard"> 标准架构：包含 InLong Agent、Manager、MQ、Sort、Dashboard 等所有
InLong 组件，同时支持`数据接入`、`数据同步`和`数据订阅`。
<img src="/img/inlong-structure-zh.png" align="center" alt="Apache InLong"/>
</TabItem>
-<TabItem value="lightweight"> 轻量化架构：只包含 InLong Sort 一个组件，也可以搭配
Manager，Dashboard 一起使用。轻量化架构简单、灵活，适合小规模数据。
+<TabItem value="lightweight"> 轻量化架构：只包含 InLong Sort 一个组件，也可以搭配
Manager，Dashboard 一起使用。轻量化架构简单、灵活，只支持`数据同步`。
<img src="/img/inlong-structure-light.png" align="center" style={{objectFit:
'cover', height: '460px', width: '860px'}} alt="Apache InLong"/>
</TabItem>
</Tabs>
@@ -62,35 +62,35 @@ Apache InLong 服务于数据采集到落地的整个生命周期，按数据的
- **inlong-audit**，对 InLong 系统的 Agent、DataProxy、Sort 模块的入流量、出流量进行实时审计对账。
## 已支持数据节点（更新中）
-| Type | Name | Version |
Architecture |
-|--------------|-------------------|------------------------------|-----------------------|
-| Extract Node | Auto Push | None | Standard
|
-| | File | None | Standard
|
-| | Kafka | 2.x |
Lightweight, Standard |
-| | MySQL | 5.6, 5.7, 8.0.x |
Lightweight, Standard |
-| | MongoDB | >= 3.6 |
Lightweight, Standard |
-| | MQTT | >= 3.1 | Standard
|
-| | Oracle | 11,12,19 |
Lightweight |
-| | PostgreSQL | 9.6, 10, 11, 12 |
Lightweight, Standard |
-| | Pulsar | 2.8.x |
Lightweight |
-| | Redis | 2.6.x | Standard
|
-| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
Lightweight, Standard |
-| Load Node | Auto Consumption | None | Standard
|
-| | Hive | 1.x, 2.x, 3.x |
Lightweight, Standard |
-| | Iceberg | 0.12.x |
Lightweight, Standard |
-| | Hudi | 0.12.x |
Lightweight, Standard |
-| | ClickHouse | 20.7+ |
Lightweight, Standard |
-| | Kafka | 2.x |
Lightweight, Standard |
-| | HBase | 2.2.x |
Lightweight, Standard |
-| | PostgreSQL | 9.6, 10, 11, 12 |
Lightweight, Standard |
-| | Oracle | 11, 12, 19 |
Lightweight, Standard |
-| | MySQL | 5.6, 5.7, 8.0.x |
Lightweight, Standard |
-| | TDSQL-PostgreSQL | 10.17 |
Lightweight, Standard |
-| | Greenplum | 4.x, 5.x, 6.x |
Lightweight, Standard |
-| | Elasticsearch | 6.x, 7.x |
Lightweight, Standard |
-| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
Lightweight, Standard |
-| | Doris | >= 0.13 |
Lightweight, Standard |
-| | StarRocks | >= 2.0 |
Lightweight, Standard |
-| | HDFS | 2.x, 3.x |
Lightweight, Standard |
+| Type | Name | Version |
+|--------------|-------------------|------------------------------|
+| Extract Node | Auto Push | None |
+| | File | None |
+| | Kafka | 2.x |
+| | MySQL | 5.6, 5.7, 8.0.x |
+| | MongoDB | >= 3.6 |
+| | MQTT | >= 3.1 |
+| | Oracle | 11,12,19 |
+| | PostgreSQL | 9.6, 10, 11, 12 |
+| | Pulsar | 2.8.x |
+| | Redis | 2.6.x |
+| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
+| Load Node | Auto Consumption | None |
+| | Hive | 1.x, 2.x, 3.x |
+| | Iceberg | 0.12.x |
+| | Hudi | 0.12.x |
+| | ClickHouse | 20.7+ |
+| | Kafka | 2.x |
+| | HBase | 2.2.x |
+| | PostgreSQL | 9.6, 10, 11, 12 |
+| | Oracle | 11, 12, 19 |
+| | MySQL | 5.6, 5.7, 8.0.x |
+| | TDSQL-PostgreSQL | 10.17 |
+| | Greenplum | 4.x, 5.x, 6.x |
+| | Elasticsearch | 6.x, 7.x |
+| | SQLServer | 2012, 2014, 2016, 2017, 2019 |
+| | Doris | >= 0.13 |
+| | StarRocks | >= 2.0 |
+| | HDFS | 2.x, 3.x |
