This is an automated email from the ASF dual-hosted git repository.
critas pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iotdb-docs.git
The following commit(s) were added to refs/heads/main by this push:
new 4c7c13ac add pipe syntactic sugar to tree model 13x (#936)
4c7c13ac is described below
commit 4c7c13ace0749d26e8321eaa4631aa4a380d4790
Author: leto-b <[email protected]>
AuthorDate: Fri Dec 26 11:13:21 2025 +0800
add pipe syntactic sugar to tree model 13x (#936)
---
.../V1.3.x/User-Manual/Data-Sync_apache.md | 34 +++++++++++++++++++++
.../V1.3.x/User-Manual/Data-Sync_timecho.md | 32 ++++++++++++++++++++
.../dev-1.3/User-Manual/Data-Sync_apache.md | 34 +++++++++++++++++++++
.../dev-1.3/User-Manual/Data-Sync_timecho.md | 32 ++++++++++++++++++++
.../V1.3.x/User-Manual/Data-Sync_apache.md | 33 ++++++++++++++++++++
.../V1.3.x/User-Manual/Data-Sync_timecho.md | 31 +++++++++++++++++++
.../dev-1.3/User-Manual/Data-Sync_apache.md | 35 +++++++++++++++++++++-
.../dev-1.3/User-Manual/Data-Sync_timecho.md | 32 +++++++++++++++++++-
8 files changed, 261 insertions(+), 2 deletions(-)
diff --git a/src/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
b/src/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
index 658b9d47..6f726a4d 100644
--- a/src/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
+++ b/src/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
@@ -123,6 +123,16 @@ WITH SINK (
**IF NOT EXISTS semantics**: Used in creation operations to ensure that the
create command is executed when the specified Pipe does not exist, preventing
errors caused by attempting to create an existing Pipe.
+**Note**:
+
+Starting from V1.3.6, when creating a full data synchronization Pipe (e.g.
Pipeid: `alldatapipe`), the system will automatically split it into two
independent Pipes:
+
+* History Pipe: The PipeId is the original name plus the suffix `_history`
(e.g. `alldatapipe_history`). The source parameter carries the default
configurations: `'realtime.enable'='false', 'inclusion'='data.insert',
'inclusion.exclusion'=''`
+* Realtime Pipe: The PipeId is the original name plus the suffix `_realtime`
(e.g. `alldatapipe_realtime`). The source parameter carries the default
configuration: `'history.enable'='false'`. If metadata synchronization is
configured, the Realtime Pipe will be responsible for sending the data.
+
+After successful creation, the original PipeId (e.g. `alldatapipe`) will no
longer be a valid identifier. When performing task operations such as starting,
stopping, deleting, or viewing, you must use the split independent PipeId (i.e.
`*_history` or `*_realtime`). For operation examples, see the [View
Task](./Data-Sync_apache.md#view-task) section
+
+
### Start Task
Start processing data:
@@ -186,6 +196,30 @@ The meanings of each column are as follows:
- **RemainingEventCount (Statistics with Delay)**: The number of remaining
events, which is the total count of all events in the current data
synchronization task, including data and schema synchronization events, as well
as system and user-defined events.
- **EstimatedRemainingSeconds (Statistics with Delay)**: The estimated
remaining time, based on the current number of events and the rate at the pipe,
to complete the transfer.
+Example:
+
+In V1.3.6 and later versions, create a full data synchronization task and view
the task details.
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
+
### Synchronization Plugins
To make the overall architecture more flexible to match different
synchronization scenario requirements, we support plugin assembly within the
synchronization task framework. The system comes with some pre-installed common
plugins that you can use directly. At the same time, you can also customize
processor plugins and Sink plugins, and load them into the IoTDB system for
use. You can view the plugins in the system (including custom and built-in
plugins) with the following statement:
diff --git a/src/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
b/src/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
index 4bfb5c6b..dc0d6573 100644
--- a/src/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
+++ b/src/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
@@ -125,6 +125,15 @@ WITH SINK (
**IF NOT EXISTS semantics**: Used in creation operations to ensure that the
create command is executed when the specified Pipe does not exist, preventing
errors caused by attempting to create an existing Pipe.
+**Note**:
+
+Starting from V1.3.6, when creating a full data synchronization Pipe (e.g.
Pipeid: `alldatapipe`), the system will automatically split it into two
independent Pipes:
+
+* History Pipe: The PipeId is the original name plus the suffix `_history`
(e.g. `alldatapipe_history`). The source parameter carries the default
configurations: `'realtime.enable'='false', 'inclusion'='data.insert',
'inclusion.exclusion'=''`
+* Realtime Pipe: The PipeId is the original name plus the suffix `_realtime`
(e.g. `alldatapipe_realtime`). The source parameter carries the default
configuration: `'history.enable'='false'`. If metadata synchronization is
configured, the Realtime Pipe will be responsible for sending the data.
+
+After successful creation, the original PipeId (e.g. `alldatapipe`) will no
longer be a valid identifier. When performing task operations such as starting,
stopping, deleting, or viewing, you must use the split independent PipeId (i.e.
`*_history` or `*_realtime`). For operation examples, see the [View
Task](./Data-Sync_timecho.md#view-task) section
+
### Start Task
Start processing data:
@@ -188,6 +197,29 @@ The meanings of each column are as follows:
- **RemainingEventCount (Statistics with Delay)**: The number of remaining
events, which is the total count of all events in the current data
synchronization task, including data and schema synchronization events, as well
as system and user-defined events.
- **EstimatedRemainingSeconds (Statistics with Delay)**: The estimated
remaining time, based on the current number of events and the rate at the pipe,
to complete the transfer.
+Example:
+
+In V1.3.6 and later versions, create a full data synchronization task and view
the task details.
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
### Synchronization Plugins
To make the overall architecture more flexible to match different
synchronization scenario requirements, we support plugin assembly within the
synchronization task framework. The system comes with some pre-installed common
plugins that you can use directly. At the same time, you can also customize
processor plugins and Sink plugins, and load them into the IoTDB system for
use. You can view the plugins in the system (including custom and built-in
plugins) with the following statement:
diff --git a/src/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
b/src/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
index d48c05be..dbaf6efe 100644
--- a/src/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
+++ b/src/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
@@ -123,6 +123,16 @@ WITH SINK (
**IF NOT EXISTS semantics**: Used in creation operations to ensure that the
create command is executed when the specified Pipe does not exist, preventing
errors caused by attempting to create an existing Pipe.
+**Note**:
+
+Starting from V1.3.6, when creating a full data synchronization Pipe (e.g.
Pipeid: `alldatapipe`), the system will automatically split it into two
independent Pipes:
+
+* History Pipe: The PipeId is the original name plus the suffix `_history`
(e.g. `alldatapipe_history`). The source parameter carries the default
configurations: `'realtime.enable'='false', 'inclusion'='data.insert',
'inclusion.exclusion'=''`
+* Realtime Pipe: The PipeId is the original name plus the suffix `_realtime`
(e.g. `alldatapipe_realtime`). The source parameter carries the default
configuration: `'history.enable'='false'`. If metadata synchronization is
configured, the Realtime Pipe will be responsible for sending the data.
+
+After successful creation, the original PipeId (e.g. `alldatapipe`) will no
longer be a valid identifier. When performing task operations such as starting,
stopping, deleting, or viewing, you must use the split independent PipeId (i.e.
`*_history` or `*_realtime`). For operation examples, see the [View
Task](./Data-Sync_apache.md#view-task) section
+
+
### Start Task
Start processing data:
@@ -186,6 +196,30 @@ The meanings of each column are as follows:
- **RemainingEventCount (Statistics with Delay)**: The number of remaining
events, which is the total count of all events in the current data
synchronization task, including data and schema synchronization events, as well
as system and user-defined events.
- **EstimatedRemainingSeconds (Statistics with Delay)**: The estimated
remaining time, based on the current number of events and the rate at the pipe,
to complete the transfer.
+Example:
+
+In V1.3.6 and later versions, create a full data synchronization task and view
the task details.
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
+
### Synchronization Plugins
To make the overall architecture more flexible to match different
synchronization scenario requirements, we support plugin assembly within the
synchronization task framework. The system comes with some pre-installed common
plugins that you can use directly. At the same time, you can also customize
processor plugins and Sink plugins, and load them into the IoTDB system for
use. You can view the plugins in the system (including custom and built-in
plugins) with the following statement:
diff --git a/src/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
b/src/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
index 4bfb5c6b..dc0d6573 100644
--- a/src/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
+++ b/src/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
@@ -125,6 +125,15 @@ WITH SINK (
**IF NOT EXISTS semantics**: Used in creation operations to ensure that the
create command is executed when the specified Pipe does not exist, preventing
errors caused by attempting to create an existing Pipe.
+**Note**:
+
+Starting from V1.3.6, when creating a full data synchronization Pipe (e.g.
Pipeid: `alldatapipe`), the system will automatically split it into two
independent Pipes:
+
+* History Pipe: The PipeId is the original name plus the suffix `_history`
(e.g. `alldatapipe_history`). The source parameter carries the default
configurations: `'realtime.enable'='false', 'inclusion'='data.insert',
'inclusion.exclusion'=''`
+* Realtime Pipe: The PipeId is the original name plus the suffix `_realtime`
(e.g. `alldatapipe_realtime`). The source parameter carries the default
configuration: `'history.enable'='false'`. If metadata synchronization is
configured, the Realtime Pipe will be responsible for sending the data.
+
+After successful creation, the original PipeId (e.g. `alldatapipe`) will no
longer be a valid identifier. When performing task operations such as starting,
stopping, deleting, or viewing, you must use the split independent PipeId (i.e.
`*_history` or `*_realtime`). For operation examples, see the [View
Task](./Data-Sync_timecho.md#view-task) section
+
### Start Task
Start processing data:
@@ -188,6 +197,29 @@ The meanings of each column are as follows:
- **RemainingEventCount (Statistics with Delay)**: The number of remaining
events, which is the total count of all events in the current data
synchronization task, including data and schema synchronization events, as well
as system and user-defined events.
- **EstimatedRemainingSeconds (Statistics with Delay)**: The estimated
remaining time, based on the current number of events and the rate at the pipe,
to complete the transfer.
+Example:
+
+In V1.3.6 and later versions, create a full data synchronization task and view
the task details.
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
### Synchronization Plugins
To make the overall architecture more flexible to match different
synchronization scenario requirements, we support plugin assembly within the
synchronization task framework. The system comes with some pre-installed common
plugins that you can use directly. At the same time, you can also customize
processor plugins and Sink plugins, and load them into the IoTDB system for
use. You can view the plugins in the system (including custom and built-in
plugins) with the following statement:
diff --git a/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
b/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
index abd8b35c..59c2f5e4 100644
--- a/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
+++ b/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_apache.md
@@ -122,6 +122,15 @@ WITH SINK (
**IF NOT EXISTS 语义**:用于创建操作中,确保当指定 Pipe 不存在时,执行创建命令,防止因尝试创建已存在的 Pipe 而导致报错。
+**注意**:V1.3.6 起,创建一个全量数据同步 Pipe (例如 Pipeid : `alldatapipe`)时,系统会自动将其拆分为两个独立的
Pipe:
+
+* 历史 Pipe:PipeId 为原名称加 _history后缀(如 `alldatapipe_history`),source 参数默认携带
`'realtime.enable'='false', 'inclusion'='data.insert', 'inclusion.exclusion'=''`
+
+* 实时 Pipe:PipeId 为原名称加 _realtime后缀(如 `alldatapipe_realtime`),source 参数默认携带
`'history.enable'='false'` ,若配置了元数据同步,则由实时 Pipe 负责发送
+
+创建成功后,原 PipeId(如 `alldatapipe`)将不再作为有效标识符。在进行启动、停止、删除、查看等任务操作时,必须使用拆分后的独立
PipeId(即 `*_history`或 `*_realtime`)。操作示例见[查看任务](./Data-Sync_apache.md#查看任务)小节
+
+
### 开始任务
开始处理数据:
@@ -187,6 +196,30 @@ SHOW PIPE <PipeId>
- **RemainingEventCount(统计存在延迟)**:剩余 event 数,当前数据同步任务中的所有 event 总数,包括数据和元数据同步的
event,以及系统和用户自定义的 event。
- **EstimatedRemainingSeconds(统计存在延迟)**:剩余时间,基于当前 event 个数和 pipe
处速率,预估完成传输的剩余时间。
+示例:
+
+在 V1.3.6 及之后的版本中,创建一个全量数据同步任务,并查看该任务详情
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
+
### 同步插件
为了使得整体架构更加灵活以匹配不同的同步场景需求,我们支持在同步任务框架中进行插件组装。系统为您预置了一些常用插件可直接使用,同时您也可以自定义
processor 插件 和 Sink 插件,并加载至 IoTDB 系统进行使用。查看系统中的插件(含自定义与内置插件)可以用以下语句:
diff --git a/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
b/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
index ffaa79dc..70afcf90 100644
--- a/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
+++ b/src/zh/UserGuide/V1.3.x/User-Manual/Data-Sync_timecho.md
@@ -124,6 +124,14 @@ WITH SINK (
**IF NOT EXISTS 语义**:用于创建操作中,确保当指定 Pipe 不存在时,执行创建命令,防止因尝试创建已存在的 Pipe 而导致报错。
+**注意**:V1.3.6 起,创建一个全量数据同步 Pipe (例如 Pipeid : `alldatapipe`)时,系统会自动将其拆分为两个独立的
Pipe:
+
+* 历史 Pipe:PipeId 为原名称加 _history后缀(如 `alldatapipe_history`),source 参数默认携带
`'realtime.enable'='false', 'inclusion'='data.insert', 'inclusion.exclusion'=''`
+
+* 实时 Pipe:PipeId 为原名称加 _realtime后缀(如 `alldatapipe_realtime`),source 参数默认携带
`'history.enable'='false'` ,若配置了元数据同步,则由实时 Pipe 负责发送
+
+创建成功后,原 PipeId(如 `alldatapipe`)将不再作为有效标识符。在进行启动、停止、删除、查看等任务操作时,必须使用拆分后的独立
PipeId(即 `*_history`或 `*_realtime`)。操作示例见[查看任务](./Data-Sync_timecho.md#查看任务)小节
+
### 开始任务
开始处理数据:
@@ -188,6 +196,29 @@ SHOW PIPE <PipeId>
- **RemainingEventCount(统计存在延迟)**:剩余 event 数,当前数据同步任务中的所有 event 总数,包括数据和元数据同步的
event,以及系统和用户自定义的 event。
- **EstimatedRemainingSeconds(统计存在延迟)**:剩余时间,基于当前 event 个数和 pipe
处速率,预估完成传输的剩余时间。
+示例:
+
+在 V1.3.6 及之后的版本中,创建一个全量数据同步任务,并查看该任务详情
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
### 同步插件
为了使得整体架构更加灵活以匹配不同的同步场景需求,我们支持在同步任务框架中进行插件组装。系统为您预置了一些常用插件可直接使用,同时您也可以自定义
processor 插件 和 Sink 插件,并加载至 IoTDB 系统进行使用。查看系统中的插件(含自定义与内置插件)可以用以下语句:
diff --git a/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
b/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
index cb9b315d..59c2f5e4 100644
--- a/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
+++ b/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_apache.md
@@ -122,6 +122,15 @@ WITH SINK (
**IF NOT EXISTS 语义**:用于创建操作中,确保当指定 Pipe 不存在时,执行创建命令,防止因尝试创建已存在的 Pipe 而导致报错。
+**注意**:V1.3.6 起,创建一个全量数据同步 Pipe (例如 Pipeid : `alldatapipe`)时,系统会自动将其拆分为两个独立的
Pipe:
+
+* 历史 Pipe:PipeId 为原名称加 _history后缀(如 `alldatapipe_history`),source 参数默认携带
`'realtime.enable'='false', 'inclusion'='data.insert', 'inclusion.exclusion'=''`
+
+* 实时 Pipe:PipeId 为原名称加 _realtime后缀(如 `alldatapipe_realtime`),source 参数默认携带
`'history.enable'='false'` ,若配置了元数据同步,则由实时 Pipe 负责发送
+
+创建成功后,原 PipeId(如 `alldatapipe`)将不再作为有效标识符。在进行启动、停止、删除、查看等任务操作时,必须使用拆分后的独立
PipeId(即 `*_history`或 `*_realtime`)。操作示例见[查看任务](./Data-Sync_apache.md#查看任务)小节
+
+
### 开始任务
开始处理数据:
@@ -187,6 +196,30 @@ SHOW PIPE <PipeId>
- **RemainingEventCount(统计存在延迟)**:剩余 event 数,当前数据同步任务中的所有 event 总数,包括数据和元数据同步的
event,以及系统和用户自定义的 event。
- **EstimatedRemainingSeconds(统计存在延迟)**:剩余时间,基于当前 event 个数和 pipe
处速率,预估完成传输的剩余时间。
+示例:
+
+在 V1.3.6 及之后的版本中,创建一个全量数据同步任务,并查看该任务详情
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
+
### 同步插件
为了使得整体架构更加灵活以匹配不同的同步场景需求,我们支持在同步任务框架中进行插件组装。系统为您预置了一些常用插件可直接使用,同时您也可以自定义
processor 插件 和 Sink 插件,并加载至 IoTDB 系统进行使用。查看系统中的插件(含自定义与内置插件)可以用以下语句:
@@ -289,7 +322,7 @@ with sink (
create pipe A2B
WITH SOURCE (
'source'= 'iotdb-source',
- 'realtime.mode' = 'stream', -- 新插入数据(pipe创建后)的抽取模式
+ 'realtime.mode' = 'stream' -- 新插入数据(pipe创建后)的抽取模式
'path' = 'root.vehicle.**', -- 同步数据的范围
'start-time' = '2023.08.23T08:00:00+00:00', -- 同步所有数据的开始 event time,包含
start-time
'end-time' = '2023.10.23T08:00:00+00:00' -- 同步所有数据的结束 event time,包含 end-time
diff --git a/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
b/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
index 8a74c427..632fe766 100644
--- a/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
+++ b/src/zh/UserGuide/dev-1.3/User-Manual/Data-Sync_timecho.md
@@ -124,6 +124,14 @@ WITH SINK (
**IF NOT EXISTS 语义**:用于创建操作中,确保当指定 Pipe 不存在时,执行创建命令,防止因尝试创建已存在的 Pipe 而导致报错。
+**注意**:V1.3.6 起,创建一个全量数据同步 Pipe (例如 Pipeid : `alldatapipe`)时,系统会自动将其拆分为两个独立的
Pipe:
+
+* 历史 Pipe:PipeId 为原名称加 _history后缀(如 `alldatapipe_history`),source 参数默认携带
`'realtime.enable'='false', 'inclusion'='data.insert', 'inclusion.exclusion'=''`
+
+* 实时 Pipe:PipeId 为原名称加 _realtime后缀(如 `alldatapipe_realtime`),source 参数默认携带
`'history.enable'='false'` ,若配置了元数据同步,则由实时 Pipe 负责发送
+
+创建成功后,原 PipeId(如 `alldatapipe`)将不再作为有效标识符。在进行启动、停止、删除、查看等任务操作时,必须使用拆分后的独立
PipeId(即 `*_history`或 `*_realtime`)。操作示例见[查看任务](./Data-Sync_timecho.md#查看任务)小节
+
### 开始任务
开始处理数据:
@@ -188,6 +196,29 @@ SHOW PIPE <PipeId>
- **RemainingEventCount(统计存在延迟)**:剩余 event 数,当前数据同步任务中的所有 event 总数,包括数据和元数据同步的
event,以及系统和用户自定义的 event。
- **EstimatedRemainingSeconds(统计存在延迟)**:剩余时间,基于当前 event 个数和 pipe
处速率,预估完成传输的剩余时间。
+示例:
+
+在 V1.3.6 及之后的版本中,创建一个全量数据同步任务,并查看该任务详情
+
+```sql
+IoTDB> create pipe alldatapipe with
source('inclusion'='all','exclusion'='auth') with
sink('node-urls'='127.0.0.1:6668')
+
+IoTDB> show pipe alldatapipe_history
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_history|2025-12-18T15:06:16.697|RUNNING|{exclusion=auth,
history.enable=true, inclusion=data.insert, inclusion.exclusion=,
realtime.enable=false}| {}|{node-urls=127.0.0.1:6668}|
| 0| 0.00|
++-------------------+-----------------------+-------+---------------------------------------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+IoTDB> show pipe alldatapipe_realtime
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+| ID| CreationTime| State|
PipeSource|PipeProcessor|
PipeSink|ExceptionMessage|RemainingEventCount|EstimatedRemainingSeconds|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+|alldatapipe_realtime|2025-12-18T15:06:16.312|RUNNING|{exclusion=auth,
history.enable=false, inclusion=all, realtime.enable=true}|
{}|{node-urls=127.0.0.1:6668}| | 0|
0.00|
++--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
+
+```
+
### 同步插件
为了使得整体架构更加灵活以匹配不同的同步场景需求,我们支持在同步任务框架中进行插件组装。系统为您预置了一些常用插件可直接使用,同时您也可以自定义
processor 插件 和 Sink 插件,并加载至 IoTDB 系统进行使用。查看系统中的插件(含自定义与内置插件)可以用以下语句:
@@ -442,7 +473,6 @@ with sink (
)
```
**注意:目前支持的网闸型号**
-
> 其他型号的网闸设备,请与天谋商务联系确认是否支持。
| 网闸类型 | 网闸型号 | 回包限制 | 发送限制
|