This is an automated email from the ASF dual-hosted git repository.
wanghailin pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 875fcd89c5 [Improve][Doc] Update the transform contribute guide (#8487)
875fcd89c5 is described below
commit 875fcd89c59faa4c7b97f3e2c1bf3be707a205cd
Author: Jia Fan <[email protected]>
AuthorDate: Wed Jan 15 13:31:25 2025 +0800
[Improve][Doc] Update the transform contribute guide (#8487)
---
docs/en/contribution/contribute-plugin.md | 2 +-
.../contribution/contribute-transform-v2-guide.md | 330 +--------------------
docs/zh/contribution/contribute-plugin.md | 4 +-
.../contribution/contribute-transform-v2-guide.md | 322 +-------------------
.../README.md | 254 ++++++----------
seatunnel-transforms-v2/README.zh.md | 254 ++++++++++++++++
6 files changed, 359 insertions(+), 807 deletions(-)
diff --git a/docs/en/contribution/contribute-plugin.md
b/docs/en/contribution/contribute-plugin.md
index 17275e35f0..03d74edf55 100644
--- a/docs/en/contribution/contribute-plugin.md
+++ b/docs/en/contribution/contribute-plugin.md
@@ -1,4 +1,4 @@
-# Contribute Connector-v2 Plugins
+# Contribute Connector-V2 Plugins
If you want to contribute Connector-V2, please click the Connector-V2
Contribution Guide below for reference. It can help you enter development more
quickly.
diff --git a/docs/en/contribution/contribute-transform-v2-guide.md
b/docs/en/contribution/contribute-transform-v2-guide.md
index 37837f9eeb..5bf32dd40d 100644
--- a/docs/en/contribution/contribute-transform-v2-guide.md
+++ b/docs/en/contribution/contribute-transform-v2-guide.md
@@ -1,329 +1,5 @@
-# Contribute Transform Guide
+# Contribute Transform-V2 Plugins
-This document describes how to understand, develop and contribute a transform.
+If you want to contribute Transform-V2, please click the Transform-V2
Contribution Guide below for reference. It can help you enter development more
quickly.
-We also provide the [Transform E2E
Test](../../../seatunnel-e2e/seatunnel-transforms-v2-e2e)
-to verify the data input and output by the transform.
-
-## Concepts
-
-Using SeaTunnel you can read or write data through the connector, but if you
need to
-process your data after reading or before writing, then need to use transform.
-
-Use transform to make simple edits to your data rows or fields, such as split
field,
-change field values, add or remove field.
-
-### DataType Transform
-
-Transform receives datatype input from upstream(source or transform) and
outputs new datatype to
-downstream(sink or transform), this process is datatype transform.
-
-Example 1：Remove fields
-
-```shell
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-| A | B |
-|-----------|-----------|
-| STRING | INT |
-```
-
-Example 2：Sort fields
-
-```shell
-| B | C | A |
-|-----------|-----------|-----------|
-| INT | BOOLEAN | STRING |
-
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-```
-
-Example 3：Update fields datatype
-
-```shell
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | STRING | STRING |
-```
-
-Example 4：Add new fields
-
-```shell
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-
-| A | B | C | D |
-|-----------|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN | DOUBLE |
-```
-
-### Data Transform
-
-After datatype transformed, Transform will receive data-row input from
upstream(source or transform),
-edit into data-row with [New Datatype](#DataType transform) and output to
downstream (sink or transform).
-This process is called data transform.
-
-### Translation
-
-Transform is decoupled from the execution engine, any transform implement can
run into all engines
-without changing the code & config, which requires the translation layer to
adapt transform and execution engine.
-
-Example：Translation datatype & data
-
-```shell
-Original:
-
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-Datatype translation:
-
-| A | B | C |
-|-------------------|-------------------|-------------------|
-| ENGINE<STRING> | ENGINE<INT> | ENGINE<BOOLEAN> |
-
-Data translation:
-
-| A | B | C |
-|-------------------|-------------------|-------------------|
-| ENGINE<"test"> | ENGINE<1> | ENGINE<false> |
-```
-
-## Core APIs
-
-### SeaTunnelTransform
-
-`SeaTunnelTransform` provides all major and primary APIs, you can subclass it
to do whatever transform.
-
-1. Receive datatype input from upstream.
-
-```java
-/**
- * Set the data type info of input data.
- *
- * @param inputDataType The data type info of upstream input.
- */
- void setTypeInfo(SeaTunnelDataType<T> inputDataType);
-```
-
-2. Outputs new datatype to downstream.
-
-```java
-/**
- * Get the data type of the records produced by this transform.
- *
- * @return Produced data type.
- */
-SeaTunnelDataType<T> getProducedType();
-```
-
-3. Edit input data and outputs new data to downstream.
-
-```java
-/**
- * Transform input data to {@link this#getProducedType()} types data.
- *
- * @param row the data need be transform.
- * @return transformed data.
- */
-T map(T row);
-```
-
-### SingleFieldOutputTransform
-
-`SingleFieldOutputTransform` abstract single field change operator
-
-1. Define output field
-
-```java
-/**
- * Outputs new field
- *
- * @return
- */
-protected abstract String getOutputFieldName();
-```
-
-2. Define output field datatype
-
-```java
-/**
- * Outputs new field datatype
- *
- * @return
- */
-protected abstract SeaTunnelDataType getOutputFieldDataType();
-```
-
-3. Define output field value
-
-```java
-/**
- * Outputs new field value
- *
- * @param inputRow The inputRow of upstream input.
- * @return
- */
-protected abstract Object getOutputFieldValue(SeaTunnelRowAccessor inputRow);
-```
-
-### MultipleFieldOutputTransform
-
-`MultipleFieldOutputTransform` abstract multiple fields change operator
-
-1. Define output fields
-
-```java
-/**
- * Outputs new fields
- *
- * @return
- */
-protected abstract String[] getOutputFieldNames();
-```
-
-2. Define output fields datatype
-
-```java
-/**
- * Outputs new fields datatype
- *
- * @return
- */
-protected abstract SeaTunnelDataType[] getOutputFieldDataTypes();
-```
-
-3. Define output field values
-
-```java
-/**
- * Outputs new fields value
- *
- * @param inputRow The inputRow of upstream input.
- * @return
- */
-protected abstract Object[] getOutputFieldValues(SeaTunnelRowAccessor
inputRow);
-```
-
-### AbstractSeaTunnelTransform
-
-`AbstractSeaTunnelTransform` abstract datatype & fields change operator
-
-1. Transform input row type and outputs new row type
-
-```java
-/**
- * Outputs transformed row type.
- *
- * @param inputRowType upstream input row type
- * @return
- */
-protected abstract SeaTunnelRowType transformRowType(SeaTunnelRowType
inputRowType);
-```
-
-2. Transform input row data and outputs new row data
-
-```java
-/**
- * Outputs transformed row data.
- *
- * @param inputRow upstream input row data
- * @return
- */
-protected abstract SeaTunnelRow transformRow(SeaTunnelRow inputRow);
-```
-
-## Develop A Transform
-
-It must implement one of the following APIs:
-- SeaTunnelTransform
-- AbstractSeaTunnelTransform
-- SingleFieldOutputTransform
-- MultipleFieldOutputTransform
-
-Add implement subclass into module `seatunnel-transforms-v2`.
-
-### Example: copy field to new field
-
-```java
-@AutoService(SeaTunnelTransform.class)
-public class CopyFieldTransform extends SingleFieldOutputTransform {
-
- private String srcField;
- private int srcFieldIndex;
- private SeaTunnelDataType srcFieldDataType;
- private String destField;
-
- @Override
- public String getPluginName() {
- return "Copy";
- }
-
- @Override
- protected void setConfig(Config pluginConfig) {
- this.srcField = pluginConfig.getString("src_field");
- this.destField = pluginConfig.getString("dest_fields");
- }
-
- @Override
- protected void setInputRowType(SeaTunnelRowType inputRowType) {
- srcFieldIndex = inputRowType.indexOf(srcField);
- srcFieldDataType = inputRowType.getFieldType(srcFieldIndex);
- }
-
- @Override
- protected String getOutputFieldName() {
- return destField;
- }
-
- @Override
- protected SeaTunnelDataType getOutputFieldDataType() {
- return srcFieldDataType;
- }
-
- @Override
- protected Object getOutputFieldValue(SeaTunnelRowAccessor inputRow) {
- return inputRow.getField(srcFieldIndex);
- }
-}
-```
-
-1. The `getPluginName` method is used to identify the transform name.
-2. The @AutoService is used to generate the
`META-INF/services/org.apache.seatunnel.api.transform.SeaTunnelTransform`
- file automatically.
-3. The `setConfig` method is used to inject user configs.
-
-## Transform Test Tool
-
-Once you add a new plugin, it is recommended to add e2e tests for it.
-We have a `seatunnel-e2e/seatunnel-transforms-v2-e2e` module to help you to do
this.
-
-For example, if you want to add an e2e test for `CopyFieldTransform`, you can
create a new test in
-`seatunnel-e2e/seatunnel-transforms-v2-e2e` module and extend the
`TestSuiteBase` class in the test.
-
-```java
-public class TestCopyFieldTransformIT extends TestSuiteBase {
-
- @TestTemplate
- public void testCopyFieldTransform(TestContainer container) {
- Container.ExecResult execResult =
container.executeJob("/copy_transform.conf");
- Assertions.assertEquals(0, execResult.getExitCode());
- }
-}
-```
-
-Once your testcase implements the `TestSuiteBase` interface and use
`@TestTemplate` annotation startup,
-it will run job to all engines, and you just need to execute the executeJob
method with your SeaTunnel configuration file,
-it will submit the SeaTunnel job.
+[Connector-v2 Contribution
Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-transform-v2/README.md)
diff --git a/docs/zh/contribution/contribute-plugin.md
b/docs/zh/contribution/contribute-plugin.md
index 514355840d..2bd46138e8 100644
--- a/docs/zh/contribution/contribute-plugin.md
+++ b/docs/zh/contribution/contribute-plugin.md
@@ -1,5 +1,5 @@
-# 贡献 Connector-v2 插件
+# 贡献 Connector-V2 插件
如果你想要贡献 Connector-V2, 可以参考下面的 Connector-V2 贡献指南。 可以帮你快速进入开发。
-[Connector-v2
贡献指南](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.md)
+[Connector-v2
贡献指南](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.zh.md)
diff --git a/docs/zh/contribution/contribute-transform-v2-guide.md
b/docs/zh/contribution/contribute-transform-v2-guide.md
index ad02b9e977..7f4847e52d 100644
--- a/docs/zh/contribution/contribute-transform-v2-guide.md
+++ b/docs/zh/contribution/contribute-transform-v2-guide.md
@@ -1,321 +1,5 @@
-# 贡献 Transform 指南
+# 贡献 Transform-V2 插件
-本文描述了如何理解、开发和贡献一个 transform。
+如果你想要贡献 Transform-V2, 可以参考下面的 Transform-V2 贡献指南。 可以帮你快速进入开发。
-我们也提供了 [Transform E2E Test](../../../seatunnel-e2e/seatunnel-transforms-v2-e2e)
-来验证 transform 的数据输入和输出。
-
-## 概念
-
-在 SeaTunnel 中你可以通过 connector 读写数据， 但如果你需要在读取数据后或者写入数据前处理数据， 你需要使用 transform。
-
-使用 transform 可以简单修改数据行和字段， 例如拆分字段、修改字段的值或者删除字段。
-
-### 类型转换
-
-Transform 从上游（source 或者 transform）获取类型输入，然后给下游（sink 或者
transform）输出新的类型，这个过程就是类型转换。
-
-案例 1：删除字段
-
-```shell
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-| A | B |
-|-----------|-----------|
-| STRING | INT |
-```
-
-案例 2：字段排序
-
-```shell
-| B | C | A |
-|-----------|-----------|-----------|
-| INT | BOOLEAN | STRING |
-
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-```
-
-案例 3：修改字段类型
-
-```shell
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | STRING | STRING |
-```
-
-案例 4：添加新的字段
-
-```shell
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-
-| A | B | C | D |
-|-----------|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN | DOUBLE |
-```
-
-### 数据转换
-
-转换类型后，Transform 会从上游（source 或者 transform）获取数据行，
使用[新的数据类型](#类型转换)编辑数据后输出到下游（sink 或者 transform）。这个过程叫数据转换。
-
-### 翻译
-
-Transform 已经从 execution engine 中解耦， 任何 transform 实现可以不需要修改和配置的适用所有引擎，
这就需要翻译层来做 transform 和 execution engine 的适配。
-
-案例：翻译数据类型和数据
-
-```shell
-原始数据:
-
-| A | B | C |
-|-----------|-----------|-----------|
-| STRING | INT | BOOLEAN |
-
-类型转换:
-
-| A | B | C |
-|-------------------|-------------------|-------------------|
-| ENGINE<STRING> | ENGINE<INT> | ENGINE<BOOLEAN> |
-
-数据转换:
-
-| A | B | C |
-|-------------------|-------------------|-------------------|
-| ENGINE<"test"> | ENGINE<1> | ENGINE<false> |
-```
-
-## 核心 APIs
-
-### SeaTunnelTransform
-
-`SeaTunnelTransform` 提供了所有主要的 API, 你可以继承它实现任何转换。
-
-1. 从上游获取数据类型。
-
-```java
-/**
- * Set the data type info of input data.
- *
- * @param inputDataType The data type info of upstream input.
- */
- void setTypeInfo(SeaTunnelDataType<T> inputDataType);
-```
-
-2. 输出新的数据类型给下游。
-
-```java
-/**
- * Get the data type of the records produced by this transform.
- *
- * @return Produced data type.
- */
-SeaTunnelDataType<T> getProducedType();
-```
-
-3. 修改输入数据并且输出新的数据到下游。
-
-```java
-/**
- * Transform input data to {@link this#getProducedType()} types data.
- *
- * @param row the data need be transform.
- * @return transformed data.
- */
-T map(T row);
-```
-
-### SingleFieldOutputTransform
-
-`SingleFieldOutputTransform` 抽象了一个单字段修改操作
-
-1. 定义输出字段
-
-```java
-/**
- * Outputs new field
- *
- * @return
- */
-protected abstract String getOutputFieldName();
-```
-
-2. 定义输出字段类型
-
-```java
-/**
- * Outputs new field datatype
- *
- * @return
- */
-protected abstract SeaTunnelDataType getOutputFieldDataType();
-```
-
-3. 定义输出字段值
-
-```java
-/**
- * Outputs new field value
- *
- * @param inputRow The inputRow of upstream input.
- * @return
- */
-protected abstract Object getOutputFieldValue(SeaTunnelRowAccessor inputRow);
-```
-
-### MultipleFieldOutputTransform
-
-`MultipleFieldOutputTransform` 抽象了多字段修改操作
-
-1. 定义多个输出的字段
-
-```java
-/**
- * Outputs new fields
- *
- * @return
- */
-protected abstract String[] getOutputFieldNames();
-```
-
-2. 定义输出字段的类型
-
-```java
-/**
- * Outputs new fields datatype
- *
- * @return
- */
-protected abstract SeaTunnelDataType[] getOutputFieldDataTypes();
-```
-
-3. 定义输出字段的值
-
-```java
-/**
- * Outputs new fields value
- *
- * @param inputRow The inputRow of upstream input.
- * @return
- */
-protected abstract Object[] getOutputFieldValues(SeaTunnelRowAccessor
inputRow);
-```
-
-### AbstractSeaTunnelTransform
-
-`AbstractSeaTunnelTransform` 抽象了数据类型和字段的修改操作
-
-1. 转换输入的行类型到新的行类型
-
-```java
-/**
- * Outputs transformed row type.
- *
- * @param inputRowType upstream input row type
- * @return
- */
-protected abstract SeaTunnelRowType transformRowType(SeaTunnelRowType
inputRowType);
-```
-
-2. 转换输入的行数据到新的行数据
-
-```java
-/**
- * Outputs transformed row data.
- *
- * @param inputRow upstream input row data
- * @return
- */
-protected abstract SeaTunnelRow transformRow(SeaTunnelRow inputRow);
-```
-
-## 开发一个 Transform
-
-Transform 必须实现下面其中一个 API:
-- SeaTunnelTransform
-- AbstractSeaTunnelTransform
-- SingleFieldOutputTransform
-- MultipleFieldOutputTransform
-
-将实现类放入模块 `seatunnel-transforms-v2`。
-
-### 案例: 拷贝字段到一个新的字段
-
-```java
-@AutoService(SeaTunnelTransform.class)
-public class CopyFieldTransform extends SingleFieldOutputTransform {
-
- private String srcField;
- private int srcFieldIndex;
- private SeaTunnelDataType srcFieldDataType;
- private String destField;
-
- @Override
- public String getPluginName() {
- return "Copy";
- }
-
- @Override
- protected void setConfig(Config pluginConfig) {
- this.srcField = pluginConfig.getString("src_field");
- this.destField = pluginConfig.getString("dest_fields");
- }
-
- @Override
- protected void setInputRowType(SeaTunnelRowType inputRowType) {
- srcFieldIndex = inputRowType.indexOf(srcField);
- srcFieldDataType = inputRowType.getFieldType(srcFieldIndex);
- }
-
- @Override
- protected String getOutputFieldName() {
- return destField;
- }
-
- @Override
- protected SeaTunnelDataType getOutputFieldDataType() {
- return srcFieldDataType;
- }
-
- @Override
- protected Object getOutputFieldValue(SeaTunnelRowAccessor inputRow) {
- return inputRow.getField(srcFieldIndex);
- }
-}
-```
-
-1. `getPluginName` 方法用来定义 transform 的名字。
-2. @AutoService 注解用来自动生成
`META-INF/services/org.apache.seatunnel.api.transform.SeaTunnelTransform` 文件
-3. `setConfig` 方法用来注入用户配置。
-
-## Transform 测试工具
-
-当你添加了一个新的插件， 推荐添加一个 e2e 测试用例来测试。
-我们有 `seatunnel-e2e/seatunnel-transforms-v2-e2e` 来帮助你实现。
-
-例如， 如果你想要添加一个 `CopyFieldTransform` 的测试用例， 你可以在
`seatunnel-e2e/seatunnel-transforms-v2-e2e`
-模块中添加一个新的测试用例， 并且在用例中继承 `TestSuiteBase` 类。
-
-```java
-public class TestCopyFieldTransformIT extends TestSuiteBase {
-
- @TestTemplate
- public void testCopyFieldTransform(TestContainer container) {
- Container.ExecResult execResult =
container.executeJob("/copy_transform.conf");
- Assertions.assertEquals(0, execResult.getExitCode());
- }
-}
-```
-
-一旦你的测试用例实现了 `TestSuiteBase` 接口， 并且添加 `@TestTemplate` 注解，它会在所有引擎运行作业，你只需要用你自己的
SeaTunnel 配置文件执行 executeJob 方法，
-它会提交 SeaTunnel 作业。
+[Connector-v2
贡献指南](https://github.com/apache/seatunnel/blob/dev/seatunnel-transform-v2/README.zh.md)
diff --git a/docs/en/contribution/contribute-transform-v2-guide.md
b/seatunnel-transforms-v2/README.md
similarity index 53%
copy from docs/en/contribution/contribute-transform-v2-guide.md
copy to seatunnel-transforms-v2/README.md
index 37837f9eeb..573bf9e36a 100644
--- a/docs/en/contribution/contribute-transform-v2-guide.md
+++ b/seatunnel-transforms-v2/README.md
@@ -2,7 +2,7 @@
This document describes how to understand, develop and contribute a transform.
-We also provide the [Transform E2E
Test](../../../seatunnel-e2e/seatunnel-transforms-v2-e2e)
+We also provide the [Transform E2E
Test](../seatunnel-e2e/seatunnel-transforms-v2-e2e)
to verify the data input and output by the transform.
## Concepts
@@ -71,7 +71,7 @@ Example 4：Add new fields
### Data Transform
After datatype transformed, Transform will receive data-row input from
upstream(source or transform),
-edit into data-row with [New Datatype](#DataType transform) and output to
downstream (sink or transform).
+edit into data-row with new datatype and output to downstream (sink or
transform).
This process is called data transform.
### Translation
@@ -103,207 +103,145 @@ Data translation:
## Core APIs
-### SeaTunnelTransform
+### TableTransformFactory
-`SeaTunnelTransform` provides all major and primary APIs, you can subclass it
to do whatever transform.
+- Used to create a factory class for transform, through which transform
instances are created using the `createTransform` method.
+- `factoryIdentifier` is used to identify the name of the current Factory,
which is also configured in the configuration file to distinguish different
transform.
+- `optionRule` is used to define the parameters supported by the current
transform. This method can be used to define the logic of the parameters, such
as which parameters are required, which are optional, which are mutually
exclusive, etc.
+ SeaTunnel will use `OptionRule` to verify the validity of the user's
configuration. Please refer to the `Option` below.
+- Make sure to add the `@AutoService(Factory.class)` annotation to
`TableTransformFactory`.
-1. Receive datatype input from upstream.
+We can receive catalog table input from upstream and the transform
configuration from `TableTransformFactoryContext`.
```java
-/**
- * Set the data type info of input data.
- *
- * @param inputDataType The data type info of upstream input.
- */
- void setTypeInfo(SeaTunnelDataType<T> inputDataType);
+ @Override
+ public TableTransform createTransform(TableTransformFactoryContext
context) {
+ return () ->
+ new SQLMultiCatalogFlatMapTransform(
+ context.getCatalogTables(), context.getOptions());
+ }
```
-2. Outputs new datatype to downstream.
+### SeaTunnelTransform
-```java
-/**
- * Get the data type of the records produced by this transform.
- *
- * @return Produced data type.
- */
-SeaTunnelDataType<T> getProducedType();
-```
+`SeaTunnelTransform` provides all major and primary APIs, you can subclass it
to do whatever transform.
-3. Edit input data and outputs new data to downstream.
+1. Get the produced catalog table list of this transform.
-```java
-/**
- * Transform input data to {@link this#getProducedType()} types data.
- *
- * @param row the data need be transform.
- * @return transformed data.
- */
-T map(T row);
-```
+ ```java
+ List<CatalogTable> getProducedCatalogTables();
+ ```
+
+ or get the produced catalog table of this transform.
+
+ ```java
+ CatalogTable getProducedCatalogTable();
+ ```
-### SingleFieldOutputTransform
+2. Handle the SchemaChangeEvent if the transform needs to change the schema.
-`SingleFieldOutputTransform` abstract single field change operator
+ ```java
+ default SchemaChangeEvent mapSchemaChangeEvent(SchemaChangeEvent
schemaChangeEvent) {
+ return schemaChangeEvent;
+ }
+ ```
-1. Define output field
+3. Edit input data and outputs new data to downstream with
`SeaTunnelMapTransform`.
-```java
-/**
- * Outputs new field
- *
- * @return
- */
-protected abstract String getOutputFieldName();
-```
+ ```java
+ T map(T row);
+ ```
+
+4. Or edit input data and outputs new data to downstream with
`SeaTunnelFlatMapTransform`.
-2. Define output field datatype
+ ```java
+ List<T> flatMap(T row);
+ ```
-```java
-/**
- * Outputs new field datatype
- *
- * @return
- */
-protected abstract SeaTunnelDataType getOutputFieldDataType();
-```
+### SingleFieldOutputTransform
-3. Define output field value
+`SingleFieldOutputTransform` abstract single field change operator
-```java
-/**
- * Outputs new field value
- *
- * @param inputRow The inputRow of upstream input.
- * @return
- */
-protected abstract Object getOutputFieldValue(SeaTunnelRowAccessor inputRow);
-```
+1. Define output field column
+
+ ```java
+ protected abstract Column getOutputColumn();
+ ```
+
+2. Define output field value
+
+ ```java
+ protected abstract Object getOutputFieldValue(SeaTunnelRowAccessor
inputRow);
+ ```
### MultipleFieldOutputTransform
`MultipleFieldOutputTransform` abstract multiple fields change operator
-1. Define output fields
+1. Define output fields column
-```java
-/**
- * Outputs new fields
- *
- * @return
- */
-protected abstract String[] getOutputFieldNames();
-```
+ ```java
+ protected abstract Column[] getOutputColumns();
+ ```
-2. Define output fields datatype
+2. Define output field values
-```java
-/**
- * Outputs new fields datatype
- *
- * @return
- */
-protected abstract SeaTunnelDataType[] getOutputFieldDataTypes();
-```
-
-3. Define output field values
-
-```java
-/**
- * Outputs new fields value
- *
- * @param inputRow The inputRow of upstream input.
- * @return
- */
-protected abstract Object[] getOutputFieldValues(SeaTunnelRowAccessor
inputRow);
-```
+ ```java
+ protected abstract Object[] getOutputFieldValues(SeaTunnelRowAccessor
inputRow);
+ ```
### AbstractSeaTunnelTransform
-`AbstractSeaTunnelTransform` abstract datatype & fields change operator
+`AbstractSeaTunnelTransform` abstract datatype, table path and fields change
operator
1. Transform input row type and outputs new row type
-
-```java
-/**
- * Outputs transformed row type.
- *
- * @param inputRowType upstream input row type
- * @return
- */
-protected abstract SeaTunnelRowType transformRowType(SeaTunnelRowType
inputRowType);
-```
+
+ ```java
+ protected abstract TableSchema transformTableSchema();
+ ```
2. Transform input row data and outputs new row data
-```java
-/**
- * Outputs transformed row data.
- *
- * @param inputRow upstream input row data
- * @return
- */
-protected abstract SeaTunnelRow transformRow(SeaTunnelRow inputRow);
-```
+ ```java
+ protected abstract R transformRow(SeaTunnelRow inputRow);
+ ```
+
+3. Transform input catalog table path and outputs new catalog table path
+
+ ```java
+ protected abstract TableIdentifier transformTableIdentifier();
+ ```
+
+### AbstractCatalogSupportFlatMapTransform & AbstractCatalogSupportMapTransform
+
+Contains the basic implementation of transform common functions and the
advanced encapsulation of transform functions.
+You can quickly implement transform development by implementing this class.
+
+### AbstractMultiCatalogFlatMapTransform & AbstractMultiCatalogMapTransform
+
+The multi-table version of AbstractCatalogSupportFlatMapTransform &
AbstractCatalogSupportMapTransform.
+Contains the encapsulation of multi-table transform. For more information
about multi-table transform, please refer to
[transform-multi-table.md](../docs/en/transform-v2/transform-multi-table.md)
## Develop A Transform
It must implement one of the following APIs:
-- SeaTunnelTransform
+- SeaTunnelMapTransform
+- SeaTunnelFlatMapTransform
- AbstractSeaTunnelTransform
+- AbstractCatalogSupportFlatMapTransform
+- AbstractCatalogSupportMapTransform
+- AbstractMultiCatalogFlatMapTransform
+- AbstractMultiCatalogMapTransform
- SingleFieldOutputTransform
- MultipleFieldOutputTransform
Add implement subclass into module `seatunnel-transforms-v2`.
-### Example: copy field to new field
-
-```java
-@AutoService(SeaTunnelTransform.class)
-public class CopyFieldTransform extends SingleFieldOutputTransform {
-
- private String srcField;
- private int srcFieldIndex;
- private SeaTunnelDataType srcFieldDataType;
- private String destField;
-
- @Override
- public String getPluginName() {
- return "Copy";
- }
-
- @Override
- protected void setConfig(Config pluginConfig) {
- this.srcField = pluginConfig.getString("src_field");
- this.destField = pluginConfig.getString("dest_fields");
- }
-
- @Override
- protected void setInputRowType(SeaTunnelRowType inputRowType) {
- srcFieldIndex = inputRowType.indexOf(srcField);
- srcFieldDataType = inputRowType.getFieldType(srcFieldIndex);
- }
-
- @Override
- protected String getOutputFieldName() {
- return destField;
- }
+Add transform info to `plugin-mapping.properties` file in seatunnel root path.
- @Override
- protected SeaTunnelDataType getOutputFieldDataType() {
- return srcFieldDataType;
- }
-
- @Override
- protected Object getOutputFieldValue(SeaTunnelRowAccessor inputRow) {
- return inputRow.getField(srcFieldIndex);
- }
-}
-```
+### Example
-1. The `getPluginName` method is used to identify the transform name.
-2. The @AutoService is used to generate the
`META-INF/services/org.apache.seatunnel.api.transform.SeaTunnelTransform`
- file automatically.
-3. The `setConfig` method is used to inject user configs.
+Please refer the [source code of
transform](src/main/java/org/apache/seatunnel/transform)
## Transform Test Tool
diff --git a/seatunnel-transforms-v2/README.zh.md
b/seatunnel-transforms-v2/README.zh.md
new file mode 100644
index 0000000000..60871cb8c8
--- /dev/null
+++ b/seatunnel-transforms-v2/README.zh.md
@@ -0,0 +1,254 @@
+# 贡献 Transform 指南
+
+本文档介绍了如何理解、开发和贡献 transform。
+
+我们还提供了 [Transform E2E 测试](../seatunnel-e2e/seatunnel-transforms-v2-e2e) 来验证
transform 的数据输入和输出。
+
+## 概念
+
+使用 SeaTunnel，你可以通过连接器读取或写入数据，但如果你需要在读取数据后或写入数据前处理数据，就需要使用 transform。
+
+使用 transform 可以对数据行或字段进行简单的编辑，例如拆分字段、修改字段值、添加或删除字段。
+
+### 数据类型 Transform
+
+Transform 从上游（源或 transform）接收数据类型输入，并将新的数据类型输出到下游（接收器或 transform）。这个过程就是数据类型转换。
+
+示例 1：删除字段
+
+```shell
+| A | B | C |
+|-----------|-----------|-----------|
+| STRING | INT | BOOLEAN |
+
+| A | B |
+|-----------|-----------|
+| STRING | INT |
+```
+
+示例 2：排序字段
+
+```shell
+| B | C | A |
+|-----------|-----------|-----------|
+| INT | BOOLEAN | STRING |
+
+| A | B | C |
+|-----------|-----------|-----------|
+| STRING | INT | BOOLEAN |
+```
+
+示例 3：更新字段数据类型
+
+```shell
+| A | B | C |
+|-----------|-----------|-----------|
+| STRING | INT | BOOLEAN |
+
+
+| A | B | C |
+|-----------|-----------|-----------|
+| STRING | STRING | STRING |
+```
+
+示例 4：添加新字段
+
+```shell
+| A | B | C |
+|-----------|-----------|-----------|
+| STRING | INT | BOOLEAN |
+
+
+| A | B | C | D |
+|-----------|-----------|-----------|-----------|
+| STRING | INT | BOOLEAN | DOUBLE |
+```
+
+### 数据 Transform
+
+在数据类型转换之后，Transform 将接收来自上游（源或 transform）的数据行输入，编辑为具有新数据类型的数据行，并将其输出到下游（接收器或
transform）。这个过程称为数据转换。
+
+### 翻译
+
+Transform 与执行引擎解耦，任何 transform 实现都可以在所有引擎中运行，而无需更改代码或配置，这需要翻译层来适配 transform
和执行引擎。
+
+示例：数据类型和数据的翻译
+
+```shell
+原始数据：
+
+| A | B | C |
+|-----------|-----------|-----------|
+| STRING | INT | BOOLEAN |
+
+数据类型翻译：
+
+| A | B | C |
+|-------------------|-------------------|-------------------|
+| ENGINE<STRING> | ENGINE<INT> | ENGINE<BOOLEAN> |
+
+数据翻译：
+
+| A | B | C |
+|-------------------|-------------------|-------------------|
+| ENGINE<"test"> | ENGINE<1> | ENGINE<false> |
+```
+
+## 核心 API
+
+### TableTransformFactory
+
+- 用于创建 transform 的工厂类，通过它可以使用 `createTransform` 方法创建 transform 实例。
+- `factoryIdentifier` 用于标识当前工厂的名称，这在配置文件中也会进行配置，以区分不同的 transform。
+- `optionRule` 用于定义当前 transform
支持的参数。此方法可以用来定义参数的逻辑，比如哪些参数是必需的，哪些是可选的，哪些是互斥的等等。SeaTunnel 会使用 `OptionRule`
来验证用户配置的有效性。请参考下面的 `Option`。
+- 确保在 `TableTransformFactory` 上添加 `@AutoService(Factory.class)` 注解。
+
+我们可以从上游接收目录表输入，并从 `TableTransformFactoryContext` 获取 transform 配置。
+
+```java
+ @Override
+ public TableTransform createTransform(TableTransformFactoryContext
context) {
+ return () ->
+ new SQLMultiCatalogFlatMapTransform(
+ context.getCatalogTables(), context.getOptions());
+ }
+```
+
+### SeaTunnelTransform
+
+`SeaTunnelTransform` 提供了所有主要和核心的 API，你可以通过继承它来实现 transform。
+
+1. 获取该 transform 产生的目录表列表。
+
+ ```java
+ List<CatalogTable> getProducedCatalogTables();
+ ```
+
+ 或者获取该 transform 产生的目录表。
+
+ ```java
+ CatalogTable getProducedCatalogTable();
+ ```
+
+2. 如果 transform 需要更改 schema，可以处理 `SchemaChangeEvent`。
+
+ ```java
+ default SchemaChangeEvent mapSchemaChangeEvent(SchemaChangeEvent
schemaChangeEvent) {
+ return schemaChangeEvent;
+ }
+ ```
+
+3. 编辑输入数据并输出新的数据到下游，使用 `SeaTunnelMapTransform`。
+
+ ```java
+ T map(T row);
+ ```
+
+4. 或者编辑输入数据并输出新的数据到下游，使用 `SeaTunnelFlatMapTransform`。
+
+ ```java
+ List<T> flatMap(T row);
+ ```
+
+### SingleFieldOutputTransform
+
+`SingleFieldOutputTransform` 抽象了单字段变换操作。
+
+1. 定义输出字段列。
+
+ ```java
+ protected abstract Column getOutputColumn();
+ ```
+
+2. 定义输出字段的值。
+
+ ```java
+ protected abstract Object getOutputFieldValue(SeaTunnelRowAccessor
inputRow);
+ ```
+
+### MultipleFieldOutputTransform
+
+`MultipleFieldOutputTransform` 抽象了多字段变换操作。
+
+1. 定义输出字段列。
+
+ ```java
+ protected abstract Column[] getOutputColumns();
+ ```
+
+2. 定义输出字段的值。
+
+ ```java
+ protected abstract Object[] getOutputFieldValues(SeaTunnelRowAccessor
inputRow);
+ ```
+
+### AbstractSeaTunnelTransform
+
+`AbstractSeaTunnelTransform` 抽象了数据类型、表路径和字段变换操作。
+
+1. 转换输入行类型并输出新行类型。
+
+ ```java
+ protected abstract TableSchema transformTableSchema();
+ ```
+
+2. 转换输入行数据并输出新数据行。
+
+ ```java
+ protected abstract R transformRow(SeaTunnelRow inputRow);
+ ```
+
+3. 转换输入目录表路径并输出新目录表路径。
+
+ ```java
+ protected abstract TableIdentifier transformTableIdentifier();
+ ```
+
+### AbstractCatalogSupportFlatMapTransform & AbstractCatalogSupportMapTransform
+
+包含了 transform 公共功能的基本实现，以及 transform 功能的高级封装。你可以通过实现这些类来快速开发 transform。
+
+### AbstractMultiCatalogFlatMapTransform & AbstractMultiCatalogMapTransform
+
+`AbstractCatalogSupportFlatMapTransform` 和
`AbstractCatalogSupportMapTransform` 的多表版本。包含了多表 transform 的封装。有关多表 transform
的更多信息，请参阅
[transform-multi-table.md](../docs/zh/transform-v2/transform-multi-table.md)
+
+## 开发一个 Transform
+
+你必须实现以下 API 中的一个：
+- SeaTunnelMapTransform
+- SeaTunnelFlatMapTransform
+- AbstractSeaTunnelTransform
+- AbstractCatalogSupportFlatMapTransform
+- AbstractCatalogSupportMapTransform
+- AbstractMultiCatalogFlatMapTransform
+- AbstractMultiCatalogMapTransform
+- SingleFieldOutputTransform
+- MultipleFieldOutputTransform
+
+将实现的子类添加到模块 `seatunnel-transforms-v2` 中。
+
+在 SeaTunnel 根路径的 `plugin-mapping.properties` 文件中添加 transform 信息。
+
+### 示例
+
+请参考 [transform 的源代码](src/main/java/org/apache/seatunnel/transform)
+
+## Transform 测试工具
+
+一旦你添加了一个新的插件，建议为它添加 e2e 测试。
+我们有一个 `seatunnel-e2e/seatunnel-transforms-v2-e2e` 模块来帮助你完成这项工作。
+
+例如，如果你想为 `CopyFieldTransform` 添加 e2e 测试，可以在
`seatunnel-e2e/seatunnel-transforms-v2-e2e` 模块中创建一个新测试，并在测试中扩展 `TestSuiteBase`
类。
+
+```java
+public class TestCopyFieldTransformIT extends TestSuiteBase {
+
+ @TestTemplate
+ public void testCopyFieldTransform(TestContainer container) {
+ Container.ExecResult execResult =
container.executeJob("/copy_transform.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ }
+}
+```
+
+一旦你的测试用例实现了 `TestSuiteBase` 接口并使用 `@TestTemplate` 注解启动，它将针对所有引擎运行作业，你只需要执行
`executeJob` 方法并提供你的 SeaTunnel 配置文件，它将提交 SeaTunnel 作业。
\ No newline at end of file
