[GitHub] [flink] luoyuxia commented on a diff in pull request #23116: [FLINK-32581][docs] Add docs for atomic CTAS and RTAS
luoyuxia commented on code in PR #23116:
URL: https://github.com/apache/flink/pull/23116#discussion_r1288417299
##
docs/content/docs/dev/table/sourcesSinks.md:
##
@@ -344,6 +344,11 @@ that a sink can still work on common data structures and
perform a conversion at
and consuming them to achieve the purpose of row(s) update.
+
+{{< gh_link
file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsStaging.java"
name="SupportsStaging" >}}
+Enables to support atomic semantic for CTAS(CREATE TABLE AS
SELECT) or RTAS([CREATE OR] REPLACE TABLE AS SELECT) in a
DynamicTableSink. The table sink is responsible for returning
StagedTable object that provide atomic semantics.
Review Comment:
```suggestion
Enables to support atomic semantic for CTAS(CREATE TABLE AS
SELECT) or RTAS([CREATE OR] REPLACE TABLE AS SELECT) in a
DynamicTableSink. The table sink is responsible for returning
StagedTable object that provides atomic semantics.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] luoyuxia commented on a diff in pull request #23116: [FLINK-32581][docs] Add docs for atomic CTAS and RTAS
luoyuxia commented on code in PR #23116:
URL: https://github.com/apache/flink/pull/23116#discussion_r1284307429
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -554,7 +554,13 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
* 暂不支持创建分区表。
* 暂不支持主键约束。
-**注意** 目前，CTAS 创建的目标表是非原子性的，如果在向表中插入数据时发生错误，该表不会被自动删除。
+**注意** 默认情况下，CTAS 是非原子性的，这意味着如果在向表中插入数据时发生错误，该表不会被自动删除。
+
+ 原子性
+
+如果要启用 CTAS 的原子性，则应确保：
+* Sink 已经实现了 CTAS 的原子性语义。通过阅读 Sink 的文档可以知道其是否已经支持了原子性语义。如果开发者想要实现原子性语义，请参考文档
[SupportsStaging]({{< ref "docs/dev/table/sourcesSinks" >}}#sink-abilities)。
Review Comment:
```suggestion
* 对应的 Connector sink 已经实现了 CTAS 的原子性语义，你可能需要阅读对应 Connector
的文档看是否已经支持了原子性语义。如果开发者想要实现原子性语义，请参考文档 [SupportsStaging]({{< ref
"docs/dev/table/sourcesSinks" >}}#sink-abilities)。
```
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -554,7 +554,13 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
* 暂不支持创建分区表。
* 暂不支持主键约束。
-**注意** 目前，CTAS 创建的目标表是非原子性的，如果在向表中插入数据时发生错误，该表不会被自动删除。
+**注意** 默认情况下，CTAS 是非原子性的，这意味着如果在向表中插入数据时发生错误，该表不会被自动删除。
+
+ 原子性
+
+如果要启用 CTAS 的原子性，则应确保：
+* Sink 已经实现了 CTAS 的原子性语义。通过阅读 Sink 的文档可以知道其是否已经支持了原子性语义。如果开发者想要实现原子性语义，请参考文档
[SupportsStaging]({{< ref "docs/dev/table/sourcesSinks" >}}#sink-abilities)。
+* 设置配置项 `table.rtas-ctas.atomicity-enabled` 为 `true`。
Review Comment:
Add a link for `table.rtas-ctas.atomicity-enabled`
##
docs/content/docs/dev/table/sourcesSinks.md:
##
@@ -344,6 +344,11 @@ that a sink can still work on common data structures and
perform a conversion at
and consuming them to achieve the purpose of row(s) update.
+
+{{< gh_link
file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsStaging.java"
name="SupportsStaging" >}}
+Enables to supports atomic semantic for CTAS or RTAS in a
DynamicTableSink. The table sink is responsible for return
`StagedTable` object that provide atomic semantics.
Review Comment:
```suggestion
Enables to support atomic semantic for CTAS(CREATE TABLE AS) or
RTAS([CREATE OR] REPLACE TABLE AS SELECT) in a DynamicTableSink.
The table sink is responsible for returning StagedTable object
that provide atomic semantics.
```
##
docs/content.zh/docs/dev/table/sourcesSinks.md:
##
@@ -268,15 +268,19 @@ Flink 会对工厂类逐个进行检查，确保其“标识符”是全局唯
{{< gh_link
file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsDeletePushDown.java"
name="SupportsDeletePushDown" >}}
-支持将 DELETE 语句中的过滤条件下推到
DynamicTableSink，sink 端可以直接根据过滤条件来删除数据。
+支持将 DELETE 语句中的过滤条件下推到
DynamicTableSink，sink 端可以直接根据过滤条件来删除数据。
{{< gh_link
file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsRowLevelDelete.java"
name="SupportsRowLevelDelete" >}}
-支持 DynamicTableSink 根据行级别的变更来删除已有的数据。该接口的实现者需要告诉
Planner 如何产生这些行变更，并且需要消费这些行变更从而达到删除数据的目的。
+支持 DynamicTableSink 根据行级别的变更来删除已有的数据。该接口的实现者需要告诉
Planner 如何产生这些行变更，并且需要消费这些行变更从而达到删除数据的目的。
{{< gh_link
file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsRowLevelUpdate.java"
name="SupportsRowLevelUpdate" >}}
-支持 DynamicTableSink 根据行级别的变更来更新已有的数据。该接口的实现者需要告诉
Planner 如何产生这些行变更，并且需要消费这些行变更从而达到更新数据的目的。
+支持 DynamicTableSink 根据行级别的变更来更新已有的数据。该接口的实现者需要告诉
Planner 如何产生这些行变更，并且需要消费这些行变更从而达到更新数据的目的。
+
+
+{{< gh_link
file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/sink/abilities/SupportsStaging.java"
name="SupportsStaging" >}}
+支持 DynamicTableSink 提供 CTAS 或 RTAS
的原子性语义。该接口的实现者需要返回一个提供原子性语义实现的 `StagedTable` 对象。
Review Comment:
```suggestion
支持 DynamicTableSink 提供 CTAS(CREATE TABLE AS) 或
RTAS([CREATE OR] REPLACE TABLE AS SELECT) 的原子性语义。该接口的实现者需要返回一个提供原子性语义实现的
StagedTable 对象。
```
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -608,6 +614,14 @@ INSERT INTO my_rtas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
* 暂不支持创建分区表。
* 暂不支持主键约束。
+**注意** 默认情况下，RTAS 是非原子性的，这意味着如果在向表中插入数据时发生错误，该表不会被自动删除或还原成原来的表。
Review Comment:
nit
```suggestion
**注意:** 默认情况下，RTAS 是非原子性的，这意味着如果在向表中插入数据时发生错误，该表不会被自动删除或还原成原来的表。
```
##
docs/content.zh/docs/dev/table/sql/create.md:
##
@@ -554,7 +554,13 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM
source_table WHERE mod(id, 1
* 暂不支持创建分区表。
* 暂不支持主键约束。
-**注意** 目前，CTAS 创建的目标表是非原子性的，如果在向表中插入数据时发生错误，该表不会被自动删除。
+**注意** 默认情况下，CTAS 是非原子性的，这意味着如果在向表中插入数据时发生错误，该表不会被自动删除。
Review Comment:
nit
```suggestion
**注意:** 默认情况下，CTAS 是非原子性的，这意味着如果在向表中插入数据时发生错误，该表不会被自动删除。
```
##
docs/content/docs/dev/table/sql/create.md:
##
@@ -609,6 +615,14
[GitHub] [flink] luoyuxia commented on a diff in pull request #23116: [FLINK-32581][docs] Add docs for atomic CTAS and RTAS
luoyuxia commented on code in PR #23116: URL: https://github.com/apache/flink/pull/23116#discussion_r1283096835 ## docs/content/docs/dev/table/sql/create.md: ## @@ -554,7 +554,14 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM source_table WHERE mod(id, 1 * Does not support creating partitioned table yet. * Does not support specifying primary key constraints yet. -**Note** The target table created by CTAS is non-atomic currently, the table won't be dropped automatically if occur errors while inserting data into the table. +**Note** By default, the target table created by CTAS is non-atomic, the table won't be dropped automatically if occur errors while inserting data into the table. + + Atomicity + +If you want to implement atomicity CTAS, then you need to fulfill the following conditions at the same time: Review Comment: Implementment shouldn't be in here as it's doc for users not for devs. ```suggestion If you want to enable atomicity for CTAS, then you shoud make sure: ``` ## docs/content.zh/docs/dev/table/sql/create.md: ## @@ -554,7 +554,14 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM source_table WHERE mod(id, 1 * 暂不支持创建分区表。 * 暂不支持主键约束。 -**注意** 目前，CTAS 创建的目标表是非原子性的，如果在向表中插入数据时发生错误，该表不会被自动删除。 +**注意** 默认情况下，CTAS 创建的目标表是非原子性的，如果在向表中插入数据时发生错误，该表不会被自动删除。 Review Comment: The comments for en doc is also suitable for the zh doc. ## docs/content/docs/dev/table/sql/create.md: ## @@ -609,6 +616,15 @@ INSERT INTO my_rtas_table SELECT id, name, age FROM source_table WHERE mod(id, 1 * Does not support creating partitioned table yet. * Does not support specifying primary key constraints yet. +**Note** By default, the target table replaced or created by RTAS is non-atomic, the table won't be dropped or restore to origin automatically if occur errors while inserting data into the table. + +### Atomicity + +If you want to implement atomicity RTAS, then you need to fulfill the following conditions at the same time: Review Comment: dito. ## docs/content/docs/dev/table/sql/create.md: ## @@ -554,7 +554,14 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM source_table WHERE mod(id, 1 * Does not support creating partitioned table yet. * Does not support specifying primary key constraints yet. -**Note** The target table created by CTAS is non-atomic currently, the table won't be dropped automatically if occur errors while inserting data into the table. +**Note** By default, the target table created by CTAS is non-atomic, the table won't be dropped automatically if occur errors while inserting data into the table. Review Comment: ```suggestion **Note** By default, CTAS is non-atomic which means the table created won't be dropped automatically if occur errors while inserting data into the table. ``` ## docs/content/docs/dev/table/sql/create.md: ## @@ -554,7 +554,14 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM source_table WHERE mod(id, 1 * Does not support creating partitioned table yet. * Does not support specifying primary key constraints yet. -**Note** The target table created by CTAS is non-atomic currently, the table won't be dropped automatically if occur errors while inserting data into the table. +**Note** By default, the target table created by CTAS is non-atomic, the table won't be dropped automatically if occur errors while inserting data into the table. + + Atomicity + +If you want to implement atomicity CTAS, then you need to fulfill the following conditions at the same time: +* Set option `table.rtas-ctas.atomicity-enabled` to `true`. +* `DynamicTableSink` implements the `SupportsStaging` interface. Review Comment: Then remove these two lines. ## docs/content/docs/dev/table/sql/create.md: ## @@ -554,7 +554,14 @@ INSERT INTO my_ctas_table SELECT id, name, age FROM source_table WHERE mod(id, 1 * Does not support creating partitioned table yet. * Does not support specifying primary key constraints yet. -**Note** The target table created by CTAS is non-atomic currently, the table won't be dropped automatically if occur errors while inserting data into the table. +**Note** By default, the target table created by CTAS is non-atomic, the table won't be dropped automatically if occur errors while inserting data into the table. + + Atomicity + +If you want to implement atomicity CTAS, then you need to fulfill the following conditions at the same time: +* Set option `table.rtas-ctas.atomicity-enabled` to `true`. Review Comment: Please make the following content be the first point: ``` * The sink has implemented the atomicity semantics for CTAS. You may refer to the corresponding doc for the sink to know the atomicity semantics is available or not. For devs who want to implement the atomicity semantics, please refer to doc [SupportsStaging](link). ``` Then add
