This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 23aa0a2a75b [Doc] Describe new partition lifecycle behaviours (#2992)
23aa0a2a75b is described below
commit 23aa0a2a75b2cbb45779f8bd2bed31337dcd6043
Author: zclllyybb <[email protected]>
AuthorDate: Wed Oct 22 11:43:54 2025 +0800
[Doc] Describe new partition lifecycle behaviours (#2992)
doc of https://github.com/apache/doris/pull/57060,
https://github.com/apache/doris/pull/57013
and fix of https://github.com/apache/doris-website/pull/2970
## Versions
- [x] dev
- [x] 3.x
- [ ] 2.1
- [ ] 2.0
## Languages
- [x] Chinese
- [x] English
## Docs Checklist
- [ ] Checked by AI
- [ ] Test Cases Built
---
.../data-partitioning/auto-partitioning.md | 46 ++++++++++++--------
.../data-partitioning/auto-partitioning.md | 49 ++++++++++++----------
.../data-partitioning/auto-partitioning.md | 4 +-
.../data-partitioning/auto-partitioning.md | 4 +-
4 files changed, 59 insertions(+), 44 deletions(-)
diff --git a/docs/table-design/data-partitioning/auto-partitioning.md
b/docs/table-design/data-partitioning/auto-partitioning.md
index 416ac994853..0ac96d7f00c 100644
--- a/docs/table-design/data-partitioning/auto-partitioning.md
+++ b/docs/table-design/data-partitioning/auto-partitioning.md
@@ -63,7 +63,7 @@ When creating a table, use the following syntax to populate
the `partitions_defi
1. AUTO RANGE PARTITION:
```sql
- AUTO PARTITION BY RANGE(<partition_expr>)
+ [AUTO] PARTITION BY RANGE(<partition_expr>)
<origin_partitions_definition>
```
@@ -98,6 +98,8 @@ When creating a table, use the following syntax to populate
the `partitions_defi
);
```
+ In AUTO RANGE PARTITION, the `AUTO` keyword can be omitted, and it still
conveys the meaning of automatic partitioning.
+
2. AUTO LIST PARTITION
```sql
@@ -228,33 +230,43 @@ Doris supports both Auto and Dynamic Partition. In this
case, both functions are
There is no conflict between the two syntaxes, just set the corresponding
clauses/attributes at the same time. Please note that it is uncertain whether
the partition in current period is created by Auto Partition or Dynamic
Partition. Different creation methods will lead to different naming formats for
the partitions.
-### Best Practice
+## Lifecycle Management
+
+:::info
+Doris supports the simultaneous use of automatic partitioning and dynamic
partitioning for lifecycle management, but it is now not recommended.
+:::
+
+In the AUTO RANGE PARTITION table, the property `partition.retention_count` is
supported, which accepts a positive integer value as a parameter (denoted as
`N`), indicating that **only the top `N` historical partitions with the largest
partition values** are retained among all historical partitions. All current
and future partitions are retained. Specifically:
-In scenarios where you need to set a limit on the partition lifecycle, you can
**disable the creation of Dynamic Partition, leaving the creation of partitions
to be completed by Auto Partition**, and complete the management of the
partition lifecycle through the Dynamic Partition's function of dynamically
reclaiming partitions:
+- Since RANGE partitions are always non-overlapping, `partition A's value >
partition B's value` is equivalent to `partition A's lower bound value >
partition B's upper bound value` which is equivalent to `partition A's upper
bound value > partition B's upper bound value`.
+- Historical partitions refer to **partitions whose upper bound is <= current
time**.
+- Current and future partitions refer to **partitions whose lower bound is >=
current time**.
+
+For example:
```sql
-create table auto_dynamic(
- k0 datetime(6) NOT NULL
-)
-auto partition by range (date_trunc(k0, 'year'))
-(
+create table auto_recycle(
+ k0 datetime(6) not null
)
-DISTRIBUTED BY HASH(`k0`) BUCKETS 2
+AUTO PARTITION BY RANGE (date_trunc(k0, 'day')) ()
+DISTRIBUTED BY HASH(`k0`) BUCKETS 1
properties(
- "dynamic_partition.enable" = "true",
- "dynamic_partition.prefix" = "p",
- "dynamic_partition.start" = "-50",
- "dynamic_partition.end" = "0", --- Dynamic Partition No Partition Creation
- "dynamic_partition.time_unit" = "year",
- "replication_num" = "1"
+ "partition.retention_count" = "3"
);
```
-This way we have both the flexibility of Auto Partition and consistency in
partition names.
+This represents keeping only the top 3 partitions with the largest date values
in the history. Assuming the current date is `2025-10-21`, and inserting data
for each day from `2025-10-16` to `2025-10-23`, after one recycling, the
remaining partitions are as follows:
+
+- p20251018000000
+- p20251019000000
+- p20251020000000 (The following partition and above: Only keep three
historical partitions)
+- p20251021000000 (The following partition and below: The current and future
partitions are not affected)
+- p20251022000000
+- p20251023000000
## Conjunct with Auto Bucket
-Only AUTO RANGE PARTITION can be used together with the [Auto
Bucket](./data-bucketing.md#auto-setting-bucket-number) feature. When using
this feature, Doris assumes that the data import is incremental in time order,
and each import only involves one partition. In other words, this usage is only
recommended for tables that are incrementally imported daily.
+Only AUTO RANGE PARTITION can be used together with the [Auto
Bucket](./data-bucketing.md#auto-setting-bucket-number) feature. When using
this feature, Doris assumes that the data import is incremental in time order,
and each import only involves one partition. In other words, this usage is only
recommended for tables that are incrementally imported batch by batch.
:::warning Note!
If the data import method does not conform to the above pattern, and both auto
partitioning and auto bucketing are used at the same time, there is a
possibility that the number of buckets in the new partition is extremely
unreasonable, which may greatly affect query performance.
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/auto-partitioning.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/auto-partitioning.md
index 6d9d30896e9..db479c72b70 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/auto-partitioning.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/auto-partitioning.md
@@ -61,7 +61,7 @@ PROPERTIES (
1. AUTO RANGE PARTITION:
```sql
- AUTO PARTITION BY RANGE(<partition_expr>)
+ [AUTO] PARTITION BY RANGE(<partition_expr>)
<origin_partitions_definition>
```
@@ -96,6 +96,8 @@ PROPERTIES (
);
```
+ 在 AUTO RANGE PARTITION 中，`AUTO` 关键字可以省略，仍然表达自动分区含义。
+
2. AUTO LIST PARTITION
```sql
@@ -217,42 +219,43 @@ show partitions from `DAILY_TRADE_VALUE`;
经过自动分区功能所创建的 PARTITION，与手动创建的 PARTITION 具有完全一致的功能性质。
-## 与动态分区联用
-
-Doris 支持自动分区和动态分区同时使用。此时，二者的功能都生效：
+## 生命周期管理
-1. 自动分区将会自动在数据导入过程中按需创建分区；
-2. 动态分区将会自动创建、回收、转储分区。
+:::info
+Doris 支持同时使用自动分区与动态分区实现生命周期管理，现已不推荐。
+:::
-二者语法功能不存在冲突，同时设置对应的子句/属性即可。请注意，当前时间所在的分区由自动分区还是动态分区创建，是不确定的。不同创建方式会导致分区的名称格式不同。
+在 AUTO RANGE PARTITION 表中，支持属性 `partition.retention_count`，接受一个正整数值作为参数（此处记为
`N`），表示在所有历史分区中，**只保留分区值最大的 `N` 个历史分区**。对于当前及未来分区，全部保留。具体来说：
-### 最佳实践
+- 由于 RANGE 分区一定不相交，`分区 A 的值 > 分区 B 的值` 等价于 `分区 A 的下界值 > 分区 A 的上界值` 等价于 `分区 A
的上界值 > 分区 A 的上界值`。
+- 历史分区指的是**分区上界 <= 当前时间**的分区。
+- 当前及未来分区指的是**分区下界 >= 当前时间**的分区。
-需要对分区生命周期设限的场景，可以**将 Dynamic Partition 的创建功能关闭，创建分区完全交由 Auto Partition 完成**，通过
Dynamic Partition 动态回收分区的功能完成分区生命周期的管理：
+例如：
```sql
-create table auto_dynamic(
- k0 datetime(6) NOT NULL
+create table auto_recycle(
+ k0 datetime(6) not null
)
-auto partition by range (date_trunc(k0, 'year'))
-(
-)
-DISTRIBUTED BY HASH(`k0`) BUCKETS 2
+AUTO PARTITION BY RANGE (date_trunc(k0, 'day')) ()
+DISTRIBUTED BY HASH(`k0`) BUCKETS 1
properties(
- "dynamic_partition.enable" = "true",
- "dynamic_partition.prefix" = "p",
- "dynamic_partition.start" = "-50",
- "dynamic_partition.end" = "0", --- Dynamic Partition 不创建分区
- "dynamic_partition.time_unit" = "year",
- "replication_num" = "1"
+ "partition.retention_count" = "3"
);
```
-这样我们同时具有了 Auto Partition 的灵活性，且分区名上保持了一致性。
+这代表只保留历史分区日期值最大的 3 个分区。假设当前日期为 `2025-10-21`，插入 `2025-10-16` 至 `2025-10-23`
中每一天的数据。则经过一次回收，剩余如下 6 个分区：
+
+- p20251018000000
+- p20251019000000
+- p20251020000000（该分区及以上：只保留三个历史分区）
+- p20251021000000（该分区及以下：当前及未来分区不受影响）
+- p20251022000000
+- p20251023000000
## 与自动分桶联用
-只有 AUTO RANGE PARTITION
可以同时使用[自动分桶](./data-bucketing.md#自动设置分桶数)功能。使用此功能时，Doris
将假设表的数据导入是按照时间顺序增量的，每次导入仅涉及一个分区。即是说，这种用法仅推荐用于按日增量导入的表。
+只有 AUTO RANGE PARTITION
可以同时使用[自动分桶](./data-bucketing.md#自动设置分桶数)功能。使用此功能时，Doris
将假设表的数据导入是按照时间顺序增量的，每次导入仅涉及一个分区。即是说，这种用法仅推荐用于逐批次增量导入的表。
:::warning 注意！
如果数据导入方式不符合上述范式，且同时使用了自动分区和自动分桶，存在新分区的分桶数极不合理的可能，较大影响查询性能。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
index 62b80bf71ef..dd661c22da0 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
@@ -256,10 +256,10 @@ properties(
## 与自动分桶联用
:::note
-这个功能从 Doris 3.1.1 开始正常工作
+这个功能从 Doris 3.1.2 开始正常工作
:::
-只有 AUTO RANGE PARTITION
可以同时使用[自动分桶](./data-bucketing.md#自动设置分桶数)功能。使用此功能时，Doris
将假设表的数据导入是按照时间顺序增量的，每次导入仅涉及一个分区。即是说，这种用法仅推荐用于按日增量导入的表。
+只有 AUTO RANGE PARTITION
可以同时使用[自动分桶](./data-bucketing.md#自动设置分桶数)功能。使用此功能时，Doris
将假设表的数据导入是按照时间顺序增量的，每次导入仅涉及一个分区。即是说，这种用法仅推荐用于逐批次增量导入的表。
:::warning 注意！
如果数据导入方式不符合上述范式，且同时使用了自动分区和自动分桶，存在新分区的分桶数极不合理的可能，较大影响查询性能。
diff --git
a/versioned_docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
b/versioned_docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
index 49f8b013ca7..9aad9e0af8c 100644
---
a/versioned_docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
+++
b/versioned_docs/version-3.x/table-design/data-partitioning/auto-partitioning.md
@@ -263,10 +263,10 @@ In some early versions prior to 2.1.7, this feature was
not disabled but not rec
## Conjunct with Auto Bucket
:::note
-This feature has been working normally since Doris 3.1.1
+This feature has been working normally since Doris 3.1.2
:::
-Only AUTO RANGE PARTITION can be used together with the [Auto
Bucket](./data-bucketing.md#auto-setting-bucket-number) feature. When using
this feature, Doris assumes that the data import is incremental in time order,
and each import only involves one partition. In other words, this usage is only
recommended for tables that are incrementally imported daily.
+Only AUTO RANGE PARTITION can be used together with the [Auto
Bucket](./data-bucketing.md#auto-setting-bucket-number) feature. When using
this feature, Doris assumes that the data import is incremental in time order,
and each import only involves one partition. In other words, this usage is only
recommended for tables that are incrementally imported batch by batch.
:::warning Note!
If the data import method does not conform to the above pattern, and both auto
partitioning and auto bucketing are used at the same time, there is a
possibility that the number of buckets in the new partition is extremely
unreasonable, which may greatly affect query performance.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
