This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new a4a45d728a2 [doc](compaction) Add description of the max-version limit
under the time-series compaction policy (#2622)
a4a45d728a2 is described below
commit a4a45d728a2d4d005db1bfdb16ac240165bbeeec
Author: Luwei <[email protected]>
AuthorDate: Sun Aug 10 22:24:34 2025 +0800
[doc](compaction) Add description of the max-version limit under the
time-series compaction policy (#2622)
- [x] dev
- [x] 3.0
- [x] 2.1
- [ ] 2.0
---
docs/admin-manual/config/be-config.md | 6 ++++++
docs/log-storage-analysis.md | 1 -
.../current/admin-manual/config/be-config.md | 6 ++++++
.../docusaurus-plugin-content-docs/current/log-storage-analysis.md | 1 -
.../version-2.1/admin-manual/config/be-config.md | 6 ++++++
.../version-2.1/log-storage-analysis.md | 2 +-
.../version-3.0/admin-manual/config/be-config.md | 6 ++++++
.../version-3.0/log-storage-analysis.md | 2 +-
versioned_docs/version-2.1/admin-manual/config/be-config.md | 6 ++++++
versioned_docs/version-2.1/log-storage-analysis.md | 2 +-
versioned_docs/version-3.0/admin-manual/config/be-config.md | 6 ++++++
versioned_docs/version-3.0/log-storage-analysis.md | 2 +-
12 files changed, 40 insertions(+), 6 deletions(-)
diff --git a/docs/admin-manual/config/be-config.md
b/docs/admin-manual/config/be-config.md
index 85b7f26863c..cb3d588c559 100644
--- a/docs/admin-manual/config/be-config.md
+++ b/docs/admin-manual/config/be-config.md
@@ -1359,3 +1359,9 @@ Default: true for cloud mode, false for non-cloud mode.
Default: [{"path":"${DORIS_HOME}/file_cache"}]
* Description: The disk paths and other parameters used for file cache,
represented as an array, with one entry for each disk. The `path` specifies the
disk path, and `total_size` limits the size of the cache; -1 or 0 will use the
entire disk space.
* format:
[{"path":"/path/to/file_cache","total_size":21474836480,{"path":"/path/to/file_cache2","total_size":21474836480}]
+
+#### `time_series_max_tablet_version_num`
+
+* Type: int
+* Description: Limit the number of versions of a single tablet under the
time-series compaction policy. It is used to prevent a large number of version
accumulation problems caused by too frequent load or untimely compaction. When
the limit is exceeded, the load task will be rejected.
+* Default value: 20000
diff --git a/docs/log-storage-analysis.md b/docs/log-storage-analysis.md
index 01b54ffc15e..4fc0dfe05d9 100644
--- a/docs/log-storage-analysis.md
+++ b/docs/log-storage-analysis.md
@@ -188,7 +188,6 @@ You can find BE configuration fields in `be/conf/be.conf`.
Refer to the followin
| - | `enable_file_cache = true` |
Enable file caching. |
| - | `file_cache_path = [{"path": "/mnt/datadisk0/file_cache",
"total_size":53687091200, "query_limit": "10737418240"},{"path":
"/mnt/datadisk1/file_cache", "total_size":53687091200,"query_limit":
"10737418240"}]` | Configure the cache path and related settings for cold data
with the following specific configurations:<br/>`path`: cache
path<br/>`total_size`: total size of the cache path in bytes, where 53687091200
bytes equals 50 GB<br/>`query_limit`: maximum amount of data tha [...]
| Write | `write_buffer_size = 1073741824` |
Increase the file size of the write buffer to reduce small files and random I/O
operations, improving performance. |
-| - | `max_tablet_version_num = 20000` |
In coordination with the time_series compaction strategy for table creation,
allow more versions to remain temporarily unmerged |
| Compaction | `max_cumu_compaction_threads = 8` |
Set to CPU core count / 4, indicating that 1/4 of CPU resources are used for
writing, 1/4 for background compaction, and 2/1 for queries and other
operations. |
| - | `inverted_index_compaction_enable = true` |
Enable inverted index compaction to reduce CPU consumption during compaction. |
| - | `enable_segcompaction = false` `enable_ordered_data_compaction
= false` | Disable two compaction features that are unnecessary for log
scenarios. |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md
index 31f20980ec3..b5061831ff1 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md
@@ -1338,3 +1338,9 @@ load tablets from header failed, failed tablets size:
xxx, path=xxx
* 描述：用于文件缓存的磁盘路径和其他参数，以数组形式表示，每个磁盘一个条目。`path` 指定磁盘路径，`total_size` 限制缓存的大小；-1 或
0 将使用整个磁盘空间。
* 格式：
[{"path":"/path/to/file_cache","total_size":21474836480},{"path":"/path/to/file_cache2","total_size":21474836480}]
+
+#### `time_series_max_tablet_version_num`
+
+* 类型：int
+* 描述：限制time series compaction 策略的 tablet 最大 version 的数量。用于防止导入过于频繁，或
compaction 不及时导致的大量 version 堆积问题。当超过限制后，导入任务将被拒绝。
+* 默认值：20000
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/log-storage-analysis.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/log-storage-analysis.md
index f992ca7da75..c260e064fc7 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/log-storage-analysis.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/log-storage-analysis.md
@@ -164,7 +164,6 @@ Apache Doris 对 Flexible Schema 的日志数据提供了几个方面的支持
| - | `enable_file_cache = true` |
开启文件缓存。 |
| - | `file_cache_path = [{"path": "/mnt/datadisk0/file_cache",
"total_size":53687091200, "query_limit": "10737418240"},{"path":
"/mnt/datadisk1/file_cache", "total_size":53687091200,"query_limit":
"10737418240"}]` | 配置冷数据的缓存路径和相关设置，具体配置说明如下：<br />`path`：缓存路径<br
/>`total_size`：该缓存路径的总大小，单位为字节，53687091200 字节等于 50 GB<br
/>`query_limit`：单次查询可以从缓存路径中查询的最大数据量，单位为字节，10737418240 字节等于 10 GB |
| 写入 | `write_buffer_size = 1073741824` |
增加写入缓冲区（buffer）的文件大小，减少小文件和随机 I/O 操作，提升性能。 |
-| - | `max_tablet_version_num = 20000` |
配合建表的 time_series compaction 策略，允许更多版本暂时未合并。 |
| Compaction | `max_cumu_compaction_threads = 8` |
设置为 CPU 核数 / 4，意味着 CPU 资源的 1/4 用于写入，1/4 用于后台 Compaction，2/1 留给查询和其他操作。 |
| - | `inverted_index_compaction_enable = true` |
开启索引合并（index compaction），减少 Compaction 时的 CPU 消耗。 |
| - | `enable_segcompaction = false` `enable_ordered_data_compaction
= false` | 关闭日志场景不需要的两个 Compaction 功能。 |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/config/be-config.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/config/be-config.md
index bc3e2a9bdbe..ff6205f47ea 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/config/be-config.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/config/be-config.md
@@ -1483,3 +1483,9 @@ load tablets from header failed, failed tablets size:
xxx, path=xxx
* 描述：Doris 自带的时区数据库。如果系统目录下未找到时区文件，则启用该目录下的数据。
* 默认值："${DORIS_HOME}/zoneinfo"
+
+#### `time_series_max_tablet_version_num`
+
+* 类型：int
+* 描述：限制time series compaction 策略的 tablet 最大 version 的数量。用于防止导入过于频繁，或
compaction 不及时导致的大量 version 堆积问题。当超过限制后，导入任务将被拒绝。从2.1.11版本开始支持这个参数
+* 默认值：20000
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/log-storage-analysis.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/log-storage-analysis.md
index b0715863aa7..5947600c3bd 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/log-storage-analysis.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/log-storage-analysis.md
@@ -82,7 +82,7 @@
| - | `enable_file_cache = true` |
开启文件缓存。 |
| - | `file_cache_path = [{"path": "/mnt/datadisk0/file_cache",
"total_size":53687091200, "query_limit": "10737418240"},{"path":
"/mnt/datadisk1/file_cache", "total_size":53687091200,"query_limit":
"10737418240"}]` | 配置冷数据的缓存路径和相关设置，具体配置说明如下：<br />`path`：缓存路径<br
/>`total_size`：该缓存路径的总大小，单位为字节，53687091200 字节等于 50 GB<br
/>`query_limit`：单次查询可以从缓存路径中查询的最大数据量，单位为字节，10737418240 字节等于 10 GB |
| 写入 | `write_buffer_size = 1073741824` |
增加写入缓冲区（buffer）的文件大小，减少小文件和随机 I/O 操作，提升性能。 |
-| - | `max_tablet_version_num = 20000` |
配合建表的 time_series compaction 策略，允许更多版本暂时未合并。 |
+| - | `max_tablet_version_num = 20000` |
配合建表的 time_series compaction 策略，允许更多版本暂时未合并。
2.1.11版本后不再需要，有单独的time_series_max_tablet_version_num配置|
| Compaction | `max_cumu_compaction_threads = 8` |
设置为 CPU 核数 / 4，意味着 CPU 资源的 1/4 用于写入，1/4 用于后台 Compaction，2/1 留给查询和其他操作。 |
| - | `inverted_index_compaction_enable = true` |
开启索引合并（index compaction），减少 Compaction 时的 CPU 消耗。 |
| - | `enable_segcompaction = false` `enable_ordered_data_compaction
= false` | 关闭日志场景不需要的两个 Compaction 功能。 |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/config/be-config.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/config/be-config.md
index 0be17807f7f..7c64cb159b3 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/config/be-config.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/config/be-config.md
@@ -1499,3 +1499,9 @@ load tablets from header failed, failed tablets size:
xxx, path=xxx
* 描述：用于文件缓存的磁盘路径和其他参数，以数组形式表示，每个磁盘一个条目。`path` 指定磁盘路径，`total_size` 限制缓存的大小；-1 或
0 将使用整个磁盘空间。
* 格式：
[{"path":"/path/to/file_cache","total_size":21474836480},{"path":"/path/to/file_cache2","total_size":21474836480}]
+
+#### `time_series_max_tablet_version_num`
+
+* 类型：int
+* 描述：限制time series compaction 策略的 tablet 最大 version 的数量。用于防止导入过于频繁，或
compaction 不及时导致的大量 version 堆积问题。当超过限制后，导入任务将被拒绝。从3.0.7版本开始支持这个参数
+* 默认值：20000
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/log-storage-analysis.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/log-storage-analysis.md
index 40e1b48d62a..3490ddacb48 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/log-storage-analysis.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/log-storage-analysis.md
@@ -82,7 +82,7 @@
| - | `enable_file_cache = true` |
开启文件缓存。 |
| - | `file_cache_path = [{"path": "/mnt/datadisk0/file_cache",
"total_size":53687091200, "query_limit": "10737418240"},{"path":
"/mnt/datadisk1/file_cache", "total_size":53687091200,"query_limit":
"10737418240"}]` | 配置冷数据的缓存路径和相关设置，具体配置说明如下：<br />`path`：缓存路径<br
/>`total_size`：该缓存路径的总大小，单位为字节，53687091200 字节等于 50 GB<br
/>`query_limit`：单次查询可以从缓存路径中查询的最大数据量，单位为字节，10737418240 字节等于 10 GB |
| 写入 | `write_buffer_size = 1073741824` |
增加写入缓冲区（buffer）的文件大小，减少小文件和随机 I/O 操作，提升性能。 |
-| - | `max_tablet_version_num = 20000` |
配合建表的 time_series compaction 策略，允许更多版本暂时未合并。 |
+| - | `max_tablet_version_num = 20000` |
配合建表的 time_series compaction
策略，允许更多版本暂时未合并。3.0.7版本后不再需要，有单独的time_series_max_tablet_version_num配置 |
| Compaction | `max_cumu_compaction_threads = 8` |
设置为 CPU 核数 / 4，意味着 CPU 资源的 1/4 用于写入，1/4 用于后台 Compaction，2/1 留给查询和其他操作。 |
| - | `inverted_index_compaction_enable = true` |
开启索引合并（index compaction），减少 Compaction 时的 CPU 消耗。 |
| - | `enable_segcompaction = false` `enable_ordered_data_compaction
= false` | 关闭日志场景不需要的两个 Compaction 功能。 |
diff --git a/versioned_docs/version-2.1/admin-manual/config/be-config.md
b/versioned_docs/version-2.1/admin-manual/config/be-config.md
index 0403b4bf99e..b472048753c 100644
--- a/versioned_docs/version-2.1/admin-manual/config/be-config.md
+++ b/versioned_docs/version-2.1/admin-manual/config/be-config.md
@@ -1498,3 +1498,9 @@ This configuration is supported since the Apache Doris
1.2 version
* Description: Doris comes with its own time zone database. If the time zone
file is not found in the system directory, the data in that directory is
enabled.
* Default: "${DORIS_HOME}/zoneinfo"
+
+#### `time_series_max_tablet_version_num`
+
+* Type: int
+* Description: Limit the number of versions of a single tablet under the
time-series compaction policy. It is used to prevent a large number of version
accumulation problems caused by too frequent load or delayed compaction. When
the limit is exceeded, the load task will be rejected. Supported since version
2.1.11
+* Default value: 20000
diff --git a/versioned_docs/version-2.1/log-storage-analysis.md
b/versioned_docs/version-2.1/log-storage-analysis.md
index 880d5262016..21522196892 100644
--- a/versioned_docs/version-2.1/log-storage-analysis.md
+++ b/versioned_docs/version-2.1/log-storage-analysis.md
@@ -187,7 +187,7 @@ You can find BE configuration fields in `be/conf/be.conf`.
Refer to the followin
| - | `enable_file_cache = true` |
Enable file caching. |
| - | `file_cache_path = [{"path": "/mnt/datadisk0/file_cache",
"total_size":53687091200, "query_limit": "10737418240"},{"path":
"/mnt/datadisk1/file_cache", "total_size":53687091200,"query_limit":
"10737418240"}]` | Configure the cache path and related settings for cold data
with the following specific configurations:<br/>`path`: cache
path<br/>`total_size`: total size of the cache path in bytes, where 53687091200
bytes equals 50 GB<br/>`query_limit`: maximum amount of data tha [...]
| Write | `write_buffer_size = 1073741824` |
Increase the file size of the write buffer to reduce small files and random I/O
operations, improving performance. |
-| - | `max_tablet_version_num = 20000` |
In coordination with the time_series compaction strategy for table creation,
allow more versions to remain temporarily unmerged |
+| - | `max_tablet_version_num = 20000` |
In coordination with the time_series compaction strategy for table creation,
allow more versions to remain temporarily unmerged. No longer required after
version 2.1.11, as there is a time_series_max_tablet_version_num configuration |
| Compaction | `max_cumu_compaction_threads = 8` |
Set to CPU core count / 4, indicating that 1/4 of CPU resources are used for
writing, 1/4 for background compaction, and 2/1 for queries and other
operations. |
| - | `inverted_index_compaction_enable = true` |
Enable inverted index compaction to reduce CPU consumption during compaction. |
| - | `enable_segcompaction = false` `enable_ordered_data_compaction
= false` | Disable two compaction features that are unnecessary for log
scenarios. |
diff --git a/versioned_docs/version-3.0/admin-manual/config/be-config.md
b/versioned_docs/version-3.0/admin-manual/config/be-config.md
index 655f466e0e5..56f5184b974 100644
--- a/versioned_docs/version-3.0/admin-manual/config/be-config.md
+++ b/versioned_docs/version-3.0/admin-manual/config/be-config.md
@@ -1365,3 +1365,9 @@ Default: true for cloud mode, false for non-cloud mode.
Default: [{"path":"${DORIS_HOME}/file_cache"}]
* Description: The disk paths and other parameters used for file cache,
represented as an array, with one entry for each disk. The `path` specifies the
disk path, and `total_size` limits the size of the cache; -1 or 0 will use the
entire disk space.
* format:
[{"path":"/path/to/file_cache","total_size":21474836480,{"path":"/path/to/file_cache2","total_size":21474836480}]
+
+#### `time_series_max_tablet_version_num`
+
+* Type: int
+* Description: Limit the number of versions of a single tablet under the
time-series compaction policy. It is used to prevent a large number of version
accumulation problems caused by too frequent load or untimely compaction. When
the limit is exceeded, the load task will be rejected. Supported since version
3.0.7
+* Default value: 20000
diff --git a/versioned_docs/version-3.0/log-storage-analysis.md
b/versioned_docs/version-3.0/log-storage-analysis.md
index d436a1281ea..0706557d887 100644
--- a/versioned_docs/version-3.0/log-storage-analysis.md
+++ b/versioned_docs/version-3.0/log-storage-analysis.md
@@ -187,7 +187,7 @@ You can find BE configuration fields in `be/conf/be.conf`.
Refer to the followin
| - | `enable_file_cache = true` |
Enable file caching. |
| - | `file_cache_path = [{"path": "/mnt/datadisk0/file_cache",
"total_size":53687091200, "query_limit": "10737418240"},{"path":
"/mnt/datadisk1/file_cache", "total_size":53687091200,"query_limit":
"10737418240"}]` | Configure the cache path and related settings for cold data
with the following specific configurations:<br/>`path`: cache
path<br/>`total_size`: total size of the cache path in bytes, where 53687091200
bytes equals 50 GB<br/>`query_limit`: maximum amount of data tha [...]
| Write | `write_buffer_size = 1073741824` |
Increase the file size of the write buffer to reduce small files and random I/O
operations, improving performance. |
-| - | `max_tablet_version_num = 20000` |
In coordination with the time_series compaction strategy for table creation,
allow more versions to remain temporarily unmerged |
+| - | `max_tablet_version_num = 20000` |
In coordination with the time_series compaction strategy for table creation,
allow more versions to remain temporarily unmerged. No longer required after
version 3.0.7, as there is a time_series_max_tablet_version_num configuration |
| Compaction | `max_cumu_compaction_threads = 8` |
Set to CPU core count / 4, indicating that 1/4 of CPU resources are used for
writing, 1/4 for background compaction, and 2/1 for queries and other
operations. |
| - | `inverted_index_compaction_enable = true` |
Enable inverted index compaction to reduce CPU consumption during compaction. |
| - | `enable_segcompaction = false` `enable_ordered_data_compaction
= false` | Disable two compaction features that are unnecessary for log
scenarios. |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
