This is an automated email from the ASF dual-hosted git repository.
rexxiong pushed a commit to branch branch-0.4
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/branch-0.4 by this push:
new 2246651a0 [CELEBORN-1345][FOLLOWUP] Add a limit to the master's
estimated partition size, fix doc
2246651a0 is described below
commit 2246651a0efa918ab5b6c193aa8308f5b8afbe6b
Author: Shuang <[email protected]>
AuthorDate: Mon Mar 25 16:22:26 2024 +0800
[CELEBORN-1345][FOLLOWUP] Add a limit to the master's estimated partition
size, fix doc
---
docs/configuration/master.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/configuration/master.md b/docs/configuration/master.md
index 94773a978..2a7c85e86 100644
--- a/docs/configuration/master.md
+++ b/docs/configuration/master.md
@@ -22,8 +22,8 @@ license: |
| celeborn.dynamicConfig.refresh.interval | 120s | Interval for refreshing the
corresponding dynamic config periodically. | 0.4.0 | |
| celeborn.dynamicConfig.store.backend | NONE | Store backend for dynamic
config. Available options: NONE, FS. Note: NONE means disabling dynamic config
store. | 0.4.0 | |
| celeborn.master.estimatedPartitionSize.initialSize | 64mb | Initial
partition size for estimation, it will change according to runtime stats. |
0.3.0 | celeborn.shuffle.initialEstimatedPartitionSize |
-| celeborn.master.estimatedPartitionSize.maxSize | <undefined> | false |
Max partition size for estimation. Default value should be
celeborn.worker.shuffle.partitionSplit.max * 2. | 0.4.1 | |
-| celeborn.master.estimatedPartitionSize.minSize | 8mb | false | Ignore
partition size smaller than this configuration of partition size for
estimation. | 0.3.0 | celeborn.shuffle.minPartitionSizeToEstimate |
+| celeborn.master.estimatedPartitionSize.maxSize | <undefined> | Max
partition size for estimation. Default value should be
celeborn.worker.shuffle.partitionSplit.max * 2. | 0.4.1 | |
+| celeborn.master.estimatedPartitionSize.minSize | 8mb | Ignore partition size
smaller than this configuration of partition size for estimation. | 0.3.0 |
celeborn.shuffle.minPartitionSizeToEstimate |
| celeborn.master.estimatedPartitionSize.update.initialDelay | 5min | Initial
delay time before start updating partition size for estimation. | 0.3.0 |
celeborn.shuffle.estimatedPartitionSize.update.initialDelay |
| celeborn.master.estimatedPartitionSize.update.interval | 10min | Interval of
updating partition size for estimation. | 0.3.0 |
celeborn.shuffle.estimatedPartitionSize.update.interval |
| celeborn.master.hdfs.expireDirs.timeout | 1h | The timeout for a expire dirs
to be deleted on HDFS. | 0.3.0 | |
