This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
The following commit(s) were added to refs/heads/master by this push:
new df6f3519b [doc] Document completely asynchronous compaction
df6f3519b is described below
commit df6f3519bd6e0772d2d7a2cbbc0ea9893b49df93
Author: Jingsong <[email protected]>
AuthorDate: Wed Aug 2 15:27:10 2023 +0800
[doc] Document completely asynchronous compaction
---
docs/content/maintenance/write-performance.md | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/docs/content/maintenance/write-performance.md
b/docs/content/maintenance/write-performance.md
index 21ac5aa6a..4bb0feec4 100644
--- a/docs/content/maintenance/write-performance.md
+++ b/docs/content/maintenance/write-performance.md
@@ -64,6 +64,20 @@ It is recommended that the parallelism of sink should be
less than or equal to t
## Compaction
+### Asynchronous Compaction
+
+Compaction is inherently asynchronous, but if you want it to be completely
asynchronous and not blocking writing,
+expect a mode to have maximum write throughput, the compaction can be done
slowly and not in a hurry.
+You can use the following strategies for your table:
+
+```shell
+num-sorted-run.stop-trigger = 2147483647
+sort-spill-threshold = 10
+```
+
+This configuration will generate more files during peak write periods and
gradually merge into optimal read
+performance during low write periods.
+
### Number of Sorted Runs to Pause Writing
When number of sorted runs is small, Paimon writers will perform compaction
asynchronously in separated threads, so
@@ -97,19 +111,6 @@ performance. However, if this value becomes too large, more
memory and CPU time
table. If you are concerned about the OOM of memory, please configure the
following option `sort-spill-threshold`.
Its value depends on your memory size.
-### Prioritize write throughput
-
-If you expect a mode to have maximum write throughput, the compaction can be
done slowly and not in a hurry.
-You can use the following strategies for your table:
-
-```shell
-num-sorted-run.stop-trigger = 2147483647
-sort-spill-threshold = 10
-```
-
-This configuration will generate more files during peak write periods and
gradually merge into optimal read
-performance during low write periods.
-
### Number of Sorted Runs to Trigger Compaction
Paimon uses [LSM tree]({{< ref "concepts/file-layouts#lsm-trees" >}}) which
supports a large number of updates. LSM organizes files in several [sorted
runs]({{< ref "concepts/file-layouts#sorted-runs" >}}). When querying records
from an LSM tree, all sorted runs must be combined to produce a complete view
of all records.
