This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 2a7a01fb0 [doc] Document Asynchronous for lookup changelog-producer
2a7a01fb0 is described below
commit 2a7a01fb091ee28af083ed19d0f49a6adc805ceb
Author: Jingsong <[email protected]>
AuthorDate: Wed Nov 29 11:33:39 2023 +0800
[doc] Document Asynchronous for lookup changelog-producer
---
docs/content/maintenance/write-performance.md | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/docs/content/maintenance/write-performance.md
b/docs/content/maintenance/write-performance.md
index 59bdc9267..6eb503539 100644
--- a/docs/content/maintenance/write-performance.md
+++ b/docs/content/maintenance/write-performance.md
@@ -39,6 +39,10 @@ Option `'changelog-producer' = 'lookup' or
'full-compaction'`, and option `'full
large impact on write performance, if it is a snapshot / full synchronization
phase you can unset these options and
then enable them again in the incremental phase.
+If you find that the input of the job shows a jagged pattern in the case of
backpressure, it may be imbalanced work
+nodes. You can consider turning on [Asynchronous Compaction]({{< ref
"#asynchronous-compaction" >}}) to observe if the
+throughput is increased.
+
## Parallelism
It is recommended that the parallelism of sink should be less than or equal to
the number of buckets, preferably equal. You can control the parallelism of the
sink with the `sink.parallelism` table property.
@@ -80,6 +84,9 @@ sort-spill-threshold = 10
This configuration will generate more files during peak write periods and
gradually merge into optimal read
performance during low write periods.
+In the case of `'changelog-producer' = 'lookup'`, by default, the lookup will
be completed at checkpointing, which
+will block the checkpoint. If you want an asynchronous lookup, you can enable
`'changelog-producer.lookup-wait' = 'true'`.
+
### Number of Sorted Runs to Pause Writing
When the number of sorted runs is small, Paimon writers will perform
compaction asynchronously in separated threads, so
