This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
The following commit(s) were added to refs/heads/master by this push:
new f477b2609 [doc] Document Local Merging not work for CDC ingestion
f477b2609 is described below
commit f477b2609ad8658a5a1b317facefc93d2efd6e6e
Author: Jingsong <[email protected]>
AuthorDate: Mon Aug 21 10:34:44 2023 +0800
[doc] Document Local Merging not work for CDC ingestion
---
docs/content/maintenance/write-performance.md | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/docs/content/maintenance/write-performance.md
b/docs/content/maintenance/write-performance.md
index f25eae4ef..21d18faad 100644
--- a/docs/content/maintenance/write-performance.md
+++ b/docs/content/maintenance/write-performance.md
@@ -166,7 +166,7 @@ Compaction will become less frequent when
`num-sorted-run.compaction-trigger` be
## Local Merging
If your job suffers from primary key data skew
-(for example, you want to count the number of views for each pages in a
website,
+(for example, you want to count the number of views for each page in a website,
and some particular pages are very popular among the users),
you can set `'local-merge-buffer-size'` so that input records will be buffered
and merged
before they're shuffled by bucket and written into sink.
@@ -175,6 +175,8 @@ This is particularly useful when the same primary key is
updated frequently betw
The buffer will be flushed when it is full. We recommend starting with `64 mb`
when you are faced with data skew but don't know where to start adjusting
buffer size.
+(Currently, Local merging not works for CDC ingestion)
+
## File Format
If you want to achieve ultimate compaction performance, you can consider using
row storage file format AVRO.
