This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 6d6633837 [doc] Re-generate configuration docs
6d6633837 is described below
commit 6d6633837a8e9b9e8f5be8ad22dfd908470e9587
Author: Jingsong <[email protected]>
AuthorDate: Fri Aug 25 18:35:23 2023 +0800
[doc] Re-generate configuration docs
---
.../shortcodes/generated/core_configuration.html | 25 ++++++++++++++++++++++
.../generated/flink_catalog_configuration.html | 6 ++++++
.../generated/flink_connector_configuration.html | 20 ++++++++++++++++-
3 files changed, 50 insertions(+), 1 deletion(-)
diff --git a/docs/layouts/shortcodes/generated/core_configuration.html
b/docs/layouts/shortcodes/generated/core_configuration.html
index 383501f8f..a1cd016a9 100644
--- a/docs/layouts/shortcodes/generated/core_configuration.html
+++ b/docs/layouts/shortcodes/generated/core_configuration.html
@@ -176,6 +176,13 @@ under the License.
<td>String</td>
<td>Read incremental changes between start timestamp (exclusive)
and end timestamp, for example, 't1,t2' means changes between timestamp t1 and
timestamp t2.</td>
</tr>
+ <tr>
+ <td><h5>local-merge-buffer-size</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>MemorySize</td>
+ <td>Local merge will buffer and merge input records before they're
shuffled by bucket and written into sink. The buffer will be flushed when it is
full.
+Mainly to resolve data skew on primary keys. We recommend starting with 64 mb
when trying out this feature.</td>
+ </tr>
<tr>
<td><h5>local-sort.max-num-file-handles</h5></td>
<td style="word-wrap: break-word;">128</td>
@@ -310,6 +317,18 @@ This config option does not affect the default filesystem
metastore.</td>
<td>Double</td>
<td>Define the default false positive probability for bloom
filters.</td>
</tr>
+ <tr>
+ <td><h5>orc.column.encoding.direct</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>Integer</td>
+ <td>Comma-separated list of fields for which dictionary encoding
is to be skipped in orc.</td>
+ </tr>
+ <tr>
+ <td><h5>orc.dictionary.key.threshold</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>Integer</td>
+ <td>If the number of distinct keys in a dictionary is greater than
this fraction of the total number of non-null rows, turn off dictionary
encoding in orc. Use 1 to always use dictionary encoding.</td>
+ </tr>
<tr>
<td><h5>orc.write.batch-size</h5></td>
<td style="word-wrap: break-word;">1024</td>
@@ -322,6 +341,12 @@ This config option does not affect the default filesystem
metastore.</td>
<td>MemorySize</td>
<td>Memory page size.</td>
</tr>
+ <tr>
+ <td><h5>parquet.enable.dictionary</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>Integer</td>
+ <td>Turn off the dictionary encoding for all fields in
parquet.</td>
+ </tr>
<tr>
<td><h5>partial-update.ignore-delete</h5></td>
<td style="word-wrap: break-word;">false</td>
diff --git a/docs/layouts/shortcodes/generated/flink_catalog_configuration.html
b/docs/layouts/shortcodes/generated/flink_catalog_configuration.html
index c1ed6dbb3..15d80eac7 100644
--- a/docs/layouts/shortcodes/generated/flink_catalog_configuration.html
+++ b/docs/layouts/shortcodes/generated/flink_catalog_configuration.html
@@ -38,5 +38,11 @@ under the License.
<td>Boolean</td>
<td>If true, the register will automatically create and delete a
topic in log system for Paimon table. Default kafka log store register is
supported, users can implement customized register for log system, for example,
create a new class which extends KafkaLogStoreFactory and return a customized
LogStoreRegister for their kafka cluster to create/delete topics.</td>
</tr>
+ <tr>
+ <td><h5>log.system.auto-register-timeout</h5></td>
+ <td style="word-wrap: break-word;">1 min</td>
+ <td>Duration</td>
+ <td>The timeout for register to create or delete topic in log
system.</td>
+ </tr>
</tbody>
</table>
diff --git
a/docs/layouts/shortcodes/generated/flink_connector_configuration.html
b/docs/layouts/shortcodes/generated/flink_connector_configuration.html
index d72eee6f6..0a5cbd1cd 100644
--- a/docs/layouts/shortcodes/generated/flink_connector_configuration.html
+++ b/docs/layouts/shortcodes/generated/flink_connector_configuration.html
@@ -38,6 +38,18 @@ under the License.
<td>String</td>
<td>The log system used to keep changes of the table.<br /><br
/>Possible values:<br /><ul><li>"none": No log system, the data is written only
to file store, and the streaming read will be directly read from the file
store.</li></ul><ul><li>"kafka": Kafka log system, the data is double written
to file store and kafka, and the streaming read will be read from kafka. If
streaming read from file, configures streaming-read-mode to file.</li></ul></td>
</tr>
+ <tr>
+ <td><h5>log.system.partitions</h5></td>
+ <td style="word-wrap: break-word;">1</td>
+ <td>Integer</td>
+ <td>The number of partitions of the log system. If log system is
kafka, this is kafka partitions.</td>
+ </tr>
+ <tr>
+ <td><h5>log.system.replication</h5></td>
+ <td style="word-wrap: break-word;">1</td>
+ <td>Integer</td>
+ <td>The number of replication of the log system. If log system is
kafka, this is kafka replicationFactor.</td>
+ </tr>
<tr>
<td><h5>lookup.async</h5></td>
<td style="word-wrap: break-word;">false</td>
@@ -54,7 +66,7 @@ under the License.
<td><h5>scan.infer-parallelism</h5></td>
<td style="word-wrap: break-word;">true</td>
<td>Boolean</td>
- <td>If it is false, parallelism of source are set by
scan.parallelism. Otherwise, source parallelism is inferred from splits number
(batch mode) or bucket number(streaming mode).</td>
+ <td>If it is false, parallelism of source are set by global
parallelism. Otherwise, source parallelism is inferred from splits number
(batch mode) or bucket number(streaming mode).</td>
</tr>
<tr>
<td><h5>scan.parallelism</h5></td>
@@ -122,6 +134,12 @@ under the License.
<td>Integer</td>
<td>Defines a custom parallelism for the sink. By default, if this
option is not defined, the planner will derive the parallelism for each
statement individually by also considering the global configuration.</td>
</tr>
+ <tr>
+ <td><h5>sink.savepoint.auto-tag</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If true, a tag will be automatically created for the snapshot
created by flink savepoint.</td>
+ </tr>
<tr>
<td><h5>sink.use-managed-memory-allocator</h5></td>
<td style="word-wrap: break-word;">false</td>
