This is an automated email from the ASF dual-hosted git repository.
JingsongLi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new a15ba908a3 [docs] Fix stale type support, procedure and CDC action
references (#9973)
a15ba908a3 is described below
commit a15ba908a36a21dba5e5c0ea4a9ef2cc533b6bc8
Author: Eunbin Son <[email protected]>
AuthorDate: Tue Sep 22 12:37:39 2026 +0900
[docs] Fix stale type support, procedure and CDC action references (#9973)
---
docs/docs/cdc-ingestion/mysql-cdc.mdx | 3 ++-
docs/docs/concepts/spec/fileformat.md | 9 ++++-----
docs/docs/flink/procedures/compaction.md | 5 ++++-
docs/docs/flink/procedures/repair.md | 12 ++++++------
docs/docs/flink/procedures/snapshots.md | 7 +++----
docs/docs/flink/sql-query.mdx | 2 +-
docs/docs/multimodal-table/global-index/btree.mdx | 2 ++
docs/docs/spark/sql-write.md | 3 +++
docs/generated/kafka_sync_database.html | 4 ----
docs/generated/mysql_sync_database.html | 4 ++++
docs/generated/pulsar_sync_database.html | 4 ----
docs/generated/temporal_functions.html | 2 +-
12 files changed, 30 insertions(+), 27 deletions(-)
diff --git a/docs/docs/cdc-ingestion/mysql-cdc.mdx
b/docs/docs/cdc-ingestion/mysql-cdc.mdx
index 5a3c794f94..f2b08b9c8e 100644
--- a/docs/docs/cdc-ingestion/mysql-cdc.mdx
+++ b/docs/docs/cdc-ingestion/mysql-cdc.mdx
@@ -299,7 +299,8 @@ Command syntax (square brackets indicate optional
arguments):
[--primary_keys <primary-keys>] \
[--mysql_conf <mysql-cdc-source-conf> [--mysql_conf
<mysql-cdc-source-conf> ...]] \
[--catalog_conf <paimon-catalog-conf> [--catalog_conf
<paimon-catalog-conf> ...]] \
- [--table_conf <paimon-table-sink-conf> [--table_conf
<paimon-table-sink-conf> ...]]
+ [--table_conf <paimon-table-sink-conf> [--table_conf
<paimon-table-sink-conf> ...]] \
+ [--table_conf_by_table <source-table>:<key>=<value> [--table_conf_by_table
<source-table>:<key>=<value> ...]]
```
<ConfigTable html={mysqlSyncDatabaseHtml} />
diff --git a/docs/docs/concepts/spec/fileformat.md
b/docs/docs/concepts/spec/fileformat.md
index ab36e5fbb2..5ece594d1e 100644
--- a/docs/docs/concepts/spec/fileformat.md
+++ b/docs/docs/concepts/spec/fileformat.md
@@ -225,7 +225,6 @@ The following table lists the type mapping from Paimon type
to JSON type.
| `TIMESTAMP_LOCAL_ZONE` | `string with format: date-time (with UTC time
zone)` |
| `ARRAY` | `array` |
| `MAP` | `object` |
-| `MULTISET` | `object` |
| `ROW` | `object` |
## Lance
@@ -250,12 +249,11 @@ The following table lists the type mapping from Paimon
type to Lance (Arrow) typ
| TIME | TIME32 / TIME64 |
| TIMESTAMP(P) | TIMESTAMP (unit based on precision) |
| ARRAY | LIST |
-| MULTISET | LIST |
| ROW | STRUCT |
Limitations:
-1. Lance file format does not support `MAP` type.
-2. Lance file format does not support `TIMESTAMP_LOCAL_ZONE` type.
+1. Lance file format does not support `MAP`, `MULTISET`,
`TIMESTAMP_LOCAL_ZONE` or `VARIANT` types.
+2. Lance file format does not support `BLOB` fields stored inline in data
files (`blob-descriptor-field` or `blob-view-field`). Regular `BLOB` fields are
written to dedicated blob files and are not affected.
## Vortex
@@ -267,7 +265,8 @@ Key features:
- **Column Projection**: Only reads requested columns from disk
Limitations:
-1. Vortex does not support `MAP` or `MULTISET` types.
+1. Vortex does not support `MAP`, `MULTISET` or `VARIANT` types.
+2. Vortex does not support `BLOB` fields stored inline in data files
(`blob-descriptor-field` or `blob-view-field`). Regular `BLOB` fields are
written to dedicated blob files and are not affected.
## Mosaic
diff --git a/docs/docs/flink/procedures/compaction.md
b/docs/docs/flink/procedures/compaction.md
index 5b26f1dfda..bec6556333 100644
--- a/docs/docs/flink/procedures/compaction.md
+++ b/docs/docs/flink/procedures/compaction.md
@@ -48,6 +48,8 @@ To compact a table. Arguments:
- `compact_strategy` (optional): this determines how to pick files to be
merged, the default is determined by the runtime execution mode. 'full'
strategy only supports batch mode. All files will be selected for merging.
'minor' strategy: Pick the set of files that need to be merged based on
specified conditions.
+- `buckets` (optional): fixed-bucket IDs, comma-separated IDs, or inclusive
ranges, for example `'0-3,7'`. Omit to include all buckets. Valid only for
fixed-bucket tables without sort compaction.
+
**Syntax**
```sql
@@ -60,7 +62,8 @@ CALL [catalog.]sys.compact(
options => 'options',
`where` => 'where',
partition_idle_time => 'partition_idle_time',
- compact_strategy => 'compact_strategy');
+ compact_strategy => 'compact_strategy',
+ buckets => 'buckets');
-- Use indexed argument
CALL [catalog.]sys.compact('table');
diff --git a/docs/docs/flink/procedures/repair.md
b/docs/docs/flink/procedures/repair.md
index cbbf919ec9..63690c7dd5 100644
--- a/docs/docs/flink/procedures/repair.md
+++ b/docs/docs/flink/procedures/repair.md
@@ -192,32 +192,32 @@ CALL [catalog.]sys.remove_unexisting_files('identifier',
'dryRun', 'parallelism'
CALL sys.remove_unexisting_files(`table` => 'mydb.myt');
-- only check what files will be removed, but not really remove them (dry run)
-CALL sys.remove_unexisting_files(`table` => 'mydb.myt', `dry_run` = true);
+CALL sys.remove_unexisting_files(`table` => 'mydb.myt', `dry_run` => true);
```
## remove_unexisting_manifests
Procedure to remove unexisting manifest file from manifset-list. for detailed
use cases. Arguments:
-- `table`: the target table identifier. Cannot be empty, you can use
database.table$branch_xx to remove branch table unexisting manifest file.
+- `tableId`: the target table identifier. Cannot be empty, you can use
database.table$branch_xx to remove branch table unexisting manifest file.
Note that user is on his own risk using this procedure, which may cause data
loss when used outside from the use cases listed in Java docs.
**Syntax**
```sql
--- Use named argument
-CALL [catalog.]sys.remove_unexisting_files(`table` => 'identifier');
+-- Use indexed argument
+CALL [catalog.]sys.remove_unexisting_manifests('identifier');
```
**Example**
```sql
-- remove unexisting manifest file in the table `mydb.myt`
-CALL sys.remove_unexisting_manifests(`table` => 'mydb.myt');
+CALL sys.remove_unexisting_manifests('mydb.myt');
-- remove unexisting manifest file in the branch table `mydb.myt$branch_rt`
-CALL sys.remove_unexisting_manifests(`table` => 'mydb.myt$branch_rt');
+CALL sys.remove_unexisting_manifests('mydb.myt$branch_rt');
```
## repair
diff --git a/docs/docs/flink/procedures/snapshots.md
b/docs/docs/flink/procedures/snapshots.md
index 3f1e91b8b8..07903186e2 100644
--- a/docs/docs/flink/procedures/snapshots.md
+++ b/docs/docs/flink/procedures/snapshots.md
@@ -38,7 +38,7 @@ To expire snapshots. Argument:
- `retain_min`: the minimum number of completed snapshots to retain.
-- `order_than`: timestamp before which snapshots will be removed.
+- `older_than`: timestamp before which snapshots will be removed.
- `max_deletes`: the maximum number of snapshots that can be deleted at once.
@@ -95,7 +95,7 @@ To expire changelogs. Argument:
- `retain_min`: the minimum number of completed changelogs to retain.
-- `order_than`: timestamp before which changelogs will be removed.
+- `older_than`: timestamp before which changelogs will be removed.
- `max_deletes`: the maximum number of changelogs that can be deleted at once.
@@ -110,8 +110,7 @@ CALL [catalog.]sys.expire_changelogs(
retain_max => 'retain_max',
retain_min => 'retain_min',
older_than => 'older_than',
- max_deletes => 'max_deletes');
-
+ max_deletes => 'max_deletes',
delete_all => 'delete_all');
-- Use indexed argument
diff --git a/docs/docs/flink/sql-query.mdx b/docs/docs/flink/sql-query.mdx
index ec1dd99e20..bcb3d34b07 100644
--- a/docs/docs/flink/sql-query.mdx
+++ b/docs/docs/flink/sql-query.mdx
@@ -253,7 +253,7 @@ Otherwise, Paimon infers source parallelism only when
global parallelism is unse
| Read path | Inference |
| --- | --- |
-| Batch data table | Estimate planned work from partition data size and
`split.target-size`; the estimate need not equal the actual split count. |
+| Batch data table | Estimate planned work from partition data size and
`source.split.target-size`; the estimate need not equal the actual split count.
|
| Other batch sources | Use their planned splits. |
| Streaming, fixed buckets | Use the bucket count. |
| Streaming, `bucket = -1` | Fall back to Flink's configured or default
parallelism. |
diff --git a/docs/docs/multimodal-table/global-index/btree.mdx
b/docs/docs/multimodal-table/global-index/btree.mdx
index e16ef839d4..c019630fab 100644
--- a/docs/docs/multimodal-table/global-index/btree.mdx
+++ b/docs/docs/multimodal-table/global-index/btree.mdx
@@ -154,8 +154,10 @@ print(pa_table)
| `btree-index.block-size` | `64 kb` | Block size used by BTree index files. |
| `btree-index.bloom-filter.enabled` | `false` | Whether to write a Bloom
filter to accelerate BTree equality and `IN` lookups. |
| `btree-index.cache-size` | `128 mb` | Cache size used by BTree index
readers. |
+| `btree-index.high-priority-pool-ratio` | `0.1` | Fraction of
`btree-index.cache-size` reserved for high-priority data such as index blocks;
the rest caches data blocks. Must be in `[0, 1)`. |
| `btree-index.fallback-scan-max-size` | `256 mb` | Maximum total size of
candidate BTree global index files to allow fallback index scans. Set to `0 b`
to disable fallback scans. |
| `btree-index.compression` | `none` | Compression algorithm used by BTree
index blocks. |
+| `btree-index.compression-level` | `1` | Compression level used by codecs
that support levels, such as `zstd`. |
The legacy `btree-index.records-per-range` and
`btree-index.build.max-parallelism` keys are still recognized as fallback keys.
diff --git a/docs/docs/spark/sql-write.md b/docs/docs/spark/sql-write.md
index 98cc810375..4cf4e661a3 100644
--- a/docs/docs/spark/sql-write.md
+++ b/docs/docs/spark/sql-write.md
@@ -231,10 +231,13 @@ USING source
ON <merge condition>
WHEN MATCHED [AND <condition>] THEN { UPDATE SET ... | DELETE }
WHEN NOT MATCHED [AND <condition>] THEN INSERT ...
+WHEN NOT MATCHED BY SOURCE [AND <condition>] THEN { UPDATE SET ... | DELETE }
```
Each `WHEN` clause can be repeated; clauses are evaluated in order, and the
first matching one wins for a given row.
+`WHEN NOT MATCHED BY SOURCE` requires Spark 3.4 or later.
+
### Examples
The examples below assume both source and target have schema `(a INT, b INT, c
STRING)`, with `a` as the primary key.
diff --git a/docs/generated/kafka_sync_database.html
b/docs/generated/kafka_sync_database.html
index e8af0f82ab..14e3e6c81f 100644
--- a/docs/generated/kafka_sync_database.html
+++ b/docs/generated/kafka_sync_database.html
@@ -14,10 +14,6 @@
<td><h5>--database</h5></td>
<td>The database name in Paimon catalog.</td>
</tr>
- <tr>
- <td><h5>--ignore_incompatible</h5></td>
- <td>It is default false, in this case, if MySQL table name exists in
Paimon and their schema is incompatible,an exception will be thrown. You can
specify it to true explicitly to ignore the incompatible tables and
exception.</td>
- </tr>
<tr>
<td><h5>--table_mapping</h5></td>
<td>The table name mapping between source database and Paimon. For
example, if you want to synchronize a source table named "test" to a Paimon
table named "paimon_test", you can specify "--table_mapping test=paimon_test".
Multiple mappings could be specified with multiple "--table_mapping" options.
"--table_mapping" has higher priority than "--table_prefix" and
"--table_suffix".</td>
diff --git a/docs/generated/mysql_sync_database.html
b/docs/generated/mysql_sync_database.html
index 8b4a62786b..9d83288fbc 100644
--- a/docs/generated/mysql_sync_database.html
+++ b/docs/generated/mysql_sync_database.html
@@ -91,5 +91,9 @@
<td><h5>--table_conf</h5></td>
<td>The configuration for Paimon table sink. Each configuration should
be specified in the format "key=value". See <a
href="../maintenance/configurations">here</a> for a complete list of table
configurations.</td>
</tr>
+ <tr>
+ <td><h5>--table_conf_by_table</h5></td>
+ <td>The configuration for one Paimon table sink, overriding the same
key from --table_conf for that table. Each configuration should be specified in
the format "source_table:key=value", where source_table is the MySQL table
name. Only table properties can be overridden per table; options prefixed with
"sink." must be set with --table_conf.</td>
+ </tr>
</tbody>
</table>
\ No newline at end of file
diff --git a/docs/generated/pulsar_sync_database.html
b/docs/generated/pulsar_sync_database.html
index 3793958a50..253da26a8d 100644
--- a/docs/generated/pulsar_sync_database.html
+++ b/docs/generated/pulsar_sync_database.html
@@ -14,10 +14,6 @@
<td><h5>--database</h5></td>
<td>The database name in Paimon catalog.</td>
</tr>
- <tr>
- <td><h5>--ignore_incompatible</h5></td>
- <td>It is default false, in this case, if MySQL table name exists in
Paimon and their schema is incompatible,an exception will be thrown. You can
specify it to true explicitly to ignore the incompatible tables and
exception.</td>
- </tr>
<tr>
<td><h5>--table_prefix</h5></td>
<td>The prefix of all Paimon tables to be synchronized. For example,
if you want all synchronized tables to have "ods_" as prefix, you can specify
"--table_prefix ods_".</td>
diff --git a/docs/generated/temporal_functions.html
b/docs/generated/temporal_functions.html
index 01f8506b22..6374df4835 100644
--- a/docs/generated/temporal_functions.html
+++ b/docs/generated/temporal_functions.html
@@ -36,7 +36,7 @@
</tr>
<tr>
<td><h5>now()</h5></td>
- <td>Get the timestamp when ingesting the record. Output type is
TIMESTAMP_LTZ(3).</td>
+ <td>Get the timestamp when ingesting the record. Output type is
TIMESTAMP(3).</td>
</tr>
</tbody>
</table>
\ No newline at end of file
