JingsongLi commented on code in PR #8730:
URL: https://github.com/apache/paimon/pull/8730#discussion_r3612130730
##########
docs/docs/spark/sql-ddl.md:
##########
@@ -206,6 +206,52 @@ CREATE TABLE my_table (
);
```
+### Manage Format Table Partitions
+
+For managed Format Tables, whose `metastore.partitioned-table` option is
`true`, partition
+metadata is stored in the catalog and Spark supports the standard partition
DDL:
+
+```sql
+ALTER TABLE my_table ADD PARTITION (dt='2025-01-01');
+ALTER TABLE my_table DROP PARTITION (dt='2025-01-01');
+MSCK REPAIR TABLE my_table;
+SHOW PARTITIONS my_table;
+```
+
+On a Format Table that is not managed, `ADD PARTITION`, `DROP PARTITION` and
+`MSCK REPAIR TABLE` fail with an error.
+
+`SHOW PARTITIONS` throws an error when the number of partitions exceeds
+`spark.paimon.format-table.show-partitions.max-results` (default: 10000). For
larger tables, use
+`CALL sys.list_format_table_partitions` to list partitions page by page.
+
+`ADD PARTITION` creates the partition directory and registers the partition;
querying a newly
+added partition before any data is written returns no rows. `DROP PARTITION`
unregisters the
+partition and deletes its directory.
+
+:::info
+
+`metastore.partitioned-table = true` enables catalog-managed partitions, which
requires an
+internal Format Table in a catalog that supports it (currently the REST
catalog) and cannot be
+combined with `format-table.implementation = engine`. The REST catalog
validates this
+combination on `CREATE TABLE`, with catalog-level table defaults
+(`spark.sql.catalog.paimon.table-default.*`) participating in the effective
options: a default
+that makes the combination invalid fails the DDL. Other catalogs treat the
option as inert.
+
+A table that carries `metastore.partitioned-table = true` where it cannot be
honored (a non-REST
+catalog, or an external table) loads as an unmanaged Format Table and a
warning is logged. To
+remove the option, use `ALTER TABLE my_table UNSET TBLPROPERTIES
('metastore.partitioned-table')`.
+On a REST catalog, an existing Format Table whose partitions were never
registered reads as empty
+until you register them with `MSCK REPAIR TABLE my_table` or
+`CALL sys.sync_format_table_metadata`.
Review Comment:
Why not just use MSCK?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
