sundapeng opened a new pull request, #8729:
URL: https://github.com/apache/paimon/pull/8729
## Relationship
Second part of splitting #8713; stacked on the core part #8728 (the diff
shown here includes
the core commit until that PR merges — please review only the `[spark]
Support managed
Format Table partition DDL` commit). Marked draft until the core part lands.
## Purpose
Spark SQL partition DDL for managed Format Tables on top of the core part:
`ALTER TABLE ... ADD/DROP PARTITION` backed by the catalog.
## Main changes
- A partition gateway (`FormatTablePartitionCatalog`) wired into
`PaimonFormatTable`;
DDL executes against the catalog, not the filesystem.
- ADD registers then creates the partition directory (empty partitions are
immediately
visible and writable). Partition values are validated against path escapes
(`.`/`..`, resolved path must stay under the table root); `ADD ...
LOCATION` is
rejected (managed table locations are derived).
- DROP unregisters first, then deletes the directory (the service never
deletes data);
a full spec that is not registered fails without touching its directory
(protects
data awaiting registration). Partial specs expand through the catalog to
leaf
partitions; expansions and unregistrations run in bounded batches.
- Strict-ADD atomicity (reject the whole batch if any partition exists) is
enforced by
the catalog service — the client forwards `IF NOT EXISTS` unchanged and
skips Spark's
client-side existence precheck.
- Session-level `metastore.partitioned-table` overrides are ignored with a
WARN (the
persisted table property wins) instead of failing table loads.
## Tests
DDL planning with a fault-injection catalog (26), partition management
round-trips (20),
gateway contract tests incl. batching and prefix pushdown (6), option
handling (5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
