sundapeng opened a new pull request, #591:
URL: https://github.com/apache/paimon-rust/pull/591
### Purpose
Linked issue: N/A (no matching issue found)
Add Spark-compatible partition administration SQL for internal Format Tables
managed by a REST Catalog. REST partition registrations are treated as
authoritative so partition commands and table scans use the same metadata
source.
### Brief change log
- Add Java-compatible REST partition request/response models and catalog
operations for listing, creating, and dropping partitions, including
pagination, batching, and error mapping.
- Mark eligible internal REST Format Tables as catalog-managed and drive
scans from registered partitions; unregistered directories stay invisible,
while registered missing directories are empty only for storage `NotFound`
errors.
- Add DataFusion SQL support for:
- `SHOW PARTITIONS table [PARTITION (...)]`
- `ALTER TABLE table ADD [IF NOT EXISTS] PARTITION (...) [...]`
- `ALTER TABLE table DROP [IF EXISTS] PARTITION (...) [, ...]`
- `[MSCK] REPAIR TABLE table [{ADD|DROP|SYNC} PARTITIONS]`
- Preserve raw partition values during repair, validate encoded paths before
mutation, register before directory creation on ADD, unregister before deletion
on DROP, and keep MSCK repair metadata-only.
- Document the supported table scope, syntax, behavior, and limitations.
### Tests
- `cargo fmt --all -- --check`
- `git diff --check`
- `cargo test --locked -p paimon --all-targets` — 1967 passed, 1 ignored
- `cargo test --locked -p paimon-datafusion --lib` — 368 passed
- `cargo test --locked -p paimon-datafusion --test
rest_format_partition_sql` — 4 passed
- `cargo clippy --locked -p paimon -p paimon-datafusion --all-targets -- -D
warnings`
- Additional `paimon-datafusion --all-targets` run: 738 passed, 2 ignored;
the two vector-search tests could not load the environment-provided
`liblumina_py.so` shared library.
### API and Format
- Public API: additive only. This adds defaulted `Catalog` partition
methods, REST partition request/response types, and
`FormatTablePartitionPaths`; existing catalog implementations remain
source-compatible.
- Storage format: unchanged.
- Behavioral scope: only internal, partitioned REST Format Tables with
`metastore.partitioned-table=true` and a non-`engine` implementation use
REST-authoritative partition scans and commands.
### Documentation
- Update `docs/src/sql.md` with syntax, examples, semantics, and
supported-table constraints.
- Add design and implementation-plan documents for REST Format Table
partition commands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
