sundapeng opened a new pull request, #9540:
URL: https://github.com/apache/paimon/pull/9540
### Purpose
As a follow-up to #8750 and #8751, this adds Spark's standard
explicit-location
form for registering existing data in a catalog-managed Format Table:
```sql
ALTER TABLE t ADD PARTITION (dt = '2026-09-02')
LOCATION 's3://bucket/existing/dt=2026-09-02';
```
For `ADD PARTITION ... LOCATION`, Spark canonicalizes and validates the
absolute
URI before registration. It rejects roots, table ancestors, and the
partition's
derived default directory, registers the location as partition metadata
without
creating or probing the supplied directory, and verifies that the REST server
stored and echoed the same canonical value. Partitions added without
`LOCATION`
keep their current derived-path behavior.
When explicit locations may exist, reads and mutation preflights validate the
complete catalog registry as one path-ownership set. Duplicate, ancestor,
descendant, table-root, and derived-path conflicts fail closed, as does a
mismatch
between the authoritative explicit-location count and the listed registry.
Locations outside the table root use a `FileIO` resolved from the client's
`CatalogContext`; locations under the table root continue using the table's
`FileIO`.
Data in an explicitly located partition is read-only to Paimon. Write
commits,
overwrites, truncates, and `ANALYZE TABLE` operations whose scope includes
such a
partition fail before publishing or deleting files or mutating catalog
metadata.
`DROP PARTITION` only unregisters it and never probes or deletes the
explicit URI.
`MSCK REPAIR TABLE` leaves the registration intact and does not inspect or
collect
statistics from that location.
The REST contract uses a dedicated `partitions/with-locations` resource and
the
`format-table-partition-location-v1` capability token. A location-aware POST
is
refused unless the server advertises the capability. Spark REST requests
advertise
the client capability; Flink requests intentionally do not. The existing
partition
resource and location-free payloads remain compatible.
This change is limited to catalog-managed Format Tables; it does not add
multi-location storage to native Paimon tables or change the table-level
`SET LOCATION` rejection from #9435.
### Tests
- 489 tests in 12 focused `paimon-api` and `paimon-core` suites covering
JSON and
REST compatibility, catalog delegation and caching, partition management,
scans,
reads, commits, `FileIO` routing, registry consistency, and path ownership.
- Spark 3: 96 focused Scala tests, 19 `FormatTablePartitionRepairTest`
tests, and
2 REST capability-header tests.
- Spark 4: 83 focused Scala tests and 2 REST capability-header tests.
- Flink REST compatibility integration tests: 2.
- Checkstyle, Spotless, Maven Enforcer, and RAT.
- JDK 8 `clean package`.
- REST OpenAPI validation: 57 Catalog operations and 6 Management operations.
- Docusaurus production build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
