shaoyu-li opened a new issue, #12429:
URL: https://github.com/apache/gravitino/issues/12429
### Describe the feature
Introduce a pluggable SPI that lets deployments delegate table location
assignment and release to an external service, instead of requiring the
location to be a statically configured, already-known path.
Today `GenericCatalogOperations#calculateTableLocation` resolves a managed
table's location through a fixed, hard-coded chain:
1. the location table property (for Lance, this is what the
`x-lance-table-location` REST header ends up as);
2. otherwise, the location schema property + table name;
3. otherwise, the location catalog property + schema name + table name;
4. otherwise, throw `IllegalArgumentException`.
The method is private and the chain is not extensible. Symmetrically, when a
table is dropped nothing at all happens to the storage that was assigned to it.
We propose adding an interface — working name `TableLocationProvider` —
discovered via `ServiceLoader` and selected by a catalog property, consulted
when a managed table is created without an explicit location, and notified when
such a table is dropped.
### Motivation
**Assignment.** All three existing sources share the same assumption:
*someone already knows the
concrete path, and supplies it as configuration or as request input.*
In enterprise deployments that assumption often does not hold. Physical
storage layout is owned by
a central data-platform service, and the path for a new table is *allocated*
at creation time
rather than configured ahead of time. Allocation typically depends on inputs
Gravitino has no
opinion about — owning team, organizational domain, region, storage tier,
quota and capacity,
retention policy — and it usually has side effects, such as creating or
reserving the backing
bucket/prefix and registering ownership for lifecycle and cost attribution.
Under the current design an operator has two options, both unsatisfying:
- **Push the problem to the client.** Every writer must call the platform's
allocation service
itself and then pass the resulting path in via `x-lance-table-location`
(or the `location`
property). This spreads a platform concern across every engine and every
user, and nothing stops
a client from passing an arbitrary path that bypasses platform policy
entirely.
- **Fork Gravitino.** Patch `calculateTableLocation` locally, which makes
tracking upstream
painful.
A first-class extension point removes both, and lets the deployment
*enforce* the policy
server-side rather than hoping clients cooperate.
**Release.** The gap on the drop side is arguably worse, because it silently
leaks.
`ManagedTableOperations#purgeTable` states it plainly:
> For Gravitino managed tables, `purgeTable` is equivalent to `dropTable`.
It only removes the
> table metadata from the entity store. Physical data deletion should be
handled by the specific
> catalog implementation if needed.
For a deployment that assigns its own paths this is at most a housekeeping
matter. For a
deployment backed by an external allocator it means storage is stranded
permanently: the allocator
is never told the space can be reclaimed, so quota accounting, cost
attribution and retention
policy all drift out of sync with reality, with no way to tell an
intentionally-dropped table
apart from one that never existed.
Gravitino does have an event mechanism — `TableEventDispatcher` dispatches
`DropTablePreEvent` /
`DropTableEvent` (and the `Purge*` equivalents), and `EventListenerPlugin`
supports a `SYNC` mode —
so this is not a total blind spot. It is, however, not usable for
reclamation as it stands:
`DropTableEvent` carries only the user, the `NameIdentifier` and an
`isExists` flag. It does not
carry the table's `location` or its properties, and by the time the
post-event fires the metadata
is already gone, so a listener knows *which* table disappeared but has no
way to find out *which
storage* it occupied.
A listener could in principle work around this by hooking
`DropTablePreEvent`, loading the table to
capture its location, and stashing it until the post-event arrives — but
that pushes a shadow copy
of Gravitino's own metadata into every deployment, and leaves dangling state
whenever the two
events do not pair up. More fundamentally, the event mechanism is
observational: it cannot *supply*
a location on the create path, which is the other half of what we need.
Assignment and release are
two ends of the same contract, and we would rather they be expressed as one.
We hit both while running the Lance REST catalog against an internal
path-allocation service, but
nothing about the problem is Lance-specific — `calculateTableLocation`,
`dropTable` and
`purgeTable` are all shared by every table format the generic lakehouse
catalog serves.
### Describe the solution
A new SPI in the generic lakehouse catalog, deliberately kept free of any
domain-specific fields:
```java
public interface TableLocationProvider extends Closeable {
/** Name used to select this provider via the catalog property. */
String name();
void initialize(Map<String, String> catalogProperties);
/**
* Returns the location to use for the table being created, or
* {@code Optional.empty()} to fall back to Gravitino's built-in
resolution.
*/
Optional<String> provideTableLocation(TableLocationContext context);
/**
* Notifies the provider that a table whose location it assigned has been
dropped,
* so the underlying storage can be released or scheduled for reclamation.
*
* <p>Invoked only after the table metadata has been successfully removed.
See the
* ordering and failure-handling notes below.
*/
default void releaseTableLocation(TableReleaseContext context) {}
}
```
`TableLocationContext` carries only what Gravitino already knows, so
implementations can pick out
whatever their backend needs (owner, domain, tier, …) from the property maps
without Gravitino
having to model those concepts:
- `NameIdentifier` of the table (metalake / catalog / schema / table)
- table format (`lance`, …)
- the table, schema and catalog property maps
`TableReleaseContext` carries the same identifiers plus the dropped table's
resolved `location` and
its properties, and a `purge` flag distinguishing `dropTable` from
`purgeTable` so the
implementation can choose soft versus hard reclamation.
### Wiring
- Discovered with `ServiceLoader`, mirroring the existing
`LakehouseTableDelegator` /
`LakehouseTableDelegatorFactory` pattern in this same module, and
`CredentialProvider` in
`common`.
- Selected per catalog via a property, e.g. `table-location-provider =
<name>`.
- `provideTableLocation` is called from `calculateTableLocation` **only
after** the explicit
table-level `location` check, so an explicitly requested location still
wins.
- `releaseTableLocation` is called from `dropTable` and `purgeTable`.
### Drop semantics — the part we would most like feedback on
We propose the following, and want to be explicit that it is a deliberate
trade-off rather than a
fully reliable protocol:
- **Ordering: after metadata removal, never before.** If the provider ran
first and succeeded but
metadata removal then failed, the result would be a table that is still
visible in Gravitino
whose storage has been reclaimed — silent data loss. Running afterwards
means the worst case is
storage that outlives its table, which is detectable and recoverable by
reconciliation. We think
leaking is strictly preferable to losing.
- **Failure handling: log and emit a metric; do not fail the drop.** By the
time the provider is
invoked the metadata is already gone and cannot be rolled back, so
propagating the failure would
report an error for an operation that did in fact take effect. The SPI
javadoc should state
clearly that delivery is best-effort and that implementations must be able
to reconcile
independently. Building reliable delivery (outbox, retry queue) would be a
much larger change and
we do not propose it here.
If maintainers would rather have a fail-the-operation contract, or prefer
that the callback be a
separate listener-style extension rather than a method on this interface, we
are happy to go that
way instead — the ordering constraint above is the only part we feel
strongly about.
### Backward compatibility
Fully preserved, by construction:
- no `table-location-provider` configured → the current three-step chain
runs unchanged and no drop
callback fires;
- provider configured but `provideTableLocation` returns `Optional.empty()`
→ falls back to the
current chain;
- explicit table-level `location` present → the provider is never consulted;
- `releaseTableLocation` is a `default` no-op, so implementations that only
care about assignment
need not implement it.
No existing behaviour changes for any deployment that does not opt in.
### Scope for a first iteration
- **In scope:** `createTable`, `dropTable`, `purgeTable`.
- **Out of scope:** `alterTable(RenameTable)`. A rename changes the table's
identity but not its
physical path, and whether an allocator needs to know is very
implementation-dependent; we would
rather not guess. The context objects are shaped so a rename callback can
be added later without
breaking existing implementations.
- **Also out of scope:** filesets in the Hadoop catalog, which have a
structurally similar problem.
Worth a separate issue if this pattern is accepted.
### Additional context
- **Relevant code:**
`catalogs/catalog-lakehouse-generic/src/main/java/org/apache/gravitino/catalog/lakehouse/generic/GenericCatalogOperations.java`
(`calculateTableLocation`, `dropTable`, `purgeTable`);
`core/src/main/java/org/apache/gravitino/catalog/ManagedTableOperations.java`
(`purgeTable` javadoc quoted above).
- **Existing `ServiceLoader`-based extension points** we would follow for
consistency:
`LakehouseTableDelegatorFactory` (same module) and `CredentialProvider` /
`CredentialProviderFactory` (`common`, `core`).
- We searched existing issues and did not find one covering pluggable
location assignment or
release; apologies if we missed a prior discussion.
- We can help with the design doc and contribute the implementation and tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
