anilkumgopalap opened a new issue, #14211:
URL: https://github.com/apache/gravitino/issues/14211
### Describe the feature
Extend the `TableLocationProvider` SPI added in #12429 (#13058) with two
optional schema-level hooks, so a provider can allocate a schema's location
when the schema is created and release it when the schema is dropped:
- `provisionSchemaLocation(SchemaLocationContext)` – called by
`GenericCatalogOperations.createSchema` before the schema is stored. Returns
the schema location, or `null` to keep the caller-supplied one.
- `unprovisionSchemaLocation(SchemaLocationContext)` – called by
`GenericCatalogOperations.dropSchema` after the schema (and, with cascade, its
tables) has been removed.
Both are `default` methods that do nothing, so
`DefaultTableLocationProvider` and existing providers are unaffected.
### Motivation
#12429 lets a deployment hand table path allocation to its own service. In
practice such services allocate paths hierarchically: the schema (database) is
registered first, with its owner and the storage domain it belongs to, and
table paths are then derived from it. Today the provider is only consulted for
tables, so:
- There is no point at which the provider can register the schema with the
external service. The only workaround is to provision it lazily on the first
table creation, which requires every table request to carry schema-level
information (owner, domain) and leaves empty schemas unregistered.
- The schema `location` the service allocates cannot be recorded on the
schema, although `DefaultTableLocationProvider` already uses the schema
location as the base path for its tables.
- Dropping a schema never releases what the service allocated for it.
### Describe the solution
```java
public interface TableLocationProvider {
// existing: name(), provisionTableLocation(...),
unprovisionTableLocation(...)
/** Allocates the location of a schema about to be created. Returns null
to keep the supplied one. */
@Nullable
default String provisionSchemaLocation(SchemaLocationContext context) {
return null;
}
/** Releases what this provider allocated for a schema that has been
dropped. */
default void unprovisionSchemaLocation(SchemaLocationContext context) {}
}
```
`SchemaLocationContext` carries the schema identifier, the schema properties
of the request (or of the stored schema on drop) and the catalog properties,
mirroring `TableLocationContext`. On drop it also reports whether the drop
cascaded and which dropped tables retained their data.
Semantics, following the existing table hooks:
- **Create:** called after request validation and before the schema is
persisted. If it throws, the schema is not created. A non-null result is stored
as the schema's `location` property. If persisting the schema fails afterwards,
`unprovisionSchemaLocation` is called on a best-effort basis.
- **Drop:** called only after the schema has been removed successfully. With
`cascade`, tables are dropped first through the existing path, so their table
hooks run before the schema hook. The context reports which dropped tables
retained their data, using the same ownership predicate the drop path already
uses (`LancePropertiesUtils.isExternalTable`, #13932), so a provider can
decline to release a schema location that still holds data. The catalog offers
no force option: whether to release is the provider's decision, consistent with
the table hooks. A failure is logged at WARN and does not fail the drop.
- **Alter:** not covered; a schema's location is immutable in this catalog.
- No lifecycle is added: providers stay constructed per catalog with no
`initialize`/`close`, as #12429 specifies.
I'm happy to contribute the implementation and tests
(`TestGenericCatalogOperations`, `TestDefaultTableLocationProvider`) and to
update `docs/lakehouse-generic-catalog.md`.
### Additional context
Follow-up to #12429 / #13058, and builds on the ownership predicate from
#13931 / #13932. Our use case is a storage registry that allocates a bucket per
schema (keyed by owning team) and table paths inside it; tables inherit owner
and domain from their schema unless overridden.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
