liangyouze commented on issue #5936:
URL: https://github.com/apache/gravitino/issues/5936#issuecomment-5065565030
I'd like to pick this up and propose a concrete design. This was also
independently requested in #10607, which suggests putting the metalake into the
request URI path — the proposal below follows that direction. Feedback welcome
before I start on the first PR.
Goal
Allow a single Iceberg REST server (both auxiliary and standalone mode) to
serve catalogs from multiple metalakes, with authentication and authorization
working correctly per metalake. Existing single-metalake deployments must keep
working unchanged.
Design overview
The core change is moving the metalake from a process-level static value
(resolved once at startup from gravitino.iceberg-rest.gravitino-metalake) to a
request-level value resolved from the URL prefix.
## 1. Prefix encoding (no wire-protocol change)
Reuse the existing single-segment prefix, encoded as {metalake}.{catalog},
e.g.:
` /iceberg/v1/mymetalake.mycatalog/namespaces/...`
Clients simply set warehouse=mymetalake.mycatalog. The current prefix
pattern {prefix:([^/]*/)?} already accepts dots, and /v1/config already echoes
the warehouse back as the prefix, so no client-side or REST-spec change is
needed. A prefix without a dot keeps today's semantics (catalog under the
default metalake), which gives us backward compatibility for free.
(Alternative considered: a two-segment path /v1/{metalake}/{catalog}/....
Rejected because the Iceberg REST spec models the prefix as a single opaque
segment, and multi-segment prefixes break the existing JAX-RS routes and
clients.)
## 2. Request context
Add a metalakeName field to IcebergRequestContext, resolved from the
prefix at the REST layer (one shared helper, e.g.
IcebergRESTUtils.parseMetalakeAndCatalog(prefix)). All downstream layers read
the metalake from the context instead of
IcebergRESTServerContext.getInstance().metalakeName(), which is demoted to
defaultMetalakeName() (used when the prefix carries no metalake).
## 3. Config provider SPI (backward compatible)
Add a default method to IcebergConfigProvider:
```
default Optional<IcebergConfig> getIcebergCatalogConfig(String
metalakeName, String catalogName) {
return getIcebergCatalogConfig(catalogName); // ignore metalake by
default
}
```
Existing third-party providers keep compiling and behaving as before.
DynamicIcebergConfigProvider implements the new method: the CatalogFetcher
becomes per-metalake (trivial for the aux-mode internal fetcher; the standalone
HTTP fetcher keeps a small cache of per-metalake GravitinoClients, since the
client binds a metalake at build time).
## 4. Authorization path
Three places currently pin the static metalake and would resolve it per
request instead:
- IcebergMetadataAuthorizationMethodInterceptor — builds all
NameIdentifiers (metalake/catalog/schema/table) and runs checkCurrentUser
against the static metalake. The prefix is already available as an annotated
parameter, so the interceptor parses (metalake, catalog) from it.
User-existence check and privilege evaluation then run against the requested
metalake.
- LoadTableAuthzHandler / LoadViewAuthzHandler /
IcebergLoadAuthzHandlerHelper — same change.
- The hook dispatchers (IcebergTableHookDispatcher,
IcebergViewHookDispatcher, IcebergNamespaceHookDispatcher) plus
IcebergOwnershipUtils, IcebergCleanupHelper, IcebergOrphanSchemaCleanup, and
the event dispatchers — all take the metalake from IcebergRequestContext
instead of a constructor-injected static value. This makes entity import and
ownership assignment land under the correct metalake.
## 5. Security
Since the metalake now comes from the URL, add an allowlist config, e.g.
gravitino.iceberg-rest.allowed-metalakes (default: only the configured
gravitino-metalake, preserving current behavior; * opts into all). Combined
with the per-metalake checkCurrentUser, a user can only reach metalakes they
belong to.
## 6. Misc
- IcebergCatalogWrapperManager cache keys become metalake-qualified; the
aux-mode catalog-cache-invalidation listener matches on the ident's metalake
instead of the single configured one.
- Credential-vending refresh endpoints already embed the raw prefix, so
they keep working.
- Events/audit will carry the request-level metalake as a natural
consequence of the context change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
