danhuawang opened a new issue, #11709:
URL: https://github.com/apache/gravitino/issues/11709
### What would you like to be improved?
Tag and Policy operations (`listTags`, `listPolicies`, etc.) fail with
`NoSuchMetadataObjectException` when targeting schema/table objects that belong
to an Iceberg catalog with `catalog-backend=rest`. The same operations work
correctly when the catalog uses `catalog-backend=jdbc` (or any backend where
Gravitino directly manages metadata).
**Error:**
```
NoSuchMetadataObjectException: Failed to list tags for metadata object
MetadataObject:
[fullName=iceberg_rest_view_oauth2.iv_16f5c53e_db.tc_p0_01_base], [type=TABLE]
due to not found
Caused by: NoSuchEntityException: No such schema entity: iv_16f5c53e_db
at
SchemaMetaService.getSchemaIdByMetalakeNameAndCatalogNameAndSchemaName(SchemaMetaService.java:111)
at
RelationalEntityStoreIdResolver.getEntityIdsRequiringSchemaIds(RelationalEntityStoreIdResolver.java:174)
at
RelationalEntityStoreIdResolver.getEntityId(RelationalEntityStoreIdResolver.java:108)
...
at TagMetaService.listTagsForMetadataObject(TagMetaService.java:182)
at TagManager.listTagsInfoForMetadataObject(TagManager.java:260)
```
**Root cause:**
`TagManager` (and similarly `PolicyManager`) resolves the target metadata
object's entity ID chain via `RelationalEntityStoreIdResolver`, which queries
the Gravitino entity store (relational DB) for:
```
metalake → catalog → schema → table
```
For a `catalog-backend=rest` Iceberg catalog, Gravitino acts as a **proxy**
— schema/table CRUD operations are forwarded to the remote Iceberg REST service
(IRC). Gravitino's own entity store does **not** persist entity records for
these schemas and tables. As a result,
`SchemaMetaService.getSchemaIdByMetalakeNameAndCatalogNameAndSchemaName` finds
no record and throws `NoSuchEntityException`.
For `catalog-backend=jdbc` (or Hive, etc.), Gravitino owns the metadata
directly, so schemas and tables have entity store records, and the entity ID
resolution succeeds.
**Impact:**
The following features are completely non-functional for REST-backend
Iceberg catalogs:
- `listTags` / `associateTag` / `getTag` on schema or table objects
- `listPolicies` / `associatePolicy` on schema or table objects
- Any other operation that relies on `RelationalEntityStoreIdResolver` to
look up a schema/table by name
This affects users who set up Gravitino with an Iceberg REST backend (a
common deployment pattern for data lake architectures).
### How should we improve?
The entity ID resolution layer assumes all metadata objects have a
corresponding record in Gravitino's entity store. This assumption is violated
by proxy-style backends (REST, and potentially others).
Possible approaches:
1. **Sync entity records on proxy operations (recommended):** When Gravitino
proxies `createSchema` / `createTable` / `loadTable` etc. for a REST backend,
it should also persist lightweight entity records in its own store. This
ensures the entity ID chain is always resolvable for tag/policy/authorization
operations. The entity store would act as a local "shadow" of the remote
catalog's metadata. This is consistent with how authorization already needs to
resolve these objects.
2. **Dynamic fallback in entity resolution:** When the entity store lookup
fails, the resolver could fall back to querying the catalog backend directly to
confirm the object exists, then either create an on-the-fly entity ID or lazily
persist the record.
3. **Lazy sync on first tag/policy operation:** Instead of syncing eagerly
on every CRUD operation, sync the entity record only when a tag/policy
operation first targets an object that has no local entity. This minimizes
write amplification but introduces latency on the first governance operation.
Option 1 is the cleanest long-term solution — it ensures all governance
features (tags, policies, authorization, audit) work uniformly regardless of
the catalog backend type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
