mehakmeet opened a new issue, #11059:
URL: https://github.com/apache/gravitino/issues/11059
### What would you like to be improved?
The Iceberg REST `POST /v1/.../namespaces/{ns}/register` endpoint in
Gravitino supports first-time registration only. There is no way to re-point an
existing table at a new metadata.json in a single atomic operation.
The current workaround — dropTable(purge=false) followed by registerTable —
is non-atomic (a query between the two steps sees the table missing) and
corrupts Gravitino's entity bindings. `IcebergTableHookDispatcher.dropTable`
unconditionally deletes the TableEntity (does not gate on purgeRequested), so
the next `registerTable` mints a new table_id and orphans every
role/owner/tag/policy binding keyed on the old table_id.
This blocks control-plane use cases that need atomic pointer updates:
Iceberg metadata replication, multi-region sync, DR/restore from a known-good
metadata.json.
## Considered alternatives
- **Wait for Iceberg release with `Catalog.registerTable(id, location,
overwrite)`
in a release**: feasible, but the convenience method is sugar over
primitives
already public in 1.10.1. Same atomicity, same backend dispatch.
Composition can
simplify to a one-line delegate when the Iceberg pin moves.
- **Add this to Gravitino's native admin API
(`/api/metalakes/.../tables/register`)
instead of extending IRS**: arguably a better fit for control-plane
operations,
since IRS positions itself as Iceberg-spec-compliant. This issue focuses
on the IRS-side change because it's the smaller scoped extension and aligns
with consumers already using IRS. Open to this approach too, but a bigger
change.
### How should we improve?
Add overwrite to the registerTable flow:
overwrite=false (default): existing behavior; AlreadyExistsException if the
table already exists.
overwrite=true: atomic in-place update of the metadata pointer.
Gravitino entity preserved — table_id unchanged, role/owner/tag/policy
bindings intact, only audit fields (last_modifier, last_modified_time) updated.
This can land on the current Iceberg 1.10.1 pin without bumping. The 3-arg
Catalog.registerTable(id, location, overwrite) exists on Iceberg main but is
not in a release yet; the primitives it composes — tableExists, loadTable,
HasTableOperations.operations(), TableOperations.refresh(),
TableMetadataParser.read(), TableOperations.commit(base, target) — are all
public API in 1.10.1. Atomicity is provided by TableOperations.commit(base,
target).
Changes affect IcebergCatalogWrapper (compose primitives),
IcebergNamespaceOperationExecutor (pass overwrite through), and
IcebergNamespaceHookDispatcher (preserve entity on overwrite path; update audit
only). When the Iceberg pin moves to a release containing the convenience
signature, the composition can simplify to a one-line delegate without API or
behavior change.
API surface: Iceberg's `RegisterTableRequest` (1.10.1) has only `name` and
`metadataLocation`, no `overwrite`. We can expose `overwrite` as a query
parameter on the existing endpoint: `POST
/v1/.../namespaces/{ns}/register?overwrite=true`. Default `false` preserves
current behavior; standard Iceberg REST clients (Spark, Trino) are unaffected.
When the Iceberg pin moves to a release where `RegisterTableRequest` gains an
`overwrite` field, the query param can be deprecated in favor of the body field
(or both can coexist).
registerView is a related but separate gap (no REST handler exists in
Gravitino today, and Iceberg's ViewCatalog.registerView is also not yet in a
release) — should be tracked as its own issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
