nevzheng opened a new issue, #11943: URL: https://github.com/apache/gravitino/issues/11943
### What would you like to be improved? **Purpose of this issue: document a known quirk and open discussion on how (or whether) to guard against it — not to mandate a specific fix.** **Suggested priority: P3 (usability / `minor`).** No data loss, no blocked functionality, and a trivial workaround exists (remove the property). The cost is purely a confusing first-run experience. Could be argued up to P2 given it's demo-visible and hits anyone copying a hive/jdbc example. On the `lakehouse-iceberg` catalog with `catalog-backend=rest`, the `warehouse` property is a **catalog selector (a name)** — it's forwarded to the remote IRC as `GET /v1/config?warehouse=<value>`. On the `hive`/`jdbc` backends the same property is a **storage-location URI**. Because the name means two different things by backend, users copying a hive/jdbc example naturally set a URI like `warehouse: "s3://warehouse/"` on a REST catalog. The catalog is then **created without complaint** (the value is accepted, never validated), and the failure is deferred to the first table/namespace operation: ``` org.apache.iceberg.exceptions.NoSuchWarehouseException: Couldn't find Iceberg configuration for catalog s3://warehouse/ ``` The error gives no hint that the fix is to remove the property (or set it to a catalog *name*). The value is forwarded verbatim: the REST branch of the required-check is skipped ([`IcebergCatalogWrapper.java:93`](https://github.com/apache/gravitino/blob/main/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ops/IcebergCatalogWrapper.java#L93)), and the receiving IRC treats it as a catalog name ([`IcebergConfigOperations.java:128`](https://github.com/apache/gravitino/blob/main/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergConfigOperations.java#L128) -> [`IcebergCatalogWrapperManager.java:123`](https://github.com/apache/gravitino/blob/main/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/IcebergCatalogWrapperManager.java#L123)). #### Reproduction (Docker quickstart) Environment: `apache/gravitino:latest` (native API on `:8090`) + `apache/gravitino-iceberg-rest:latest` (IRC on `:9001`), as reported on the `gravitino-irc-quickstart` setup. ```bash # Register a REST-backend catalog with a URI-shaped warehouse (the natural mistake # when copying a jdbc/hive example). This SUCCEEDS — nothing rejects it. curl -s -X POST http://localhost:8090/api/metalakes/v3check/catalogs \ -H 'Accept: application/vnd.gravitino.v1+json' \ -H 'Content-Type: application/json' \ -d '{ "name": "irc_probe", "type": "RELATIONAL", "provider": "lakehouse-iceberg", "comment": "probe with URI warehouse", "properties": { "catalog-backend": "rest", "uri": "http://gravitino-irc:9001/iceberg";, "warehouse": "s3://warehouse/" } }' # -> HTTP 200, catalog created. # Any subsequent operation triggers the IRC handshake and fails: curl -s http://localhost:8090/api/metalakes/v3check/catalogs/irc_probe/schemas # -> {"code":1006,"type":"NoSuchWarehouseException", # "message":"... Couldn't find Iceberg configuration for catalog s3://warehouse/"} # Removing 'warehouse' (or setting it to a real catalog name) works: curl -s -X POST http://localhost:8090/api/metalakes/v3check/catalogs \ -H 'Accept: application/vnd.gravitino.v1+json' \ -H 'Content-Type: application/json' \ -d '{ "name": "irc_probe_ok", "type": "RELATIONAL", "provider": "lakehouse-iceberg", "properties": { "catalog-backend": "rest", "uri": "http://gravitino-irc:9001/iceberg"; } }' curl -s http://localhost:8090/api/metalakes/v3check/catalogs/irc_probe_ok/schemas # -> 200 ``` #### Tests that expose the issue The server-side behavior is already covered — a URI `warehouse` is just another unknown catalog name and returns `404` (which the Iceberg REST client rethrows as `NoSuchWarehouseException`). Adding the URI case to the existing parameterized test makes it explicit: ```java // iceberg/iceberg-rest-server/.../rest/TestIcebergConfig.java (testConfigWithNonExistentWarehouses) @ValueSource(strings = {"invalid-catalog", "warehouse_123", "s3://warehouse/"}) // s3:// -> 404 ``` Verified locally: the added `"s3://warehouse/"` case passes (returns 404) unchanged, confirming the value is treated as a catalog name, not a location. Related: #5756 made `warehouse` optional for REST (fixed); #9810 proposes hiding it in the Web UI; PR #9811 documented the distinction. This issue covers the remaining gap — the **REST-API / catalog-init path gives no runtime hint** when a URI-shaped value is used. ### How should we improve? Filing to surface the quirk and gather maintainer input. Some options, in ascending effort / intrusiveness: - **Option A — Improve the error message only.** Catch `NoSuchWarehouseException` on the REST path and append: "on the REST backend, `warehouse` selects a catalog by name; remove it or set a catalog name." Low effort, but the hint arrives late (first operation, not creation) and threads through Iceberg's exception. - **Option B — WARN at catalog init when `warehouse` is URI-shaped.** In the branch that already special-cases REST ([`IcebergCatalogWrapper.java:93`](https://github.com/apache/gravitino/blob/main/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/ops/IcebergCatalogWrapper.java#L93)): ```java } else if (IcebergCatalogBackend.REST.equals(catalogBackend)) { String warehouse = icebergConfig.get(IcebergConfig.CATALOG_WAREHOUSE); if (StringUtils.isNotBlank(warehouse) && warehouse.contains("://")) { LOG.warn( "The 'warehouse' value '{}' looks like a storage-location URI, but for the REST " + "catalog backend 'warehouse' selects a catalog by name. It is sent to the Iceberg " + "REST server as a catalog selector and will likely fail with " + "NoSuchWarehouseException. Remove 'warehouse' to use the default catalog, or set " + "it to the target catalog's name.", warehouse); } } ``` Non-breaking, fires at creation, correctly scoped (hive/jdbc take the other branch). Pairs with a unit test asserting construction succeeds (no throw) plus the URI case in `TestIcebergConfig`. - **Option C — Reject a URI-shaped `warehouse` at creation.** Fail fast with a clear message. Best UX for the common case, **but** `warehouse` is a server-defined opaque identifier per the Iceberg REST spec, and some non-Gravitino IRC servers legitimately use URI/ARN-shaped identifiers — a hard reject risks breaking valid federation. - **Option D — Hide/omit `warehouse` for REST in the Web UI.** Tracked separately in #9810; complements but doesn't cover the REST-API path. **Leaning toward Option B** as the smallest non-breaking change that fires early and gives the exact hint the raw error omits — while avoiding Option C's risk to the legitimate opaque-identifier case. But the intent here is to document the quirk and discuss; happy to go with whatever the maintainers prefer (including doc-only, since PR #9811 already covers part of it). *Reported by Mark Hoerth; filed on his behalf.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
