2416210017 opened a new issue, #68009: URL: https://github.com/apache/doris/issues/68009
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version **`[Bug][Iceberg] Querying HMS Iceberg View fails with NoSuchIcebergTableException (type=ICEBERG-VIEW) on 4.0.8`** ### What's Wrong? ## Description Querying an Iceberg **View** stored in Hive Metastore fails on Doris 4.0.8, even though: - The catalog is a valid Iceberg HMS catalog - `enable_query_iceberg_views = true` - `SHOW CREATE TABLE` succeeds and returns the view DDL - The underlying Iceberg **tables** referenced by the view can be queried Doris recognizes the object as an external view during planning, but then loads it via **`loadTable` / `HiveTableOperations`**, which rejects `type=ICEBERG-VIEW`. ## Environment - Doris: `doris-4.0.8-rc02-bc8ea1bac6d` - Catalog: Iceberg + HMS (`type=iceberg`, `iceberg.catalog.type=hms`) - Metastore: Hive Metastore (SASL disabled / simple) - Storage: Kerberos HDFS (HA) - FE config: `enable_query_iceberg_views = true` Catalog (sensitive values redacted): ```sql CREATE CATALOG `linkhouse` PROPERTIES ( "type" = "iceberg", "iceberg.catalog.type" = "hms", "warehouse" = "hdfs://<ns>/<warehouse>", "hive.metastore.uris" = "thrift://<hms-host>:32087", "hive.metastore.sasl.enabled" = "false", "hive.metastore.authentication.type" = "simple", "hdfs.authentication.type" = "kerberos", ... ); ``` ### What You Expected? ## How to reproduce 1. Create an Iceberg HMS catalog as above. 2. In Spark/Iceberg, create an Iceberg View whose HMS table type is `ICEBERG-VIEW`, e.g. a `UNION ALL` of two Iceberg tables. 3. In Doris: ```sql SHOW FRONTEND CONFIG LIKE '%enable_query_iceberg_views%'; -- Value = true SHOW CREATE TABLE <catalog>.<db_view>.<view_name>; -- succeeds, returns CREATE VIEW ... AS SELECT ... UNION ALL ... SELECT * FROM <catalog>.<db_view>.<view_name> LIMIT 10; ``` ## Actual behavior ```text ERROR 1105 (HY000): errCode = 2, detailMessage = NoSuchIcebergTableException: Not an iceberg table: <catalog>.<db>.<view> (type=ICEBERG-VIEW) ``` Sometimes the first attempt times out in Nereids while holding table locks: ```text Nereids cost too much time (186s > 30s) plan time: {"lock_tables":186906, "analyze":-1, ...} ``` Then subsequent queries fail immediately with `NoSuchIcebergTableException`. `SHOW CREATE TABLE` works and shows a normal Iceberg view definition (SELECT + UNION ALL over two Iceberg tables). ## Expected behavior With `enable_query_iceberg_views=true` and an HMS Iceberg catalog, `SELECT` on an Iceberg View should load view metadata via `loadView` / `getIcebergView`, expand the view SQL, and query the underlying tables — as documented for Iceberg views since 3.1.0. ## Stack trace (FE) ```text java.lang.RuntimeException: NoSuchIcebergTableException: Not an iceberg table: linkhouse.dc_stg_view.retail_gms_bill_delivery_return_dtl_ods (type=ICEBERG-VIEW) at org.apache.doris.datasource.iceberg.IcebergMetadataCache.loadTableCacheValue(IcebergMetadataCache.java:153) at org.apache.doris.datasource.iceberg.IcebergMetadataCache.getSnapshotCache(IcebergMetadataCache.java:113) at org.apache.doris.datasource.iceberg.IcebergUtils.getLatestSnapshotCacheValue(IcebergUtils.java:1551) at org.apache.doris.datasource.iceberg.IcebergUtils.getSnapshotCacheValue(IcebergUtils.java:1559) at org.apache.doris.datasource.iceberg.IcebergUtils.getIcebergSchema(IcebergUtils.java:1584) at org.apache.doris.datasource.iceberg.IcebergExternalTable.getFullSchema(IcebergExternalTable.java:258) at org.apache.doris.datasource.ExternalView.getFullSchema(ExternalView.java:71) at org.apache.doris.nereids.trees.plans.logical.LogicalView.computeOutput(LogicalView.java:122) ... Caused by: org.apache.iceberg.exceptions.NoSuchIcebergTableException: Not an iceberg table: ... (type=ICEBERG-VIEW) at org.apache.iceberg.hive.HiveOperationsBase.validateTableIsIceberg(HiveOperationsBase.java:103) at org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:108) at org.apache.iceberg.hive.HiveCatalog.loadTable(...) at org.apache.doris.datasource.iceberg.IcebergMetadataOps.loadTable(IcebergMetadataOps.java:1111) ``` ## Analysis Planning already treats the object as an `ExternalView` (`LogicalView.computeOutput` → `ExternalView.getFullSchema`). When computing the view output schema, Doris still calls: `IcebergExternalTable.getFullSchema` → `IcebergUtils.getIcebergSchema` → **snapshot cache / `loadTable`** HMS reports `type=ICEBERG-VIEW`, so `HiveTableOperations.validateTableIsIceberg` throws `NoSuchIcebergTableException`. The view path should use `loadView` / `getIcebergView` (and view schema), not table snapshot loading. This looks like a regression or incomplete wiring of [#51376](https://github.com/apache/doris/pull/51376) (`support iceberg view query`) on the Nereids schema path in 4.0.8. ## Workaround Do not query the Iceberg View in Doris. Query the two base Iceberg tables (or create a Doris-side VIEW over them). Base table `SELECT` works. ## Additional notes Unrelated FE warning (Ranger policy refresh 401) appears in logs but is not the cause of `ICEBERG-VIEW` / `loadTable`. Happy to provide a minimal Spark DDL for the view if needed. ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
