raghav-reglobe opened a new issue, #65988: URL: https://github.com/apache/doris/issues/65988
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master (reproduced on a 2026-07-22 master build) ### What's Wrong? Querying `information_schema.tables` while switched to an external Iceberg catalog (REST catalog in our case) issues one blocking `loadTable` against the remote catalog **for every table in the catalog**, even when the query only projects cheap columns like `TABLE_SCHEMA`/`TABLE_NAME`. On a REST catalog with a few thousand tables, a single listing takes minutes and floods the catalog service with per-table GETs. Since the loads happen inside the `listTableStatus` thrift handler, clients with statement timeouts (BI tools, dashboards doing schema discovery) just see the query time out. Repeated runs pay the full cost again once the catalog metadata refresh interval passes. Measured on our cluster: a catalog with 3,882 tables took 2m52s; one with 4,365 tables took 5m38s, with thousands of catalog GETs per scan. jstack of the master FE during a `SELECT table_schema, table_name FROM information_schema.tables` scan: ``` at org.apache.iceberg.rest.RESTSessionCatalog.loadTable(RESTSessionCatalog.java:399) ... at org.apache.doris.datasource.iceberg.IcebergUtils.getIcebergTable(IcebergUtils.java:943) at org.apache.doris.datasource.iceberg.IcebergExternalTable.getIcebergTable(IcebergExternalTable.java:150) at org.apache.doris.datasource.iceberg.IcebergExternalTable.properties(IcebergExternalTable.java:429) at org.apache.doris.datasource.iceberg.IcebergExternalTable.getComment(IcebergExternalTable.java:155) at org.apache.doris.service.FrontendServiceImpl.listTableStatus(FrontendServiceImpl.java:756) ``` Root cause is the interaction of two changes: - #64263 made `IcebergExternalTable.getComment()` read the comment property from the table metadata, which requires loading the table from the remote catalog. - #64933 introduced required-columns pruning in `listTableStatus`, guarding every status column (ENGINE, CREATE_TIME, TABLE_ROWS, ...) — but the `status.setComment(table.getComment())` fill remained unconditional. So the one column whose fill is a remote round-trip is also the one column that is always computed. ### What You Expected? A projection that does not include `TABLE_COMMENT` should not load table metadata from the remote catalog, the same way it already skips row counts and the other pruned status columns. ### How to Reproduce? 1. Create an Iceberg REST catalog with a large number of tables (a few hundred is enough to see it clearly). 2. `SWITCH` to the catalog and run `SELECT table_schema, table_name FROM information_schema.tables;` 3. Observe one `loadTable` request per table on the catalog service, and minutes of latency on large catalogs. ### Anything Else? Fix appears to be a one-hunk change: guard the comment fill with the same `needTableStatusColumn(requiredColumns, "TABLE_COMMENT")` check used by the other columns. I will attach a PR. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
