drexler-sky commented on code in PR #16891:
URL: https://github.com/apache/iceberg/pull/16891#discussion_r3479427159
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java:
##########
@@ -368,6 +369,31 @@ public Identifier[] listTables(String[] namespace) {
.toArray(Identifier[]::new);
}
+ @Override
+ public TableSummary[] listTableSummaries(String[] namespace) {
+ // Build summaries directly from the catalog listings to avoid loading
every table, which the
+ // default TableCatalog.listTableSummaries implementation would do.
Iceberg tables are always
+ // reported as EXTERNAL (see SparkTable#properties), and views are
reported as VIEW.
+ List<TableSummary> summaries = Lists.newArrayList();
+
+ // Collect views first. Most catalogs return only tables from listTables,
but HiveCatalog with
+ // list-all-tables=true returns every metastore entry, including views.
De-duplicate against the
+ // view identifiers so a view is never also reported as a table.
+ Set<Identifier> viewIdents = Sets.newHashSet(listViews(namespace));
+
+ for (Identifier ident : listTables(namespace)) {
Review Comment:
Right, the default implementation does a loadTable per identifier just to
read the table_type property. Avoiding that per-table load is the whole point
of this override. Every Iceberg table is EXTERNAL and every view is VIEW, so we
can build the summaries directly from listTables/listViews without loading
anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
