mchades commented on code in PR #10998:
URL: https://github.com/apache/gravitino/pull/10998#discussion_r3252829804
##########
catalogs/catalog-hive/src/main/java/org/apache/gravitino/catalog/hive/HiveCatalogOperations.java:
##########
@@ -398,6 +408,17 @@ public NameIdentifier[] listTables(Namespace namespace)
throws NoSuchSchemaExcep
catalogName, schemaIdent.name(), hudiFilter,
MAX_TABLES));
removeHudiTables(allTables, hudiTables);
}
+
+ // Always filter out VIRTUAL_VIEW entries so they don't appear in table
listings
+ String viewFilter =
+ String.format("%stableType like \"VIRTUAL_VIEW\"",
HIVE_FILTER_FIELD_PARAMS);
+ List<String> views =
+ clientPool.run(
+ c ->
+ c.listTableNamesByFilter(
+ catalogName, schemaIdent.name(), viewFilter,
MAX_TABLES));
+ allTables.removeAll(views);
Review Comment:
No, we can not. As noted in the previous code comment (line 377), filtering
in the first HMS query may miss tables when metadata is incomplete, so I kept
the current two-step approach.
> // When a table is created using the HMS interface without specifying the
`tableType`,
> // although Hive treats it as a `MANAGED_TABLE`, it cannot be
queried through the `getTable`
> // interface in HMS with the specified `tableType`. This is because
when creating a table
> // without specifying the `tableType`, the underlying engine of HMS
does not store the
> // information of `tableType`. However, once the `getTable`
interface specifies a
> // `tableType`, HMS will use it as a filter condition to query its
underlying storage and
> // these types of tables will be filtered out.
> // Therefore, in order to avoid missing these types of tables, we
need to query HMS twice. The
> // first time is to retrieve all types of table names (including the
missing type tables), and
> // then based on
> // those names we can obtain metadata for each individual table and
get the type we needed.
> List<String> allTables = clientPool.run(c ->
c.getAllTables(catalogName, schemaIdent.name()));
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
