mchades opened a new pull request, #11193:
URL: https://github.com/apache/gravitino/pull/11193
### What changes were proposed in this pull request?
- Fix a `startsWith` over-removal bug in `HiveCatalogOperations#listTables`
that
incorrectly dropped tables whose names happened to start with a Hudi base
table
name (e.g. `<base>_root`).
- Extract the non-Hive filtering logic into a focused private method and
consolidate filter-string construction.
- Document the Hive Metastore `listTableNamesByFilter` limitation (dot-free
parameter keys only) and provide a user-side workaround on the
`list-all-tables` catalog property and in the Hive catalog docs.
### Why are the changes needed?
`HiveCatalogOperations#listTables` filters out non-Hive tables (Iceberg,
Paimon,
Hudi) via `IMetaStoreClient#listTableNamesByFilter`. Two issues:
1. Spark-managed Hudi tables registered via `saveAsTable` only set
`spark.sql.sources.provider=hudi` — a dotted key the HMS filter grammar
cannot match — so they leak into the listing.
2. The `_ro` / `_rt` cleanup used `startsWith`, removing unrelated tables
that
merely shared a prefix with a Hudi base table.
(1) cannot be fully fixed server-side (HMS limitation, confirmed against Hive
`ExpressionTree` and Trino's `HiveUtil`); (2) is fixed here, and the gap from
(1) is now clearly documented with a one-line user-side workaround.
Fix: #10561
### Does this PR introduce _any_ user-facing change?
- Updated description of the `list-all-tables` catalog property.
- New "Known limitation" + "Workaround" section in
`docs/apache-hive-catalog.md`.
- No API or behavior change for callers using the default
`list-all-tables=false`.
### How was this patch tested?
- `./gradlew :catalogs:catalog-hive:test -PskipITs --tests
"org.apache.gravitino.catalog.hive.TestHiveCatalogOperations"` passes.
- Manual review of generated filter strings against Hive `ExpressionTree`
JDOQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
