yihua opened a new pull request, #20272: URL: https://github.com/apache/hudi/pull/20272
### Describe the issue this Pull Request addresses part of #16933, stacked on #20247 (review that first; this PR shows its commits until it merges) `CatalogBackedTableMetadata`, the opt-in partition listing from the catalog, built its identifier from `hoodie.database.name` in the table config, or the session's current database when absent, so it listed the wrong entry, or none, once that copy was stale or the table was registered under another name. It also trusted the catalog's partition list unconditionally, so a partition meta sync had not recorded yet was silently dropped from query results. ### Summary and Changelog The listing now keys on the catalog identity options that #20247 passes down from the Spark catalog integration. With the listing option on and no identity present (a path read, a procedure, a table-valued function) it logs once at INFO and lists as if the option were off; it never guesses an identifier. It uses the catalog's partitions only while the entry's `last_commit_time_sync` (and completion time, when recorded) match the table's latest completed commit, otherwise it lists the file system; table-service commits and conditional sync trigger that fallback by design. Tests: `TestCatalogBackedTableMetadata` (no-identity fallback, fresh entry used, catalog name and database differing from the table name) and `TestPartitionPushDownWhenListingPaths` (stale catalog falls back; failed before with a dropped partition). ### Impact A read with the listing option enabled but no catalog identity now lists the file system instead of guessing a database. Reads through `HoodieCatalog` keep using the catalog while its sync markers match the latest commit. ### Risk Level low. Behavior changes only for an opt-in option, and only toward the file-system listing used when the option is off. ### Documentation Update The option's documentation is updated in `DataSourceOptions`. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
