yihua opened a new pull request, #20247: URL: https://github.com/apache/hudi/pull/20247
### Describe the issue this Pull Request addresses part of #16933, follows #20240 `hoodie.database.name` in `hoodie.properties` is a copy of an identity the catalog owns, and the Spark read path used it, or the session's current database when absent, as a key into the catalog for the simple bucket index options. That reads the wrong entry, or none, once the copy is stale or the table is registered under another name. The catalog-backed partition listing has the same defect and is fixed separately in #20272, stacked on this PR. ### Summary and Changelog The catalog integration now passes the identity Spark resolved down to the read path as two internal options, `hoodie.datasource.read.catalog.table.database` and `hoodie.datasource.read.catalog.table.name`, set by `HoodieInternalV2Table.v1TableWrapper` (storage properties, which Spark forwards as options) and by each Spark version's V2-to-V1 fallback; `HoodieCatalogTable.catalogProperties` strips them so they never reach write configs or TBLPROPERTIES. The bucket index lookup in `HoodieFileIndex` keys on that identity and skips the merge when it is absent; `getDatabaseName` and the session fallback are gone. `upgrade_table` and `downgrade_table` pass a database only for a qualified table name instead of stamping the session's current one. Unread copies of the database name in `HoodieCatalogTable` and `FileSystemBackedTableMetadata` are removed. Tests: `TestHoodieFileIndex` (identity used, no-identity skip, failure path) and `TestUpgradeOrDowngradeProcedure` (no session database in the write config). All fallbacks compile on Spark 3.3 through 4.2. This is step one of #16933: the config stays, but nothing keys a catalog operation on it any more. Step two removes it from the table config. ### Impact A bucket index read through `HoodieCatalog` merges the bucket options of the entry it was resolved from; a read with a declared bucket index but no catalog identity no longer looks anything up. No other user-facing change. ### Risk Level low. The identity options are internal and stripped from table properties. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
