yihua opened a new pull request, #20274: URL: https://github.com/apache/hudi/pull/20274
### Describe the issue this Pull Request addresses part of #16933, stacked on #20247 (review that first; this PR shows its commits until it merges) `hoodie.database.name` in `hoodie.properties` is a copy of an identity the catalog owns. It records whatever database the first writer happened to run under, goes stale on rename, re-registration or sharing across catalogs, and has twice been used as a key into a catalog with the wrong result (#20235, #16933). #20247 removed those lookups. This PR stops persisting the key. ### Summary and Changelog New tables are created without the key by the table builder, the Spark datasource and SQL paths, bootstrap, Flink, and the 3-to-4 upgrade handler, whose checksum now covers only the table name. `TableBuilder.setDatabaseName` stays as a deprecated no-op, `HoodieTableConfig.DATABASE_NAME` is deprecated after 1.3.0, and `getDatabaseName` is deprecated but still reads the key from existing tables. The remaining consumers no longer need the persisted key: the writer config validation ignores the database so a table registered under another database still accepts writes; the metaserver client requires the database from the metaserver config and fails fast without it; the table service manager client reads it from its own config and falls back to the persisted key; the Hive incremental lookup with `hoodie.incremental.use.database` takes the database from the Hive table name in the job conf when it names this table; `drop_partition` and `truncate_table` resolve an unqualified name in the session's current database instead of `default`. Meta-sync inference and Flink's operator naming are unchanged. A table re-initialized in place (`truncate_table`, an Overwrite re-init) keeps the key it already has; only new tables are created without it. The Hive incremental lookup now reads the Hive table name from the JobConf and keys its meta clients by base path, so two same-named tables in different databases no longer collide. Removing the key from existing tables is left to the next table version upgrade, which must rewrite the table checksum along with it. Tests: red/green for each change (no key written via the builder, Spark datasource, SQL and Flink; writer validation; metaserver fail-fast; table service manager database; current-database procedures; Hive incremental database; 3-to-4 change set; checksum behavior when the key is removed). ### Impact A DataFrame write that relied on meta sync inferring its target database from `hoodie.database.name` keeps working on existing tables, since the persisted key is still merged into the write options. On tables created from this release on, the database has to be passed on every write or the meta-sync database set explicitly, otherwise sync falls back to `default`. `drop_partition` and `truncate_table` with an unqualified name now act in the current database. `InputPathHandler.getTableMetaClientMap` is keyed by base path instead of table name. ### Risk Level medium. The release-note behavior above is a user-visible change for new tables; every other path has a read-only fallback to the persisted key for existing tables, and the key is deprecated rather than removed. Verified by the red/green tests listed and compiling every touched module on Spark 3.5 and Flink 2.1. ### Documentation Update Config documentation for `hoodie.database.name` says the key is no longer written; the release note above belongs in the next release notes. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
