rangareddy opened a new pull request, #19289: URL: https://github.com/apache/hudi/pull/19289
### Describe the issue this Pull Request addresses Closes #17359 (HUDI-8843, follow-up of #11922 which fixed the same problem for AWS Glue). When `hoodie.datasource.hive_sync.sync_comment=true`, comments of partition columns were never synced to the Hive metastore: - **hms** sync mode: `HMSDDLExecutor.updateTableComments` only updated the columns of the storage descriptor and ignored the table's partition keys, so partition column comments were silently dropped. - **jdbc/hiveql** sync modes: the generated `ALTER TABLE ... CHANGE COLUMN` statement targeted the partition column, which Hive rejects with `Invalid column reference`, failing the entire sync. - Additionally, `createTable` never populated any comments (regular or partition columns), so comments only appeared from the second sync onward. ### Summary and Changelog With `sync_comment=true`, column and partition column comments are now synced to HMS when the table is created, and partition column comments are kept in sync afterwards in `hms` sync mode. - `HMSDDLExecutor.createTable`: populates column and partition column comments from the storage schema field docs when `sync_comment=true`. - `HMSDDLExecutor.updateTableComments`: also updates comments of the table's partition keys (comment-only changes pass HMS alter-table validation, which compares partition key names/types only). This matches the behavior of `AWSGlueCatalogSyncClient.updateTableComments`. - `HiveSchemaUtil.generateCreateDDL` (used by jdbc/hiveql sync modes): appends `COMMENT '...'` clauses to both regular and partition column definitions when `sync_comment=true`. - `QueryBasedDDLExecutor.updateTableComments`: skips partition columns with a warning instead of emitting an invalid `ALTER TABLE ... CHANGE COLUMN` statement, since HiveQL has no DDL to alter partition column comments. This fixes the jdbc-mode sync failure; comments are still applied at create time. - Tests: new `testSyncCommentsForPartitionColumns` (create path, all sync modes expect comments) and `testUpdateCommentsForPartitionColumns` (alter path, `hms` expects the partition comment, query-based modes gracefully skip) in `TestHiveSyncTool`, with a new fixture schema `partition-doced-test.avsc` whose partition field has a doc. ### Impact Users get column and partition column comments in HMS from the first sync when `hoodie.datasource.hive_sync.sync_comment=true`. No change when the config is disabled (default). Fixes a sync failure in jdbc/hiveql modes when the partition field has a doc in the table schema. ### Risk Level Low. All changes are gated behind `hoodie.datasource.hive_sync.sync_comment` (default `false`), except the partition-column skip in `QueryBasedDDLExecutor.updateTableComments` which turns a guaranteed sync failure into a warning. Verified with `TestHiveSyncTool` comment tests across hms/jdbc/hiveql sync modes and `TestHiveSchemaUtil`. ### Documentation Update None. No new configs; behavior now matches the existing description of `hoodie.datasource.hive_sync.sync_comment`. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
