rangareddy commented on issue #17364: URL: https://github.com/apache/hudi/issues/17364#issuecomment-5351239598
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8871). **Findings: needs the reproduction run against a current build with Hive sync enabled - I could not settle it statically.** This one is worth prioritising because the report claims a **data** issue rather than only a schema mismatch, and it comes with a complete PySpark reproduction (write `trips_table`, add a column in the middle of the column list rather than appending, then read through Hive). On `master` the sync path still funnels through `hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:492`: ```java syncClient.updateTableSchema(tableName, schema, schemaDiff); ``` Whether inserting a column mid-list produces a correct mapping in the Hive table depends on how `schemaDiff` is computed and on whether Hive resolves the underlying parquet columns by position or by name. That interaction is not decidable from the source, and getting it wrong in either direction would be a bad outcome to assert without evidence. For whoever picks this up, the questions worth answering in order: 1. Does the Hive table's column list after sync match the Hudi schema's order, or does the new column land at the end? 2. If the order differs, do reads return values from the wrong column (data issue) or just fail (schema issue)? 3. Does it reproduce on both COW and MOR, and on both the snapshot and read-optimized Hive tables? Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
