csarushan1729 opened a new pull request, #987: URL: https://github.com/apache/incubator-xtable/pull/987
Fixes #978 ## What is the purpose of the pull request When a Hudi table has `hoodie.schema.on.read.enable=true` and a column is renamed, Hudi keeps a stable field ID via its InternalSchema. Previously XTable only consumed the Avro schema (which has no stable field IDs), so the Iceberg target treated the rename as a drop + add (new field ID). As a result, pre-rename Parquet files returned NULL for the renamed column even though the sync reported SUCCESS. This PR makes XTable prefer Hudi’s InternalSchema (when present) and convert it to XTable’s InternalSchema while preserving the original field IDs. Iceberg then records a proper rename and keeps the old physical name as a name-mapping alias, so older files remain readable. ## Brief change log - Added `HudiInternalSchemaConverter` that converts Hudi `InternalSchema` → XTable `InternalSchema` while retaining field IDs - Updated `HudiTableExtractor` to prefer Hudi InternalSchema (via the public `getTableInternalSchemaFromCommitMetadata()`) and fall back to the existing Avro path when InternalSchema is absent - No behavioural change for tables that never enabled schema-on-read ## Verify this pull request This change added tests and can be verified as follows: - Unit / compilation coverage for the new converter and the updated extraction paths - Manual verification against the reproduction steps in #978 (Hudi COW table with schema-on-read rename → sync to Iceberg → pre-rename rows should no longer be NULL) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
