wombatu-kun opened a new pull request, #20346: URL: https://github.com/apache/hudi/pull/20346
### Describe the issue this Pull Request addresses Closes #20345 Tables below version 9 recorded their ordering fields only as `hoodie.table.ordering.fields`, which Hudi 0.x (version 6) and 1.0.x (version 8) do not read, and `ALTER TABLE RENAME` dropped `hoodie.table.precombine.field` from tables that had it. ### Summary and Changelog - `HoodieTableConfig.create` also records the ordering field of a table below version 9 under `hoodie.table.precombine.field`, the key its readers know and the one the 9 to 8 downgrade writes. Those readers take that key as one field name, so it is written only when a single ordering field is declared, with that field, whatever the payload. Table creation and every re-creation from the table's own properties go through it. - `TestHoodieTableMetaClient` pins both keys per table version, for a new table, for one ordered on two fields, for a `MySqlDebeziumAvroPayload` table, and for one re-created from properties that carry only the legacy key. A Spark SQL test does the same through CREATE TABLE and ALTER TABLE RENAME at versions 6 and 8. ### Impact Tables created or re-created below version 9 with a single ordering field carry `hoodie.table.precombine.field` next to `hoodie.table.ordering.fields`, with the same value, so 0.x and 1.0.x find their ordering field again and order records on the same field as 1.x. Tables ordered on several fields keep only the new key, as before, since 1.0.2 would otherwise fail MOR merges with `Field: ts,seq does not exist in the table schema` instead of warning. 1.x reads `hoodie.table.ordering.fields` first, so its behavior is unchanged, and the 8 to 9 upgrade still removes the legacy key. Tables created before this fix keep only the new key until they are re-created. #20276 pins the exact table configs of versions 6 and 8, so whichever of the two lands second adds the legacy key there. ### Risk Level low The change adds one key below version 9, only next to a single declared ordering field and with the same value, and leaves every value 1.x reads as it was. Verified: the single-field tests fail on master with `precombine key at version 6 ==> expected: <ts> but was: <null>`, and Hudi 0.15.0 and 1.0.2 resolve `getPreCombineField()` to the ordering field on single-field tables written with this change (null without it). ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
