prathik-onehouse opened a new pull request, #20747:
URL: https://github.com/apache/hudi/pull/20747

   ### Describe the issue this Pull Request addresses
   
   Closes #20746
   
   Tables below version 9 that a 1.x writer created before #20346 still lack 
`hoodie.table.precombine.field`, so 0.x and 1.0.x see no ordering field on 
them. Stacked on #20346, review that first.
   
   ### Summary and Changelog
   
   The legacy key now also reaches existing tables. #20346's rule moves from 
`HoodieTableConfig.create` into `HoodieTableConfig.getMissingPrecombineField`, 
next to `getMissingKeyGeneratorClass`. The write client's table config 
backfill, which already restores the key generator class below version 8, adds 
the precombine key on the next write, in the same update of 
`hoodie.properties`; the commit-path check allocates nothing for a table that 
carries every key. The 8 to 7 and 7 to 6 downgrades add it too, so a table 
rolled back to 0.x does not depend on a later 1.x write. Tests cover the 
helper, the downgrade handlers, an 8 to 6 downgrade procedure, and the backfill 
on the next write through Spark SQL and Hudi Streamer.
   
   ### Impact
   
   0.x and 1.0.x order records on the same field as 1.x on existing tables 
below version 9 after their next 1.x write or a downgrade. Such a table gets 
one extra `hoodie.properties` update on its first commit; tables at version 9 
and above, and tables ordered on several fields, are unchanged.
   
   ### Risk Level
   
   low
   
   The change only adds a missing key below version 9, with the value 1.x 
already orders on, through the existing backup-and-verify update of 
`hoodie.properties`. Unlike #20346, `create` no longer overwrites a precombine 
key the properties already carry, which only matters for the repair-props 
commands, which re-create `hoodie.properties` from a user-supplied file.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] fix(core): backfill the leg... prathik-onehouse (via GitHub) via commits

Reply via email to