wombatu-kun opened a new pull request, #20368: URL: https://github.com/apache/hudi/pull/20368
### Describe the issue this Pull Request addresses Closes #20367 With the Spark record type, a delete written into an inline delete block lost the value of a nested ordering field. As a result, a stale delete removed a newer row. ### Summary and Changelog - `HoodieSparkRecord.getOrderingValueAsJava` now resolves an ordering field by its nested path, the way `doGetOrderingValue` already does, instead of checking only the top-level fields. A field that does not exist still gets the default ordering value. - `TestMORDataSource` writes a stale delete and a newer delete against a nested `bigint`, `string` and `decimal` ordering field at versions 8 and 9, and checks that only the newer one applies. The Avro record type serves as the reference. ### Impact Spark-record-type writes to MOR tables at versions 8 and 9 now keep the value of a nested ordering field in their delete blocks, so event-time ordering applies to deletes as it does to updates. Version 9 is the default of 1.2.x. Top-level ordering fields resolve as before. `writeDelete` of the inline log handle is the only caller. Version 10 native logs and the Avro record type already resolved the field by path. With #20341, deletes that win precombine take this path too. ### Risk Level low The lookup now resolves the field by its path, the same lookup `getColumnValueAsJava` already used to read the value. Verified: - On master, the Spark rows of the new test fail with `expected: <List(id1)> but was: <WrappedArray()>`. - With the fix, the new test and the other 115 tests of `TestMORDataSource` pass. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
