yihua opened a new pull request, #982:
URL: https://github.com/apache/hudi-rs/pull/982
## Description
Merges resolved the ordering field only among top-level columns, so a nested
ordering field such as `meta.ts` gave no ordering value. A merge-on-read
snapshot then let the log record replace the base row whatever its `meta.ts`,
and a copy-on-write upsert failed with a missing-column error. A shared
dotted-path resolver in `util::arrow` (split as
`HoodieUnsafeRowUtils.composeNestedFieldPath` does, with a null parent struct
reading as a null value) is now used by both file group readers and by the
write side (incoming dedupe, the cross-partition event-time check, and partial
upserts that set the enclosing struct).
Closes #919
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
A new Spark-written MOR fixture (`v9_nested_ordering_field`, Hudi 1.1.1,
Avro data and delete blocks) reproduces the issue's writes plus a stale and an
applied delete, and matches Spark on both reader versions, by key and by record
position. Native-write tests cover the MOR snapshot with a delete, COW upsert,
partial upsert of the ordering struct, and a stale cross-partition move; unit
tests cover the resolver and ordering values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]