wombatu-kun commented on issue #20367: URL: https://github.com/apache/hudi/issues/20367#issuecomment-6234104912
Nested ordering fields are already a supported config on the Spark side; they fit sources that keep the event time inside a struct, such as an event or CDC envelope. The datasource and the Streamer have resolved them by path since 0.5.0 (#684, #864), HUDI-4051 (#5517) allowed them in Spark SQL "to align with Spark DataSource behaviors", and `TestInsertTable2` covers a nested `preCombineField` with both record types. With several fields, each one is resolved on its own path and the values are compared as a tuple, which is what `doGetOrderingValue` and the bulk-insert dedup from #13526 already do; this PR only makes `getOrderingValueAsJava` do the same. Unlike #19823, where the v10 writer throws, this failure is silent: a stale delete removes a newer row. Failing fast here instead would make deletes throw on tables whose upserts already order correctly. If we want to drop nested ordering fields altogether, I'd track that in a separate issue, because tables created since 0.5.0 may rely on them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
