Danny Chen created HUDI-8316:
--------------------------------
Summary: No precombine key support for Spark write/read
Key: HUDI-8316
URL: https://issues.apache.org/jira/browse/HUDI-8316
Project: Apache Hudi
Issue Type: New Feature
Components: reader-core, writer-core
Reporter: Danny Chen
Fix For: 1.1.0
The preCombine key is widely used for ordering the Hudi payloads during both
write and read, previously in Flink, we add a keyword 'no_precombine' as the
value for scenarios that does not need the ordering(proc_time sequence)
It's valuable if we generalize this use case and support no preCombine key use
cases, just use the proc_time sequence for the payloads ordering.
The tricky part is previously we have a default value: "ts" for this field, I
think we can use proc_time sequence when the table schema does not include
"ts", and use the "ts" for ordering for backward compatibility.
There is also use case that the use wanna to force the proc_time sequence of
payloads, we can support this use case with a very specific payload:
ProcTimeAvroPayload.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
