[jira] [Created] (HUDI-1278) Need a generic payload class which can skip late arriving data based on specific fields

[Balaji Varadarajan (Jira)]

Balaji Varadarajan created HUDI-1278:
----------------------------------------

             Summary: Need a generic payload class which can skip late arriving 
data based on specific fields
                 Key: HUDI-1278
                 URL: https://issues.apache.org/jira/browse/HUDI-1278
             Project: Apache Hudi
          Issue Type: New Feature
          Components: DeltaStreamer, Spark Integration
            Reporter: Balaji Varadarajan
             Fix For: 0.6.1


Context : 
[https://lists.apache.org/thread.html/rd5d805d29c2f704d8ff2729457d27bca42e890bc01fc8e5e1f1943e3%40%3Cdev.hudi.apache.org%3E]

We need to implement a Payload class (like OverwriteWithLatestAvroPayload) 
which will skip late arriving data.

Notes:
 # combineAndGetUpdateValue() would need work
 # The ordering needs to be specified based on 1 or more fields and should be 
configurable.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)