[
https://issues.apache.org/jira/browse/HUDI-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-6188:
---------------------------------
Labels: pull-request-available (was: )
> Unify the logic of intra-partition upsert and cross-partition upsert in flink
> state index.
> ------------------------------------------------------------------------------------------
>
> Key: HUDI-6188
> URL: https://issues.apache.org/jira/browse/HUDI-6188
> Project: Apache Hudi
> Issue Type: Improvement
> Components: index
> Reporter: Ying Lin
> Assignee: Ying Lin
> Priority: Minor
> Labels: pull-request-available
>
> Now when partitioning upsert, according to {{precombine.field}} parameter,
> keep the record with the largest value after upserting.
> This is widely used to solve the case of out-of-order data, by setting the
> {{precombine.field}} to the event time to keep records with the largest event
> time.
> However, when using the FLINK_STATE index type, if cross-partition occurs,
> the precombine.field parameter will not fully take effect.
> In the case of cross-partitioning, the current logic uses data that arrives
> later, even if the event time is smaller.
> It may be necessary to unify the logic of intra-partition upsert and
> cross-partition upsert, which is convenient for users to understand and use.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
