[
https://issues.apache.org/jira/browse/HUDI-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-5950:
---------------------------------
Labels: pull-request-available (was: )
> Fix pending instants determination for MDT compaction
> -----------------------------------------------------
>
> Key: HUDI-5950
> URL: https://issues.apache.org/jira/browse/HUDI-5950
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: sivabalan narayanan
> Priority: Major
> Labels: pull-request-available
>
> Currently the logic to deduce pending instants in MDT is as below
> * a = we get latest completed delta commit from MDT.
> * Find any inflights in DT timeline *before* \{a}
> and if we don't find any such inflights, we will go ahead and may be compact
> MDT.
> But what incase the latest delta commit in MDT succeeded in MDT, but failed
> in DT. so, it could potentially result in triggering compaction in MDT which
> should not happen.
>
> So, the right fix is
> * a = we get latest completed delta commit from MDT.
> * Find any inflights in DT timeline *before or equals* to \{a}
> This should take care of not triggering compaction in MDT when here are
> inflights in DT which is committed to MDT.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
