[
https://issues.apache.org/jira/browse/HUDI-7478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-7478.
----------------------------
Fix Version/s: 0.15.0
Resolution: Fixed
Fixed via master branch: 8bc9a4bc875107026d596c82bcd9cee9fc997e4c
> Fix max delta commits guard check w/ MDT
> -----------------------------------------
>
> Key: HUDI-7478
> URL: https://issues.apache.org/jira/browse/HUDI-7478
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: sivabalan narayanan
> Assignee: Vova Kolmakov
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.15.0
>
>
> protected static void checkNumDeltaCommits(HoodieTableMetaClient metaClient,
> int maxNumDeltaCommitsWhenPending) \{
> final HoodieActiveTimeline activeTimeline =
> metaClient.reloadActiveTimeline();
> Option<HoodieInstant> lastCompaction =
> activeTimeline.filterCompletedInstants()
> .filter(s -> s.getAction().equals(COMPACTION_ACTION)).lastInstant();
> int numDeltaCommits = lastCompaction.isPresent()
> ?
> activeTimeline.getDeltaCommitTimeline().findInstantsAfter(lastCompaction.get().getTimestamp()).countInstants()
> : activeTimeline.getDeltaCommitTimeline().countInstants();
> if (numDeltaCommits > maxNumDeltaCommitsWhenPending) {
> throw new HoodieMetadataException(String.format("Metadata table's
> deltacommits exceeded %d: "
> + "this is likely caused by a pending instant in the data
> table. Resolve the pending instant "
> + "or adjust `%s`, then restart the pipeline.",
> maxNumDeltaCommitsWhenPending,
> HoodieMetadataConfig.METADATA_MAX_NUM_DELTACOMMITS_WHEN_PENDING.key()));
> }
> }
> Here we account for action type "compaction. But compaction completed instant
> will have "commit" as action. So, we need to fix it.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
