[
https://issues.apache.org/jira/browse/HUDI-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538147#comment-17538147
]
sivabalan narayanan commented on HUDI-4112:
-------------------------------------------
{code:java}
Exception in thread "main" org.apache.hudi.exception.HoodieRestoreException:
Failed to restore to 20220426091934506
at
org.apache.hudi.client.BaseHoodieWriteClient.restoreToInstant(BaseHoodieWriteClient.java:728)
at
com.infinilake.sparkJobs.SavepointRestoreJob.main(SavepointRestoreJob.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.hudi.exception.HoodieMetadataException: The instant
[20220426192933855__deltacommit__COMPLETED] required to sync rollback of
20220426192933855 has been archived
at
org.apache.hudi.metadata.HoodieTableMetadataUtil.lambda$processRollbackMetadata$14(HoodieTableMetadataUtil.java:538)
at java.util.HashMap$Values.forEach(HashMap.java:982)
at
java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
at
org.apache.hudi.metadata.HoodieTableMetadataUtil.processRollbackMetadata(HoodieTableMetadataUtil.java:515)
at
org.apache.hudi.metadata.HoodieTableMetadataUtil.lambda$null$8(HoodieTableMetadataUtil.java:446)
at java.util.Collections$SingletonList.forEach(Collections.java:4824)
at
org.apache.hudi.metadata.HoodieTableMetadataUtil.lambda$processRestoreMetadata$9(HoodieTableMetadataUtil.java:446)
at java.util.HashMap$Values.forEach(HashMap.java:982)
at
java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
at
org.apache.hudi.metadata.HoodieTableMetadataUtil.processRestoreMetadata(HoodieTableMetadataUtil.java:446)
at
org.apache.hudi.metadata.HoodieTableMetadataUtil.convertMetadataToRecords(HoodieTableMetadataUtil.java:416)
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.lambda$update$7(HoodieBackedTableMetadataWriter.java:706)
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.processAndCommit(HoodieBackedTableMetadataWriter.java:669)
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:706)
at
org.apache.hudi.table.action.BaseActionExecutor.lambda$writeTableMetadata$3(BaseActionExecutor.java:85)
at org.apache.hudi.common.util.Option.ifPresent(Option.java:97)
at
org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:85)
at
org.apache.hudi.table.action.restore.BaseRestoreActionExecutor.writeToMetadata(BaseRestoreActionExecutor.java:154)
at
org.apache.hudi.table.action.restore.BaseRestoreActionExecutor.finishRestore(BaseRestoreActionExecutor.java:129)
at
org.apache.hudi.table.action.restore.BaseRestoreActionExecutor.execute(BaseRestoreActionExecutor.java:95)
at
org.apache.hudi.table.HoodieSparkMergeOnReadTable.restore(HoodieSparkMergeOnReadTable.java:175)
at
org.apache.hudi.client.BaseHoodieWriteClient.restoreToInstant(BaseHoodieWriteClient.java:714){code}
{code:java}
Caused by: org.apache.hudi.exception.HoodieMetadataException: The instant
[20220426192933855__deltacommit__COMPLETED] required to sync rollback of
20220426192933855 has been archived {code}
> Relax constraint in metadata table that rollback of a commit that got
> archived in MDT throws exception
> ------------------------------------------------------------------------------------------------------
>
> Key: HUDI-4112
> URL: https://issues.apache.org/jira/browse/HUDI-4112
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: sivabalan narayanan
> Priority: Blocker
> Fix For: 0.11.1
>
>
> when we are trying to rollback a commit, and if the commit it archived in
> MDT, when this rollback is getting applied to MDT, we throw exception.
>
> excerpt from HoodieTableMetadataUtil.java
>
> {code:java}
> HoodieInstant syncedInstant = new HoodieInstant(false,
> HoodieTimeline.DELTA_COMMIT_ACTION, instantToRollback);
> if
> (metadataTableTimeline.getCommitsTimeline().isBeforeTimelineStarts(syncedInstant.getTimestamp()))
> {
> throw new HoodieMetadataException(String.format("The instant %s required to
> sync rollback of %s has been archived",
> syncedInstant, instantToRollback));
> }
> shouldSkip = !metadataTableTimeline.containsInstant(syncedInstant);
> if (!hasNonZeroRollbackLogFiles && shouldSkip) {
> LOG.info(String.format("Skipping syncing of rollbackMetadata at %s, since
> this instant was never committed to Metadata Table",
> instantToRollback));
> return;
> } {code}
>
> This is very much valid in case of restore operation.
> C1, C2, C3, C4, C5, C6.
> C2 savepointed.
> but MDT could have archived C2(aggressive archival commits) since all C1 to
> C6 are committed. So, when we trigger restore to C1, it will invoke rollback
> of C6, C5... C2.
> So, with savepoint and restore flow, this is a valid scenario and we need to
> relax the constraint.
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
