[
https://issues.apache.org/jira/browse/HUDI-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raymond Xu updated HUDI-3449:
-----------------------------
Sprint: Hudi-Sprint-Feb-14, Hudi-Sprint-Feb-22, Hudi-Sprint-Mar-01 (was:
Hudi-Sprint-Feb-14, Hudi-Sprint-Feb-22)
> Async compaction cannot proceed due to archived deltacommit in metadata table
> -----------------------------------------------------------------------------
>
> Key: HUDI-3449
> URL: https://issues.apache.org/jira/browse/HUDI-3449
> Project: Apache Hudi
> Issue Type: Bug
> Components: writer-core
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.11.0
>
>
> Environment: deltastreamer continuous mode writing MOR table, async
> compaction and clean enabled. Metadata table enabled.
> Repeatly hitting the following issue after the deltastreamer is running for
> some time:
> {code:java}
> org.apache.hudi.exception.HoodieMetadataException: The instant
> [20220216224839969__deltacommit__COMPLETED] required to sync rollback of
> 20220216224839969 has been archived
> at
> org.apache.hudi.metadata.HoodieTableMetadataUtil.lambda$processRollbackMetadata$15(HoodieTableMetadataUtil.java:521)
> at java.util.HashMap$Values.forEach(HashMap.java:981)
> at
> java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
> at
> org.apache.hudi.metadata.HoodieTableMetadataUtil.processRollbackMetadata(HoodieTableMetadataUtil.java:498)
> at
> org.apache.hudi.metadata.HoodieTableMetadataUtil.convertMetadataToRollbackRecords(HoodieTableMetadataUtil.java:472)
> at
> org.apache.hudi.metadata.HoodieTableMetadataUtil.convertMetadataToRecords(HoodieTableMetadataUtil.java:437)
> at
> org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:726)
> at
> org.apache.hudi.table.action.BaseActionExecutor.lambda$writeTableMetadata$2(BaseActionExecutor.java:77)
> at org.apache.hudi.common.util.Option.ifPresent(Option.java:96)
> at
> org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:77)
> at
> org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.finishRollback(BaseRollbackActionExecutor.java:244)
> at
> org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.runRollback(BaseRollbackActionExecutor.java:122)
> at
> org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.execute(BaseRollbackActionExecutor.java:144)
> at
> org.apache.hudi.table.HoodieSparkMergeOnReadTable.rollback(HoodieSparkMergeOnReadTable.java:172)
> at
> org.apache.hudi.table.HoodieTable.rollbackInflightCompaction(HoodieTable.java:524)
> at
> org.apache.hudi.client.SparkRDDWriteClient.compact(SparkRDDWriteClient.java:342)
> at
> org.apache.hudi.client.BaseHoodieWriteClient.compact(BaseHoodieWriteClient.java:919)
> at
> org.apache.hudi.client.HoodieSparkCompactor.compact(HoodieSparkCompactor.java:52)
> at
> org.apache.hudi.async.AsyncCompactService.lambda$null$1(AsyncCompactService.java:89)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
