vamshikrishnakyatham opened a new issue, #14007:
URL: https://github.com/apache/hudi/issues/14007
### Bug Description
**What happened:**
Clean operation fails when attempting to rollback commits that appear
chronologically before a compaction in the timeline but were not actually
included in the compaction scope. The validation logic incorrectly blocks
rollback of these instants (commits requested before compaction but completed
after compaction).
**What you expected:**
Clean operation should successfully rollback commits that were not actually
compacted, regardless of their position in the timeline relative to compaction
timestamps.
**Steps to reproduce:**
1. Perform multiple delta commits on a Hudi table
2. Trigger a compaction while some commits are still pending
3. Run a clean operation that attempts to rollback one of these commits
because of timeout
We observe HoodieRollbackException due to incorrect timeline based validation
### Environment
**Hudi version:** 1.1.0
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**
### Logs and Stack Trace
org.apache.hudi.exception.HoodieRollbackException: Failed to rollback
s3a://xyz 20250925012123905
at
org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:1231)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:1165)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:1088)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:1070)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:1065)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.lambda$clean$cbd664b5$1(BaseHoodieTableServiceClient.java:846)
at
org.apache.hudi.common.util.CleanerUtils.rollbackFailedWrites(CleanerUtils.java:217)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.clean(BaseHoodieTableServiceClient.java:845)
at
org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:930)
at
com.onehouse.hudi.table.standaloneservices.CleanTask.runTask(CleanTask.java:33)
at
com.onehouse.hudi.table.standaloneservices.TableServiceTaskRunner.executeSingleTask(TableServiceTaskRunner.java:130)
at
com.onehouse.hudi.table.standaloneservices.TableServiceTaskRunner.executeTasks(TableServiceTaskRunner.java:123)
at
com.onehouse.hudi.table.standaloneservices.TableServiceTaskRunner.run(TableServiceTaskRunner.java:92)
at com.onehouse.util.TaskFuture.lambda$new$0(TaskFuture.java:33)
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hudi.exception.HoodieMetadataException: Commit being
rolled back 20250925012123905 is earlier than the latest compaction
20250925012523368. There are 8 deltacommits after this compaction:
[[20250925012447357__20250925013432341__deltacommit__COMPLETED],
[20250925012518125__20250925012831379__deltacommit__COMPLETED],
[20250925012851950__20250925013157886__deltacommit__COMPLETED],
[20250925013447468__20250925014634434__deltacommit__COMPLETED],
[20250925013748434__20250925014142539__deltacommit__COMPLETED],
[20250925014147869__20250925014458106__deltacommit__COMPLETED],
[20250925014647316__20250925015522155__deltacommit__COMPLETED],
[20250925015718106__20250925020108423__deltacommit__COMPLETED]]
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.validateRollback(HoodieBackedTableMetadataWriter.java:1665)
at
org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:1635)
at
org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:114)
at
org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.finishRollback(BaseRollbackActionExecutor.java:279)
at
org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.runRollback(BaseRollbackActionExecutor.java:125)
at
org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.execute(BaseRollbackActionExecutor.java:153)
at
org.apache.hudi.table.HoodieSparkMergeOnReadTable.rollback(HoodieSparkMergeOnReadTable.java:218)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:1218)
... 17 more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
