yihua commented on code in PR #5245:
URL: https://github.com/apache/hudi/pull/5245#discussion_r932532922
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1113,9 +1113,28 @@ protected Map<String, Option<HoodiePendingRollbackInfo>>
getPendingRollbackInfos
protected Map<String, Option<HoodiePendingRollbackInfo>>
getPendingRollbackInfos(HoodieTableMetaClient metaClient, boolean
ignoreCompactionAndClusteringInstants) {
List<HoodieInstant> instants =
metaClient.getActiveTimeline().filterPendingRollbackTimeline().getInstants().collect(Collectors.toList());
Map<String, Option<HoodiePendingRollbackInfo>> infoMap = new HashMap<>();
- for (HoodieInstant instant : instants) {
+ for (HoodieInstant rollbackInstant : instants) {
+ HoodieRollbackPlan rollbackPlan;
+ try {
+ rollbackPlan = RollbackUtils.getRollbackPlan(metaClient,
rollbackInstant);
+ } catch (IOException e) {
Review Comment:
The deletion only happens for the requested rollback plan. Any inflight
rollback is not affected. The assumption here is that even if the requested
rollback plan is inaccessible and deleted, it can be requested again by a new
writer, which is still safe. We don't want the hanging rollback plan to block
metadata table compaction.
The corruption is mainly due to writes of a rollback plan not atomic, and
the job fails during that time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
