yanenze commented on issue #4227:
URL: https://github.com/apache/hudi/issues/4227#issuecomment-990029507
maybe i have resolved this problem by modify this class
AbstractHoodieWriteClient.getPendingRollbackInfos like this
protected Map<String, Option<HoodiePendingRollbackInfo>>
getPendingRollbackInfos(@NotNull HoodieTableMetaClient metaClient) {
return
metaClient.getActiveTimeline().filterPendingRollbackTimeline().getInstants().map(
entry -> {
try {
HoodieRollbackPlan rollbackPlan =
RollbackUtils.getRollbackPlan(metaClient, entry);
LOG.info("数据回滚" +
rollbackPlan.getInstantToRollback().getCommitTime() + ":" + entry.toString());
return
Pair.of(rollbackPlan.getInstantToRollback().getCommitTime(), Option.of(new
HoodiePendingRollbackInfo(entry, rollbackPlan)));
} catch (IOException e) {
throw new HoodieIOException("Fetching rollback plan failed for "
+ entry, e);
}
} //源码修改
).filter(distinctByKey(pair->pair.getKey())).collect(Collectors.toMap(Pair::getKey,
Pair::getValue));
}
//源码修改
@NotNull
@Contract(pure = true)
static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor)
{
Map<Object,Boolean> seen = new ConcurrentHashMap<>();
return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) ==
null;
}
i run my program for three days and that problem dosen‘t occured.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
