danny0405 commented on code in PR #5535:
URL: https://github.com/apache/hudi/pull/5535#discussion_r876565034
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1558,13 +1558,15 @@ private void tryUpgrade(HoodieTableMetaClient
metaClient, Option<String> instant
new UpgradeDowngrade(metaClient, config, context,
upgradeDowngradeHelper);
if
(upgradeDowngrade.needsUpgradeOrDowngrade(HoodieTableVersion.current())) {
- // Ensure no inflight commits by setting EAGER policy and explicitly
cleaning all failed commits
- List<String> instantsToRollback = getInstantsToRollback(metaClient,
HoodieFailedWritesCleaningPolicy.EAGER, instantTime);
+ if
(config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()) {
+ // Ensure no inflight commits by setting EAGER policy and explicitly
cleaning all failed commits
+ List<String> instantsToRollback = getInstantsToRollback(metaClient,
HoodieFailedWritesCleaningPolicy.EAGER, instantTime);
- Map<String, Option<HoodiePendingRollbackInfo>> pendingRollbacks =
getPendingRollbackInfos(metaClient);
- instantsToRollback.forEach(entry -> pendingRollbacks.putIfAbsent(entry,
Option.empty()));
+ Map<String, Option<HoodiePendingRollbackInfo>> pendingRollbacks =
getPendingRollbackInfos(metaClient);
+ instantsToRollback.forEach(entry ->
pendingRollbacks.putIfAbsent(entry, Option.empty()));
Review Comment:
Hello @nsivabalan @n3nash , i have refactored the code a little to
1. move the failed writes cleaning all together to when starting the new
commit
2. the cleaning strategy for commit actions as tweaked: force to clean if
metadata table is enabled in case we commit metadata table successfully but
data set table failed
Please review again if you have time, thanks so much in advance ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
