priyeshkaratha commented on code in PR #11021:
URL: https://github.com/apache/ozone/pull/11021#discussion_r3801072839
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java:
##########
@@ -1357,8 +1357,16 @@ private void onSuccess(String bucketName) {
private void handleAndClearFullList(OmBucketInfo bucket,
LimitedExpiredObjectList keysList,
boolean dir, OmLifecycleScanState.Builder scanStateBuilder, boolean
scanFinished) {
if (moveToTrashEnabled.get() && bucket.getBucketLayout() != OBJECT_STORE
&& getEffectiveOzoneTrash() != null) {
- moveToTrash(bucket, keysList, dir);
- sendSaveScanStateRequest(scanStateBuilder, scanFinished);
+ int failedMoves = moveToTrash(bucket, keysList, dir);
+ if (failedMoves == 0) {
Review Comment:
Not advancing scan state when failedMoves > 0 creates an infinite retry loop
for any permanently-failing key. will block other keys too?
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java:
##########
@@ -1617,9 +1628,11 @@ public OzoneManagerProtocolProtos.OMResponse run()
throws Exception {
}
} catch (IOException | InterruptedException e) {
LOG.error("Failed to send RenameKeysRequest", e);
+ failedMoves++;
Review Comment:
All rename failures are counted identically. A KEY_NOT_FOUND response (key
was already moved to trash in a prior cycle) is treated the same as a transient
RPC failure. Is my understanding correct?
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java:
##########
@@ -1617,9 +1628,11 @@ public OzoneManagerProtocolProtos.OMResponse run()
throws Exception {
}
} catch (IOException | InterruptedException e) {
LOG.error("Failed to send RenameKeysRequest", e);
+ failedMoves++;
}
}
keysList.clear();
+ return failedMoves;
Review Comment:
keysList.clear() is still unconditional. If any moves failed, the caller
rolls back the scan position, but these keys have already been removed from the
pending list. On the next scan pass the service re-discovers them, tries to
rename the already-moved ones again, and those produce new KEY_NOT_FOUND
failures. which can cause infinite loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
