steveloughran commented on code in PR #4331:
URL: https://github.com/apache/hadoop/pull/4331#discussion_r906166340
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -531,13 +541,40 @@ public Pair<AbfsRestOperation, Boolean> renamePath(
url,
requestHeaders);
try {
+ incrementAbfsRenamePath();
op.execute(tracingContext);
- return Pair.of(op, false);
+ // AbfsClientResult contains the AbfsOperation, If recovery happened or
+ // not, and the incompleteMetaDataState is true or false.
+ // If we successfully rename a path and isMetadataIncompleteState was
+ // true, then rename was recovered, else it didn't, this is why
+ // isMetadataIncompleteState is used for renameRecovery(as the 2nd
param).
+ return new AbfsClientRenameResult(op, isMetadataIncompleteState,
isMetadataIncompleteState);
} catch (AzureBlobFileSystemException e) {
// If we have no HTTP response, throw the original exception.
if (!op.hasResult()) {
throw e;
}
+
+ // ref: HADOOP-18242. Rename failure occurring due to a rare case of
+ // tracking metadata being in incomplete state.
+ if (op.getResult().getStorageErrorCode()
+ .equals(RENAME_DESTINATION_PARENT_PATH_NOT_FOUND.getErrorCode())
+ && !isMetadataIncompleteState) {
+ //Logging
+ ABFS_METADATA_INCOMPLETE_RENAME_FAILURE
+ .info("Rename Failure attempting to resolve tracking metadata
state and retrying.");
+
+ // Doing a HEAD call resolves the incomplete metadata state and
+ // then we can retry the rename operation.
+ getPathStatus(source, false, tracingContext);
Review Comment:
I've had one more thought here. the path status contains the etag, doesn't
it? so if sourceEtag was null, now we can set it. That way if the rename
failure is followed
immediately buy the rename-failure-but-it-really-happened event of
HADOOP-18163, we are lined up for recovery
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
