[
https://issues.apache.org/jira/browse/HADOOP-18242?focusedWorklogId=774253&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774253
]
ASF GitHub Bot logged work on HADOOP-18242:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/May/22 21:59
Start Date: 24/May/22 21:59
Worklog Time Spent: 10m
Work Description: steveloughran commented on code in PR #4331:
URL: https://github.com/apache/hadoop/pull/4331#discussion_r880982514
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java:
##########
@@ -167,4 +169,30 @@ public void testPosixRenameDirectory() throws Exception {
new Path(testDir2 + "/test1/test2/test3"));
}
+ @Test
+ public void testRenameWithNoDestinationParentDir() throws Exception {
Review Comment:
add similar test case for resilient rename api, here or in
ITestAbfsManifestStoreOperations
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -538,6 +544,23 @@ public Pair<AbfsRestOperation, Boolean> renamePath(
if (!op.hasResult()) {
throw e;
}
+
Review Comment:
have renamePath return a struct rather than a pair, with one of the fields
being the "retried for metadata issue" flag. this can be passed up to the
private resilient rename interface, and then to the manifest committer. it
could then include this in its statistic reports
Issue Time Tracking
-------------------
Worklog Id: (was: 774253)
Time Spent: 50m (was: 40m)
> ABFS Rename Failure when tracking metadata is in incomplete state
> -----------------------------------------------------------------
>
> Key: HADOOP-18242
> URL: https://issues.apache.org/jira/browse/HADOOP-18242
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Reporter: Mehakmeet Singh
> Assignee: Mehakmeet Singh
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> If a node in the datacenter crashes while processing an operation,
> occasionally it can leave the Storage-internal blob tracking metadata in an
> incomplete state. We expect this to happen occasionally, and so all API’s
> are designed in such a way that if this incomplete state is observed on a
> blob, the situation is resolved before the current operation proceeds.
> However, this incident has exposed a bug specifically with the Rename API,
> where the incomplete state fails to resolve, leading to this incorrect
> failure. As a temporary mitigation, if any other operation is performed on
> this blob – GetBlobProperties, GetBlob, GetFileProperties, SetFileProperties,
> etc – it should resolve the incomplete state, and rename will no longer hit
> this issue.
> StackTrace:
> {code:java}
> 2022-03-22 17:52:19,789 DEBUG [regionserver/euwukwlss-hg50:16020.logRoller]
> services.AbfsClient: HttpRequest:
> 404,RenameDestinationParentPathNotFound,cid=ef5cbf0f-5d4a-4630-8a59-3d559077fc24,rid=35fef164-101f-000b-1b15-3ed818000000,sent=0,recv=212,PUT,https://euwqdaotdfdls03.dfs.core.windows.net/eykbssc/apps/hbase/data/oldWALs/euwukwlss-hg50.tdf.qa%252C16020%252C1647949929877.1647967939315?timeout=90
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
