[
https://issues.apache.org/jira/browse/HADOOP-18242?focusedWorklogId=780083&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780083
]
ASF GitHub Bot logged work on HADOOP-18242:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jun/22 18:11
Start Date: 09/Jun/22 18:11
Worklog Time Spent: 10m
Work Description: steveloughran commented on code in PR #4331:
URL: https://github.com/apache/hadoop/pull/4331#discussion_r893636106
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -104,9 +104,14 @@ public class AbfsClient implements Closeable {
private final ListeningScheduledExecutorService executorService;
/**
- * Has the Rename operation been retried once or not?
+ * Is Abfs metadata been in an incomplete State resulting in a rename
+ * failure?
*/
- private boolean hasRetriedRenameOnce;
+ private boolean isMetadataIncompleteState;
+
+ /** logging the rename failure if metadata is in an incomplete state*/
Review Comment:
nit, add a . to keep javadoc quiet
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsRenameRetryRecovery.java:
##########
@@ -0,0 +1,159 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs;
+
+import java.lang.reflect.Field;
+
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException;
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AzureBlobFileSystemException;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClient;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClientResult;
+import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation;
+import org.apache.hadoop.fs.azurebfs.services.TestAbfsClient;
+import org.apache.hadoop.fs.statistics.IOStatistics;
+
+import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.RENAME_DESTINATION_PARENT_PATH_NOT_FOUND;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.OP_RENAME;
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class TestAbfsRenameRetryRecovery extends AbstractAbfsIntegrationTest {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(TestAbfsRenameRetryRecovery.class);
+
+ public TestAbfsRenameRetryRecovery() throws Exception {
+ }
+
+ /**
+ * Mock the AbfsClient to run a metadata incomplete scenario with recovery
+ * rename.
+ */
+ @Test
+ public void testRenameFailuresDueToIncompleteMetadata() throws Exception {
+ String sourcePath = getMethodName() + "Source";
+ String destNoParentPath = "/NoParent/Dest";
+ boolean doesDestParentDirExist = false;
+ AzureBlobFileSystem fs = getFileSystem();
+
+ AbfsClient mockClient = TestAbfsClient.getMockAbfsClient(
+ fs.getAbfsStore().getClient(),
+ fs.getAbfsStore().getAbfsConfiguration());
+
+ AzureBlobFileSystemStore abfsStore = fs.getAbfsStore();
+ abfsStore = setAzureBlobSystemStoreField(abfsStore, "client", mockClient);
+
+ // SuccessFul Result.
+ AbfsRestOperation successOp = mock(AbfsRestOperation.class);
+ AbfsClientResult successResult = mock(AbfsClientResult.class);
Review Comment:
easier just to construct one with `new`
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientResult.java:
##########
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+/**
+ * A class to store the Result of an AbfsClient Operation, signifying the
+ * AbfsRestOperation and the rename recovery.
Review Comment:
needs to include Rename in the title in case we need to add more for others.
java 18 makes these struct types trivial to write, FWIW
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java:
##########
@@ -181,17 +181,18 @@ public void testRenameWithNoDestinationParentDir() throws
Exception {
byte[] data = dataset(1024, 'a', 'z');
writeDataset(fs, sourcePath, data, data.length, 1024, true);
- // Check if we have retried the rename operation.
- boolean hasRenameRetriedOnce = fs.getAbfsClient().isHasRetriedRenameOnce();
- assertFalse("Rename shouldn't be retried before attempting to rename",
+ // Check if we have seen an incomplete state.
+ boolean hasRenameRetriedOnce =
fs.getAbfsClient().isMetadataIncompleteState();
+ assertFalse("No incomplete state should be seen before attempting to "
+ + "rename",
hasRenameRetriedOnce);
// Verify that Renaming on a destination with no parent dir wasn't
// successful.
assertFalse(fs.rename(sourcePath, destPath));
- // Verify that Rename operation was retried once after not succeeding.
- hasRenameRetriedOnce = fs.getAbfsClient().isHasRetriedRenameOnce();
+ // Verify that metadata was in an incomplete state after the rename
failure.
+ hasRenameRetriedOnce = fs.getAbfsClient().isMetadataIncompleteState();
Review Comment:
check the iostats of the fs instead through IOStatisticAssertions
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientResult.java:
##########
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+/**
+ * A class to store the Result of an AbfsClient Operation, signifying the
+ * AbfsRestOperation and the rename recovery.
+ */
+public class AbfsClientResult {
+
+ /** Abfs Rest Operation. */
+ private final AbfsRestOperation op;
+ /** Flag indicating recovery took place. */
+ private final boolean renameRecovered;
+ /** Abfs storage tracking metadata is in an incomplete state.*/
+ private final boolean isIncompleteMetadataState;
+
+ public AbfsClientResult(
Review Comment:
nit, javadocs for all this, put args on separate lines.
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:
##########
@@ -1973,4 +1977,17 @@ public static String extractEtagHeader(AbfsHttpOperation
result) {
}
return etag;
}
+
+ /**
+ * Increment Rename recovery based counters in IOStatistics.
+ * @param abfsClientResult Result of an ABFS operation.
+ */
+ private void populateRenameRecoveryStatistics(AbfsClientResult
abfsClientResult) {
+ if(abfsClientResult.isRenameRecovered()) {
Review Comment:
nit, add space after the if
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsRenameRetryRecovery.java:
##########
@@ -0,0 +1,159 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs;
+
+import java.lang.reflect.Field;
+
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException;
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AzureBlobFileSystemException;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClient;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClientResult;
+import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation;
+import org.apache.hadoop.fs.azurebfs.services.TestAbfsClient;
+import org.apache.hadoop.fs.statistics.IOStatistics;
+
+import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.RENAME_DESTINATION_PARENT_PATH_NOT_FOUND;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.OP_RENAME;
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class TestAbfsRenameRetryRecovery extends AbstractAbfsIntegrationTest {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(TestAbfsRenameRetryRecovery.class);
+
+ public TestAbfsRenameRetryRecovery() throws Exception {
+ }
+
+ /**
+ * Mock the AbfsClient to run a metadata incomplete scenario with recovery
+ * rename.
+ */
+ @Test
+ public void testRenameFailuresDueToIncompleteMetadata() throws Exception {
+ String sourcePath = getMethodName() + "Source";
+ String destNoParentPath = "/NoParent/Dest";
+ boolean doesDestParentDirExist = false;
+ AzureBlobFileSystem fs = getFileSystem();
+
+ AbfsClient mockClient = TestAbfsClient.getMockAbfsClient(
+ fs.getAbfsStore().getClient(),
+ fs.getAbfsStore().getAbfsConfiguration());
+
+ AzureBlobFileSystemStore abfsStore = fs.getAbfsStore();
+ abfsStore = setAzureBlobSystemStoreField(abfsStore, "client", mockClient);
+
+ // SuccessFul Result.
+ AbfsRestOperation successOp = mock(AbfsRestOperation.class);
+ AbfsClientResult successResult = mock(AbfsClientResult.class);
+ when(successResult.getOp()).thenReturn(successOp);
+ when(successResult.isIncompleteMetadataState()).thenReturn(false);
+
+ // Failed Result.
+ AbfsRestOperation failedOp = mock(AbfsRestOperation.class);
+ AbfsClientResult recoveredMetaDataIncompleteResult =
mock(AbfsClientResult.class);
+ when(recoveredMetaDataIncompleteResult.getOp()).thenReturn(failedOp);
+
when(recoveredMetaDataIncompleteResult.isIncompleteMetadataState()).thenReturn(true);
+
+ // No destination Parent dir exception.
+ AzureBlobFileSystemException destParentNotFound
+ = getMockAbfsRestOperationException(
+ RENAME_DESTINATION_PARENT_PATH_NOT_FOUND.getStatusCode(),
+ RENAME_DESTINATION_PARENT_PATH_NOT_FOUND.getErrorCode());
+
+ // We need to throw an exception once a rename is triggered with
+ // destination having no parent, but after a retry it needs to succeed.
+ when(mockClient.renamePath(sourcePath, destNoParentPath, null, null,
+ null)).thenAnswer(
+
+ new Answer<AbfsClientResult>() {
+ @Override
+ public AbfsClientResult answer(InvocationOnMock invocationOnMock)
+ throws Throwable {
+ IOStatistics ioStatistics = fs.getIOStatistics();
+ long renameCalls = ioStatistics.counters().get(OP_RENAME);
+ LOG.info("Rename Calls from IOstatistics: {}", renameCalls);
+ if (renameCalls == 0) {
+ throw destParentNotFound;
+ }
+ return recoveredMetaDataIncompleteResult;
+ }
+ }).thenReturn(recoveredMetaDataIncompleteResult);
+
+ // Dest parent not found exc. to be raised.
+ intercept(AzureBlobFileSystemException.class,
+ () -> mockClient.renamePath(sourcePath,
+ destNoParentPath, null, null,
+ null));
+
+ AbfsClientResult resultOfSecondRenameCall =
+ mockClient.renamePath(sourcePath,
+ destNoParentPath, null, null,
+ null);
+
+ // the second rename call should be the recoveredResult due to
+ // metaDataIncomplete
+ Assertions.assertThat(resultOfSecondRenameCall)
+ .describedAs("This result should be recovered result due to MetaData "
+ + "being in incomplete state")
+ .isEqualTo(recoveredMetaDataIncompleteResult);
Review Comment:
sameAs or EqualTo?
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsRenameRetryRecovery.java:
##########
@@ -0,0 +1,159 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs;
+
+import java.lang.reflect.Field;
+
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException;
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AzureBlobFileSystemException;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClient;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClientResult;
+import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation;
+import org.apache.hadoop.fs.azurebfs.services.TestAbfsClient;
+import org.apache.hadoop.fs.statistics.IOStatistics;
+
+import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.RENAME_DESTINATION_PARENT_PATH_NOT_FOUND;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.OP_RENAME;
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class TestAbfsRenameRetryRecovery extends AbstractAbfsIntegrationTest {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(TestAbfsRenameRetryRecovery.class);
+
+ public TestAbfsRenameRetryRecovery() throws Exception {
+ }
+
+ /**
+ * Mock the AbfsClient to run a metadata incomplete scenario with recovery
+ * rename.
+ */
+ @Test
+ public void testRenameFailuresDueToIncompleteMetadata() throws Exception {
+ String sourcePath = getMethodName() + "Source";
+ String destNoParentPath = "/NoParent/Dest";
+ boolean doesDestParentDirExist = false;
+ AzureBlobFileSystem fs = getFileSystem();
+
+ AbfsClient mockClient = TestAbfsClient.getMockAbfsClient(
+ fs.getAbfsStore().getClient(),
+ fs.getAbfsStore().getAbfsConfiguration());
+
+ AzureBlobFileSystemStore abfsStore = fs.getAbfsStore();
+ abfsStore = setAzureBlobSystemStoreField(abfsStore, "client", mockClient);
+
+ // SuccessFul Result.
+ AbfsRestOperation successOp = mock(AbfsRestOperation.class);
+ AbfsClientResult successResult = mock(AbfsClientResult.class);
+ when(successResult.getOp()).thenReturn(successOp);
+ when(successResult.isIncompleteMetadataState()).thenReturn(false);
+
+ // Failed Result.
+ AbfsRestOperation failedOp = mock(AbfsRestOperation.class);
+ AbfsClientResult recoveredMetaDataIncompleteResult =
mock(AbfsClientResult.class);
+ when(recoveredMetaDataIncompleteResult.getOp()).thenReturn(failedOp);
+
when(recoveredMetaDataIncompleteResult.isIncompleteMetadataState()).thenReturn(true);
+
+ // No destination Parent dir exception.
+ AzureBlobFileSystemException destParentNotFound
+ = getMockAbfsRestOperationException(
+ RENAME_DESTINATION_PARENT_PATH_NOT_FOUND.getStatusCode(),
+ RENAME_DESTINATION_PARENT_PATH_NOT_FOUND.getErrorCode());
+
+ // We need to throw an exception once a rename is triggered with
+ // destination having no parent, but after a retry it needs to succeed.
+ when(mockClient.renamePath(sourcePath, destNoParentPath, null, null,
Review Comment:
the normal way to do this in mockito is give some counters of when to return
different things. doing it in iostats is a nice idea, but doesn't really fit in
with the normal design
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsRenameRetryRecovery.java:
##########
@@ -0,0 +1,159 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs;
+
+import java.lang.reflect.Field;
+
+import org.assertj.core.api.Assertions;
+import org.junit.Test;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException;
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AzureBlobFileSystemException;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClient;
+import org.apache.hadoop.fs.azurebfs.services.AbfsClientResult;
+import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation;
+import org.apache.hadoop.fs.azurebfs.services.TestAbfsClient;
+import org.apache.hadoop.fs.statistics.IOStatistics;
+
+import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.RENAME_DESTINATION_PARENT_PATH_NOT_FOUND;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.OP_RENAME;
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class TestAbfsRenameRetryRecovery extends AbstractAbfsIntegrationTest {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(TestAbfsRenameRetryRecovery.class);
+
+ public TestAbfsRenameRetryRecovery() throws Exception {
+ }
+
+ /**
+ * Mock the AbfsClient to run a metadata incomplete scenario with recovery
+ * rename.
+ */
+ @Test
+ public void testRenameFailuresDueToIncompleteMetadata() throws Exception {
+ String sourcePath = getMethodName() + "Source";
+ String destNoParentPath = "/NoParent/Dest";
+ boolean doesDestParentDirExist = false;
+ AzureBlobFileSystem fs = getFileSystem();
+
+ AbfsClient mockClient = TestAbfsClient.getMockAbfsClient(
+ fs.getAbfsStore().getClient(),
+ fs.getAbfsStore().getAbfsConfiguration());
+
+ AzureBlobFileSystemStore abfsStore = fs.getAbfsStore();
+ abfsStore = setAzureBlobSystemStoreField(abfsStore, "client", mockClient);
+
+ // SuccessFul Result.
+ AbfsRestOperation successOp = mock(AbfsRestOperation.class);
+ AbfsClientResult successResult = mock(AbfsClientResult.class);
+ when(successResult.getOp()).thenReturn(successOp);
+ when(successResult.isIncompleteMetadataState()).thenReturn(false);
+
+ // Failed Result.
+ AbfsRestOperation failedOp = mock(AbfsRestOperation.class);
Review Comment:
if the tests are in the same package, you can use new()
Issue Time Tracking
-------------------
Worklog Id: (was: 780083)
Time Spent: 1.5h (was: 1h 20m)
> ABFS Rename Failure when tracking metadata is in incomplete state
> -----------------------------------------------------------------
>
> Key: HADOOP-18242
> URL: https://issues.apache.org/jira/browse/HADOOP-18242
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Reporter: Mehakmeet Singh
> Assignee: Mehakmeet Singh
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> If a node in the datacenter crashes while processing an operation,
> occasionally it can leave the Storage-internal blob tracking metadata in an
> incomplete state. We expect this to happen occasionally, and so all API’s
> are designed in such a way that if this incomplete state is observed on a
> blob, the situation is resolved before the current operation proceeds.
> However, this incident has exposed a bug specifically with the Rename API,
> where the incomplete state fails to resolve, leading to this incorrect
> failure. As a temporary mitigation, if any other operation is performed on
> this blob – GetBlobProperties, GetBlob, GetFileProperties, SetFileProperties,
> etc – it should resolve the incomplete state, and rename will no longer hit
> this issue.
> StackTrace:
> {code:java}
> 2022-03-22 17:52:19,789 DEBUG [regionserver/euwukwlss-hg50:16020.logRoller]
> services.AbfsClient: HttpRequest:
> 404,RenameDestinationParentPathNotFound,cid=ef5cbf0f-5d4a-4630-8a59-3d559077fc24,rid=35fef164-101f-000b-1b15-3ed818000000,sent=0,recv=212,PUT,https://euwqdaotdfdls03.dfs.core.windows.net/eykbssc/apps/hbase/data/oldWALs/euwukwlss-hg50.tdf.qa%252C16020%252C1647949929877.1647967939315?timeout=90
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
