[
https://issues.apache.org/jira/browse/HADOOP-19450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17929580#comment-17929580
]
ASF GitHub Bot commented on HADOOP-19450:
-----------------------------------------
bhattmanish98 commented on code in PR #7364:
URL: https://github.com/apache/hadoop/pull/7364#discussion_r1967079356
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java:
##########
@@ -733,4 +737,53 @@ protected void checkFuturesForExceptions(List<Future<?>>
futures, int exceptionV
}
assertEquals(exceptionCaught, exceptionVal);
}
+
+ /**
+ * Assumes that recovery through client transaction ID is enabled.
+ * Namespace is enabled for the given AzureBlobFileSystem.
+ * Service type is DFS.
+ * Assumes that the client transaction ID is enabled in the configuration.
+ *
+ * @param fs the AzureBlobFileSystem instance to check
+ * @throws AzureBlobFileSystemException in case of an error
+ */
+ protected void assumeRecoveryThroughClientTransactionID(
+ AzureBlobFileSystem fs, boolean isCreate)
+ throws AzureBlobFileSystemException {
+ // Assumes that recovery through client transaction ID is enabled.
+ Assume.assumeTrue(getConfiguration().getIsClientTransactionIdEnabled());
+ // Assumes that service type is DFS.
+ assumeDfsServiceType();
+ // Assumes that namespace is enabled for the given AzureBlobFileSystem.
+ Assume.assumeTrue(
+ fs.getIsNamespaceEnabled(getTestTracingContext(fs, true)));
+ if (isCreate) {
+ // Assume that create client is DFS client.
+ Assume.assumeTrue(
+ AbfsServiceType.DFS.equals(
+
fs.getAbfsStore().getAbfsConfiguration().getIngressServiceType()));
+ // Assume that append blob is not enabled in DFS client.
+ Assume.assumeFalse(isAppendBlobEnabled());
+ }
+ }
+
+ /**
+ * Mocks the behavior of adding a client transaction ID to the request
headers
+ * for the given AzureBlobFileSystem. This method generates a random
transaction ID
+ * and adds it to the headers of the {@link AbfsDfsClient}.
+ *
+ * @param abfsDfsClient The {@link AbfsDfsClient} mocked AbfsDfsClient.
+ * @param clientTransactionId An array to hold the generated transaction ID.
+ */
+ protected void mockAddClientTransactionIdToHeader(AbfsDfsClient
abfsDfsClient,
+ String[] clientTransactionId) {
+ Mockito.doAnswer(addClientTransactionId -> {
+ clientTransactionId[0] = UUID.randomUUID().toString();
+ List<AbfsHttpHeader> headers = addClientTransactionId.getArgument(0);
+ headers.add(
+ new AbfsHttpHeader(X_MS_CLIENT_TRANSACTION_ID,
+ clientTransactionId[0]));
+ return clientTransactionId[0];
+ }).when(abfsDfsClient).addClientTransactionIdToHeader(Mockito.anyList());
+ }
}
Review Comment:
Setting client transaction id header to true will not help, as there can be
cases where server changes is not deployed and if we set the flag as true, it
will not return client transaction id in the get path status response.
> [ABFS] Rename/Create path idempotency client-level resolution
> -------------------------------------------------------------
>
> Key: HADOOP-19450
> URL: https://issues.apache.org/jira/browse/HADOOP-19450
> Project: Hadoop Common
> Issue Type: Task
> Components: fs/azure
> Affects Versions: 3.5.0
> Reporter: Manish Bhatt
> Assignee: Manish Bhatt
> Priority: Major
> Labels: pull-request-available
>
> CreatePath and RenamePath APIs are idempotent as subsequent retries on same
> resource don’t change the server state. However, when client experiences
> connection break on the CreatePath and the RenamePath APIs, client cannot
> make sense if the request is accepted by the server or not.
> On connection failure, the client retries the request. The server might
> return 404 (sourceNotFound) in case of RenamePath API and 409
> (pathAlreadyExists) in case of CreatePath (overwrite=false) API. Now the
> client doesn’t have a path forward. Reason being, in case of CreatePath,
> client doesn’t know if the path was created on the original request or the
> path was already there for some other request, in case of RenamePath, client
> doesn’t know if the source was removed because of the original-try or it was
> not there on the first place.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
