[
https://issues.apache.org/jira/browse/HADOOP-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17866613#comment-17866613
]
ASF GitHub Bot commented on HADOOP-19208:
-----------------------------------------
anujmodi2021 opened a new pull request, #6946:
URL: https://github.com/apache/hadoop/pull/6946
Jira: https://issues.apache.org/jira/browse/HADOOP-19208
PR merged to trunk: https://github.com/apache/hadoop/pull/6893
### Original Description of PR
ABFS driver needs to know the type of account being used. It relies on the
user to inform the account type using the config
`fs.azure.account.hns.enabled`. If not configured, driver makes a [GetAcl Head
Call](https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/get-properties)
call to determine the account type.
Expectation is getAcl() will fail with 400 Bad Request if made on the FNS
Account.
If succeeded or fails with any other error code, it means it's an HNS
Account.
#### Current Implementation:
Today when someone tries to create a new file system, getAcl is called to
determine HNS/FNS account. This call fails with 404 as file system is not yet
created. Then another getAcl is needed to determine HNS/FNS post filesystem
creation. Leading to redundant getAcl calls.
#### New Implementation
1. If getAcl fails with 400 then only we will determine the account as HNS
disabled. For any other error code including 404, we will infer the account as
HNS enabled. getAcl call will continue to throw exception in cases where it
fails with non 400 error code as earlier.
2. In case of 404, getAcl() will still fail as always but store will capture
the HNS enabled/disabled to avoid further calls. File system init will not fail
for 404 as per the design.
3. In case of error apart from 400, 404 like 5xx or others, getAcl() will
still fail leading to file system initialization failure. Any inference made on
account type here won't be used further.
This PR also fixes a test case failing on trunk.
`testGetAclCallOnHnsConfigAbsence(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate)`
### How was this patch tested?
New test around the production code changes, and existing test suite ran to
validate the patch.
### Test Suite Results
Metric related tests are fixed in the the
https://github.com/apache/hadoop/pull/6847
> ABFS: Fixing logic to determine HNS nature of account to avoid extra getAcl()
> calls
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-19208
> URL: https://issues.apache.org/jira/browse/HADOOP-19208
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
> Labels: pull-request-available
>
> ABFS driver needs to know the type of account being used. It relies on the
> user to inform the account type using the config
> `fs.azure.account.hns.enabled`.
> If not configured, driver makes a getAcl call to determine the account type.
> Expectation is getAcl() will fail with 400 Bad Request if made on the FNS
> Account.
> For any other case including 200, 404 it will indicate account is HNS.
> Today, when determining this, the logic only checks status code to be either
> 200 or 400. In case of 404, nothing is inferred, and this leads to more
> getAcl again and again until 200 or 400 comes.
> Fix is to update the logic such that if getAcl() fails with 400, it is FNS
> account. For all other cases it will be an HNS account. In case of
> throttling, if all retries are exhausted, FS init itself will fail.
> This is also to fix a test case failing on trunk.
> {{testGetAclCallOnHnsConfigAbsence(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate)}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
