[
https://issues.apache.org/jira/browse/HADOOP-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17871412#comment-17871412
]
ASF GitHub Bot commented on HADOOP-19187:
-----------------------------------------
anujmodi2021 commented on code in PR #6879:
URL: https://github.com/apache/hadoop/pull/6879#discussion_r1705904659
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemInitAndCreate.java:
##########
@@ -96,6 +110,33 @@ public void testNoGetAclCallOnHnsConfigPresence() throws
Exception {
store.getIsNamespaceEnabled(tracingContext);
Mockito.verify(client, Mockito.times(0))
- .getAclStatus(Mockito.anyString(), Mockito.any(TracingContext.class));
+ .getAclStatus(Mockito.anyString(), any(TracingContext.class));
+ }
+
+ // Todo: [FnsOverBlob] Remove this test case once Blob Endpoint Support is
ready and enabled.
+ @Test
+ public void testFileSystemInitFailsWithBlobEndpoitUrl() throws Exception {
+ Configuration configuration = getRawConfiguration();
+ String defaultUri = configuration.get(FS_DEFAULT_NAME_KEY);
+ String blobUri = defaultUri.replace(ABFS_DFS_DOMAIN_NAME,
ABFS_BLOB_DOMAIN_NAME);
+ AzureBlobFileSystemException ex =
+ intercept(AzureBlobFileSystemException.class, () ->
Review Comment:
Thanks for the great suggestion. Taken
##########
hadoop-tools/hadoop-azure/src/site/markdown/fnsBlob.md:
##########
@@ -0,0 +1,80 @@
+<!---
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# ABFS Driver for FNS Accounts
+
+### Note: FNS-BLOB Support is being built and not yet ready for usage.
+
+## Background
+ABFS driver is recommended to be used only with HNS Enabled ADLS Gen-2 accounts
+for big data analytics because of being more performant and scalable.
+
+However, to enable users of legacy WASB Driver to migrate to ABFS driver
without
+needing them to upgrade their general purpose V2 accounts (HNS-Disabled),
Support
+for FNS accounts is being added to ABFS driver.
+Refer to [WASB Deprication](./wasb.html) for more details.
+
+## Azure Service Endpoints Used by ABFS Driver
+Azure Services offers two set of endpoints for interacting with storage
accounts:
+1. [Azure Blob
Storage](https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api)
referred as Blob Endpoint
+2. [Azure Data Lake
Storage](https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/operation-groups)
referred as DFS Endpoint
+
+ABFS Driver by default is designed to work with DFS Endpoint only which
primarily
+supports HNS Enabled Accounts only. However, azure services does not
recommended
+to interact with FNS accounts using DFS Endpoint.
+
+To enable ABFS Driver to work with FNS Accounts, Support for Blob Endpoint is
being added.
+ABFS Driver will only allow FNS Accounts to be accessed using Blob Endpoint.
+HNS Enabled accounts will still use DFS Endpoint which continues to be the
+recommended stack based on performance and feature capabilities.
+
+## Configuring ABFS Driver for FNS Accounts
+Following configurations will be introduced to configure ABFS Driver for FNS
Accounts:
+1. Account Type: Must be set to `false` to indicate FNS Account
+ ```xml
+ <property>
+ <name>fs.azure.account.hns.enabled</name>
+ <value>false</value>
+ </property>
+ ```
+
+2. Account Url: It is the URL used to initialize the file system. It is either
passed
+directly to file system or configured as default uri using "fs.DefaultFS"
configuration.
+In both the cases the URL used must be the blob endpoint url of the account.
+ ```xml
+ <property>
+ <name>fs.defaultFS</name>
+ <value>https://ACCOUNT_NAME.blob.core.windows.net</value>
+ </property>
+ ```
+3. Service Type for FNS Accounts: This will allow an override to choose service
+type specially in cases where any local DNS resolution is set for the account
and driver is
+unable to detect the intended endpoint from above configured URL. If this is
set
+to blob for HNS Enabled Accounts, FS init will fail with InvalidConfiguration
error.
+ ```xml
+ <property>
+ <name>fs.azure.fns.account.service.type</name>
+ <value>BLOB</value>
+ </property>
+ ```
+
+4. Service Type for Ingress Operations: This will allow an override to choose
service
+type only for Ingress Related Operations. All other operations will still use
the
Review Comment:
Added details
> ABFS: [FnsOverBlob]Making AbfsClient Abstract for supporting both DFS and
> Blob Endpoint
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-19187
> URL: https://issues.apache.org/jira/browse/HADOOP-19187
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
> Labels: pull-request-available
>
> Azure Services support two different set of APIs.
> Blob:
> [https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api]
>
> DFS:
> [https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/operation-groups]
>
> As per the plan in HADOOP-19179, this task enables ABFS Driver to work with
> both set of APIs as per the requirement.
> Scope of this task is to refactor the ABfsClient so that ABFSStore can choose
> to interact with the client it wants based on the endpoint configured by user.
> The blob endpoint support will remain "Unsupported" until the whole code is
> checked-in and well tested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
