[
https://issues.apache.org/jira/browse/HADOOP-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872021#comment-17872021
]
ASF GitHub Bot commented on HADOOP-19187:
-----------------------------------------
rakeshadr commented on code in PR #6879:
URL: https://github.com/apache/hadoop/pull/6879#discussion_r1709602052
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java:
##########
@@ -434,10 +445,70 @@ public AbfsConfiguration(final Configuration rawConfig,
String accountName)
}
}
+ public AbfsConfiguration(final Configuration rawConfig, String accountName)
+ throws IllegalAccessException, IOException {
+ this(rawConfig, accountName, AbfsServiceType.DFS);
+ }
+
public Trilean getIsNamespaceEnabledAccount() {
return Trilean.getTrilean(isNamespaceEnabledAccount);
}
+ /**
+ * Returns the service type to be used based on the filesystem configuration.
+ * Precedence is given to service type configured for FNS Accounts using
+ * "fs.azure.fns.account.service.type". If not configured, then the service
+ * type identified from url used to initialize filesystem will be used.
+ * @return the service type.
+ */
+ public AbfsServiceType getFsConfiguredServiceType() {
+ return getEnum(FS_AZURE_FNS_ACCOUNT_SERVICE_TYPE, fsConfiguredServiceType);
+ }
+
+ /**
+ * Returns the service type configured for FNS Accounts to override the
+ * service type identified by URL used to initialize the filesystem.
+ * @return the service type.
+ */
+ public AbfsServiceType getConfiguredServiceTypeForFNSAccounts() {
+ return getEnum(FS_AZURE_FNS_ACCOUNT_SERVICE_TYPE, null);
+ }
+
+ /**
+ * Returns the service type to be used for Ingress Operations irrespective
of account type.
+ * Default value is the same as the service type configured for the file
system.
+ * @return the service type.
+ */
+ public AbfsServiceType getIngressServiceType() {
+ return getEnum(FS_AZURE_INGRESS_SERVICE_TYPE,
getFsConfiguredServiceType());
+ }
+
+ public boolean isDfsToBlobFallbackEnabled() {
+ return isDfsToBlobFallbackEnabled;
+ }
+
+ /**
+ * Checks if the service type configured is valid for account type used.
+ * HNS Enabled accounts cannot have service type as BLOB.
+ * @param isHNSEnabled Flag to indicate if HNS is enabled for the account.
+ * @throws InvalidConfigurationValueException if the service type is invalid.
+ */
+ public void validateConfiguredServiceType(boolean isHNSEnabled)
+ throws InvalidConfigurationValueException {
+ // Todo: [FnsOverBlob] - Remove this check, Failing FS Init with Blob
Endpoint Until FNS over Blob is ready.
+ if (getFsConfiguredServiceType() == AbfsServiceType.BLOB) {
+ throw new InvalidConfigurationValueException(FS_DEFAULT_NAME_KEY,
+ "Blob Endpoint Support not yet available");
+ }
+ if (isHNSEnabled && getConfiguredServiceTypeForFNSAccounts() ==
AbfsServiceType.BLOB) {
+ throw new InvalidConfigurationValueException(
+ FS_AZURE_FNS_ACCOUNT_SERVICE_TYPE, "Cannot be BLOB for HNS Account");
+ } else if (isHNSEnabled && fsConfiguredServiceType ==
AbfsServiceType.BLOB) {
+ throw new InvalidConfigurationValueException(FS_DEFAULT_NAME_KEY,
Review Comment:
Ok, fine, please cover this in the follow-up PR.
> ABFS: [FnsOverBlob]Making AbfsClient Abstract for supporting both DFS and
> Blob Endpoint
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-19187
> URL: https://issues.apache.org/jira/browse/HADOOP-19187
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
> Labels: pull-request-available
>
> Azure Services support two different set of APIs.
> Blob:
> [https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api]
>
> DFS:
> [https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/operation-groups]
>
> As per the plan in HADOOP-19179, this task enables ABFS Driver to work with
> both set of APIs as per the requirement.
> Scope of this task is to refactor the ABfsClient so that ABFSStore can choose
> to interact with the client it wants based on the endpoint configured by user.
> The blob endpoint support will remain "Unsupported" until the whole code is
> checked-in and well tested.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
