[
https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843638#comment-17843638
]
ASF GitHub Bot commented on HADOOP-18516:
-----------------------------------------
anujmodi2021 commented on code in PR #6552:
URL: https://github.com/apache/hadoop/pull/6552#discussion_r1590561673
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java:
##########
@@ -980,33 +981,59 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
+ /**
+ * Returns the SASTokenProvider implementation to be used to generate SAS
token.<br>
+ * Users can choose between a custom implementation of {@link
SASTokenProvider}
+ * or an in house implementation {@link FixedSASTokenProvider}.<br>
+ * For Custom implementation "fs.azure.sas.token.provider.type" needs to be
provided.<br>
+ * For Fixed SAS Token use "fs.azure.sas.fixed.token" needs to be
provided.<br>
+ * In case both are provided, Preference will be given to Custom
implementation.<br>
+ * Avoid using a custom tokenProvider implementation just to read the
configured
+ * fixed token, as this could create confusion. Also,implementing the
SASTokenProvider
+ * requires relying on the raw configurations. It is more stable to depend on
+ * the AbfsConfiguration with which a filesystem is initialized, and
eliminate
+ * chances of dynamic modifications and spurious situations.<br>
+ * @return sasTokenProvider object based on configurations provided
+ * @throws AzureBlobFileSystemException
+ */
public SASTokenProvider getSASTokenProvider() throws
AzureBlobFileSystemException {
AuthType authType = getEnum(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
AuthType.SharedKey);
if (authType != AuthType.SAS) {
throw new SASTokenProviderException(String.format(
- "Invalid auth type: %s is being used, expecting SAS", authType));
+ "Invalid auth type: %s is being used, expecting SAS.", authType));
}
try {
- String configKey = FS_AZURE_SAS_TOKEN_PROVIDER_TYPE;
- Class<? extends SASTokenProvider> sasTokenProviderClass =
- getTokenProviderClass(authType, configKey, null,
- SASTokenProvider.class);
-
- Preconditions.checkArgument(sasTokenProviderClass != null,
- String.format("The configuration value for \"%s\" is invalid.",
configKey));
-
- SASTokenProvider sasTokenProvider = ReflectionUtils
- .newInstance(sasTokenProviderClass, rawConfig);
- Preconditions.checkArgument(sasTokenProvider != null,
- String.format("Failed to initialize %s", sasTokenProviderClass));
-
- LOG.trace("Initializing {}", sasTokenProviderClass.getName());
- sasTokenProvider.initialize(rawConfig, accountName);
- LOG.trace("{} init complete", sasTokenProviderClass.getName());
- return sasTokenProvider;
+ Class<? extends SASTokenProvider> customSasTokenProviderImplementation =
+ getTokenProviderClass(authType, FS_AZURE_SAS_TOKEN_PROVIDER_TYPE,
+ null, SASTokenProvider.class);
+ String configuredFixedToken = this.getString(FS_AZURE_SAS_FIXED_TOKEN,
null);
+
+ Preconditions.checkArgument(
+ customSasTokenProviderImplementation != null || configuredFixedToken
!= null,
+ "At least one of the \"%s\" and \"%s\" must be set.",
+ FS_AZURE_SAS_TOKEN_PROVIDER_TYPE, FS_AZURE_SAS_FIXED_TOKEN);
+
+ // Prefer Custom SASTokenProvider Implementation if configured.
+ if (customSasTokenProviderImplementation != null) {
+ LOG.trace("Using Custom SASTokenProvider implementation because it is
given precedence when it is set.");
+ SASTokenProvider sasTokenProvider = ReflectionUtils.newInstance(
+ customSasTokenProviderImplementation, rawConfig);
+ Preconditions.checkArgument(sasTokenProvider != null,
+ "Failed to initialize %s", customSasTokenProviderImplementation);
+
+ LOG.trace("Initializing {}",
customSasTokenProviderImplementation.getName());
+ sasTokenProvider.initialize(rawConfig, accountName);
+ LOG.trace("{} init complete",
customSasTokenProviderImplementation.getName());
+ return sasTokenProvider;
+ } else {
+ LOG.trace("Using FixedSASTokenProvider implementation");
+ FixedSASTokenProvider fixedSASTokenProvider = new
FixedSASTokenProvider(configuredFixedToken);
+ return fixedSASTokenProvider;
+ }
} catch (Exception e) {
Review Comment:
Taken but with SasTokenProviderException
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/FixedSASTokenProvider.java:
##########
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.services;
+
+import java.io.IOException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.azurebfs.extensions.SASTokenProvider;
+import org.apache.hadoop.util.Preconditions;
+
+/**
+ * In house implementation of {@link SASTokenProvider} to use a fixed SAS
token with ABFS.
+ * Use this to avoid implementing a Custom Token Provider just to return fixed
SAS.
+ * Fixed SAS Token to be provided using the config "fs.azure.sas.fixed.token".
+ */
+public class FixedSASTokenProvider implements SASTokenProvider {
+ private String fixedSASToken;
+
+ public FixedSASTokenProvider(final String fixedSASToken) {
+ this.fixedSASToken = fixedSASToken;
+ Preconditions.checkArgument(fixedSASToken != null &&
!fixedSASToken.isEmpty(),
Review Comment:
Taken
> [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider
> Implementation
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-18516
> URL: https://issues.apache.org/jira/browse/HADOOP-18516
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0
> Reporter: Sree Bhattacharyya
> Assignee: Anuj Modi
> Priority: Minor
> Labels: pull-request-available
>
> This PR introduces a new configuration for Fixed SAS Tokens:
> *"fs.azure.sas.fixed.token"*
> Using this new configuration, users can configure a fixed SAS Token in the
> account settings files itself. Ideally, this should be used with SAS Tokens
> that are scoped at a container or account level (Service or Account SAS),
> which can be considered to be a constant for one account or container, over
> multiple operations.
> The other method of using a SAS Token remains valid as well, where a user
> provides a custom implementation of the SASTokenProvider interface, using
> which a SAS Token are obtained.
> When an Account SAS Token is configured as the fixed SAS Token, and it is
> used, it is ensured that operations are within the scope of the SAS Token.
> The code checks for whether the fixed token and the token provider class
> implementation are configured. In the case of both being set, preference is
> given to the custom SASTokenProvider implementation. It must be noted that if
> such an implementation provides a SAS Token which has a lower scope than
> Account SAS, some filesystem and service level operations might be out of
> scope and may not succeed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
