[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384890291
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -266,16 +295,25 @@ public AbfsRestOperation createPath(final String path,
final boolean isFile, fin
return op;
}
- public AbfsRestOperation renamePath(final String source, final String
destination, final String continuation)
+ public AbfsRestOperation renamePath(String source, final String destination,
final String continuation)
throws AzureBlobFileSystemException {
final List requestHeaders = createDefaultHeaders();
-final String encodedRenameSource = urlEncode(FORWARD_SLASH +
this.getFileSystem() + source);
+final AbfsUriQueryBuilder srcQueryBuilder = new AbfsUriQueryBuilder();
+appendSASTokenToQuery(source, SASTokenProvider.RENAME_SOURCE_OPERATION,
srcQueryBuilder);
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384890212
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -266,16 +295,23 @@ public AbfsRestOperation createPath(final String path,
final boolean isFile, fin
return op;
}
- public AbfsRestOperation renamePath(final String source, final String
destination, final String continuation)
+ public AbfsRestOperation renamePath(String source, final String destination,
final String continuation)
throws AzureBlobFileSystemException {
final List requestHeaders = createDefaultHeaders();
-final String encodedRenameSource = urlEncode(FORWARD_SLASH +
this.getFileSystem() + source);
+final AbfsUriQueryBuilder srcQueryBuilder = new AbfsUriQueryBuilder();
+appendSASTokenToQuery(source, SASTokenProvider.RENAME_SOURCE_OPERATION,
srcQueryBuilder);
+String sasToken = srcQueryBuilder.toString();
+
+final String encodedRenameSource =
+urlEncode(FORWARD_SLASH + this.getFileSystem() + source) + sasToken;
+LOG.trace("Rename source queryparam added {}", encodedRenameSource);
Review comment:
As per the recent Thomas's comment, will continue with the Auth check within
appendSASTokenToQuery() method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384889428
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
##
@@ -59,6 +64,20 @@ public String toString() {
throw new IllegalArgumentException("Query string param is not
encode-able: " + entry.getKey() + "=" + entry.getValue());
}
}
+// append SAS Token
+if (sasToken != null) {
+ sasToken = sasToken.startsWith(AbfsHttpConstants.QUESTION_MARK)
+ ? sasToken.substring(1)
+ : sasToken;
Review comment:
Removed the handling of starting '?' in SAS Token query.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384882542
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
##
@@ -65,6 +66,21 @@ public void setup() throws Exception {
super.setup();
}
+ @Test
+ public void testSASTokenProviderInitializeException() throws Exception {
+final AzureBlobFileSystem fs = this.getFileSystem();
+
+final AzureBlobFileSystem testFs = new AzureBlobFileSystem();
+
+MockSASTokenProvider.setThrowExceptionAtInit(true);
Review comment:
Updated PR to fix test failures. Re-ran tests and all passed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384442216
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -578,35 +584,37 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
- public String getAbfsExternalAuthorizationClass() {
-return this.abfsExternalAuthorizationClass;
- }
-
- public AbfsAuthorizer getAbfsAuthorizer() throws IOException {
-String authClassName = getAbfsExternalAuthorizationClass();
-AbfsAuthorizer authorizer = null;
+ public SASTokenProvider getSASTokenProvider() throws
AzureBlobFileSystemException {
+AuthType authType = getEnum(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
AuthType.SharedKey);
+if (authType != AuthType.SAS) {
+ throw new SASTokenProviderException(String.format(
Review comment:
Testcase added.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r384442099 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/abfs.md ## @@ -626,7 +626,7 @@ points for third-parties to integrate their authentication and authorization services into the ABFS client. * `CustomDelegationTokenManager` : adds ability to issue Hadoop Delegation Tokens. -* `AbfsAuthorizer` permits client-side authorization of file operations. +* `SASTokenProvider` permits client-side authorization of file operations. Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384442045
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
##
@@ -59,6 +64,20 @@ public String toString() {
throw new IllegalArgumentException("Query string param is not
encode-able: " + entry.getKey() + "=" + entry.getValue());
}
}
+// append SAS Token
+if (sasToken != null) {
+ sasToken = sasToken.startsWith(AbfsHttpConstants.QUESTION_MARK)
+ ? sasToken.substring(1)
+ : sasToken;
Review comment:
Uribuilder toString() will auto add '?' at the start. Like in the case of
the test SASGenerator code added to this PR, the SAS token generator would end
up having a '?'. SAS token being a query URL it might be better to handle to
presence or absence for '?' in the received token ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384439390
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -266,16 +295,23 @@ public AbfsRestOperation createPath(final String path,
final boolean isFile, fin
return op;
}
- public AbfsRestOperation renamePath(final String source, final String
destination, final String continuation)
+ public AbfsRestOperation renamePath(String source, final String destination,
final String continuation)
throws AzureBlobFileSystemException {
final List requestHeaders = createDefaultHeaders();
-final String encodedRenameSource = urlEncode(FORWARD_SLASH +
this.getFileSystem() + source);
+final AbfsUriQueryBuilder srcQueryBuilder = new AbfsUriQueryBuilder();
+appendSASTokenToQuery(source, SASTokenProvider.RENAME_SOURCE_OPERATION,
srcQueryBuilder);
+String sasToken = srcQueryBuilder.toString();
+
+final String encodedRenameSource =
+urlEncode(FORWARD_SLASH + this.getFileSystem() + source) + sasToken;
+LOG.trace("Rename source queryparam added {}", encodedRenameSource);
Review comment:
Are you suggesting we move the AuthType == SAS check back to the API methods
? It was moved to appendSASTokenToQuery() as it was a repeating code across API
methods.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r384438052 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/SASGenerator.java ## @@ -30,11 +30,12 @@ import org.apache.hadoop.fs.azurebfs.services.AbfsUriQueryBuilder; /** - * Created by tmarq on 2/17/20. + * Test container SAS generator Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384437745
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -578,35 +585,37 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
- public String getAbfsExternalAuthorizationClass() {
-return this.abfsExternalAuthorizationClass;
- }
-
- public AbfsAuthorizer getAbfsAuthorizer() throws IOException {
-String authClassName = getAbfsExternalAuthorizationClass();
-AbfsAuthorizer authorizer = null;
+ public SASTokenProvider getSASTokenProvider() throws
AzureBlobFileSystemException {
+AuthType authType = getEnum(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
AuthType.SharedKey);
+if (authType != AuthType.SAS) {
+ throw new SASTokenProviderException(String.format(
+"Invalid auth type: %s is being used, expecting SAS", authType));
+}
try {
- if (authClassName != null && !authClassName.isEmpty()) {
-@SuppressWarnings("unchecked")
-Class authClass = (Class)
rawConfig.getClassByName(authClassName);
-authorizer = authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
-LOG.trace("Initializing {}", authClassName);
-authorizer.init();
-LOG.trace("{} init complete", authClassName);
+ String configKey = FS_AZURE_SAS_TOKEN_PROVIDER_TYPE;
+ Class sasTokenProviderClass =
+ getClass(configKey, null, SASTokenProvider.class);
+ if (sasTokenProviderClass == null) {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r384357453
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java
##
@@ -189,6 +195,32 @@ public void teardown() throws Exception {
}
}
+
+ public void loadConfiguredFileSystem() throws Exception {
+ // disable auto-creation of filesystem
+
abfsConfig.setBoolean(AZURE_CREATE_REMOTE_FILESYSTEM_DURING_INITIALIZATION,
+ false);
+
+ // AbstractAbfsIntegrationTest always uses a new instance of FileSystem,
+ // need to disable that and force filesystem provided in test configs.
+ String[] authorityParts =
+ (new
java.net.URI(rawConfig.get(FS_AZURE_CONTRACT_TEST_URI))).getRawAuthority().split(
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r383185517 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/SASGenerator.java ## @@ -0,0 +1,127 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.azurebfs.utils; + +import java.io.UnsupportedEncodingException; +import java.time.format.DateTimeFormatter; +import java.time.Instant; +import java.time.ZoneId; +import java.util.Locale; +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; + +import org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants; +import org.apache.hadoop.fs.azurebfs.services.AbfsUriQueryBuilder; + +/** + * Created by tmarq on 2/17/20. Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383185313
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##
@@ -1130,8 +1131,9 @@ private void initializeClient(URI uri, String
fileSystemName, String accountName
SharedKeyCredentials creds = null;
AccessTokenProvider tokenProvider = null;
+SASTokenProvider sasTokenProvider = null;
-if (abfsConfiguration.getAuthType(accountName) == AuthType.SharedKey) {
+if (this.authType == AuthType.SharedKey) {
Review comment:
Removed this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r383185606 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java ## @@ -18,20 +18,23 @@ package org.apache.hadoop.fs.azurebfs; +import java.io.*; import java.util.Arrays; import java.util.List; import java.util.UUID; +import org.apache.hadoop.fs.azurebfs.constants.*; Review comment: Fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383185199
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -578,35 +585,37 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
- public String getAbfsExternalAuthorizationClass() {
-return this.abfsExternalAuthorizationClass;
- }
-
- public AbfsAuthorizer getAbfsAuthorizer() throws IOException {
-String authClassName = getAbfsExternalAuthorizationClass();
-AbfsAuthorizer authorizer = null;
+ public SASTokenProvider getSASTokenProvider() throws
AzureBlobFileSystemException {
+AuthType authType = getEnum(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
AuthType.SharedKey);
+if (authType != AuthType.SAS) {
+ throw new SASTokenProviderException(String.format(
+"Invalid auth type: %s is being used, expecting SAS", authType));
+}
try {
- if (authClassName != null && !authClassName.isEmpty()) {
-@SuppressWarnings("unchecked")
-Class authClass = (Class)
rawConfig.getClassByName(authClassName);
-authorizer = authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
-LOG.trace("Initializing {}", authClassName);
-authorizer.init();
-LOG.trace("{} init complete", authClassName);
+ String configKey = FS_AZURE_SAS_TOKEN_PROVIDER_TYPE;
+ Class sasTokenProviderClass =
+ getClass(configKey, null, SASTokenProvider.class);
+ if (sasTokenProviderClass == null) {
+throw new IllegalArgumentException(
+String.format("The configuration value for \"%s\" is invalid.",
configKey));
}
-} catch (
-IllegalAccessException
-| InstantiationException
-| ClassNotFoundException
-| IllegalArgumentException
-| InvocationTargetException
-| NoSuchMethodException
-| SecurityException
-| AbfsAuthorizationException e) {
- throw new IOException(e);
+
+ SASTokenProvider sasTokenProvider = ReflectionUtils
+ .newInstance(sasTokenProviderClass, rawConfig);
+ if (sasTokenProvider == null) {
+throw new IllegalArgumentException("Failed to initialize " +
sasTokenProviderClass);
+ }
+
+ LOG.trace("Initializing {}", sasTokenProviderClass.getName());
+ sasTokenProvider.initialize(rawConfig, accountName);
+ LOG.trace("{} init complete", sasTokenProviderClass.getName());
+ return sasTokenProvider;
+} catch(IllegalArgumentException e) {
+ throw e;
+} catch (Exception e) {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383185095
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
##
@@ -40,6 +41,10 @@ public void addQuery(final String name, final String value)
{
}
}
+ public void setSASToken(final String sasToken) {
+this.sasToken = sasToken;
Review comment:
Didn't modify as the QueryBuilder will be used in a state where SasToken
need not be mandatorily set. If set, will be used by ToString().
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383184005
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -550,6 +631,17 @@ public AbfsRestOperation checkAccess(String path, String
rwx)
return op;
}
+ private void appendSASTokenToQuery(String path, String operation,
AbfsUriQueryBuilder queryBuilder) throws SASTokenProviderException {
+try {
+ LOG.trace("Fetch SAS token for {}", path);
+ String sasToken = sasTokenProvider.getSASToken(this.accountName,
this.filesystem, path, operation);
+ queryBuilder.setSASToken(sasToken);
+ LOG.trace("SAS token fetch complete for {}", path);
+} catch (Exception ex) {
+ throw new SASTokenProviderException("Failed to acquire a SAS token.",
ex);
Review comment:
Added
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383183882
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -266,17 +299,29 @@ public AbfsRestOperation createPath(final String path,
final boolean isFile, fin
return op;
}
- public AbfsRestOperation renamePath(final String source, final String
destination, final String continuation)
+ public AbfsRestOperation renamePath(String source, final String destination,
final String continuation)
throws AzureBlobFileSystemException {
final List requestHeaders = createDefaultHeaders();
+String sasToken = "";
+
+if (authType == AuthType.SAS) {
+ final AbfsUriQueryBuilder queryBuilder = new AbfsUriQueryBuilder();
+ appendSASTokenToQuery(source, SASTokenProvider.RENAME_SOURCE_OPERATION,
queryBuilder);
+ sasToken = queryBuilder.toString();
+}
-final String encodedRenameSource = urlEncode(FORWARD_SLASH +
this.getFileSystem() + source);
+final String encodedRenameSource =
+urlEncode(FORWARD_SLASH + this.getFileSystem() + source) + sasToken;
requestHeaders.add(new AbfsHttpHeader(X_MS_RENAME_SOURCE,
encodedRenameSource));
requestHeaders.add(new AbfsHttpHeader(IF_NONE_MATCH, STAR));
final AbfsUriQueryBuilder abfsUriQueryBuilder =
createDefaultUriQueryBuilder();
abfsUriQueryBuilder.addQuery(QUERY_PARAM_CONTINUATION, continuation);
+if (authType == AuthType.SAS) {
Review comment:
Moved the authType check to appendSASToken method, though I have not renamed
the method that adds the SAS token. Looks fine the way it is with the check for
authType done within.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383183047
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -550,6 +631,17 @@ public AbfsRestOperation checkAccess(String path, String
rwx)
return op;
}
+ private void appendSASTokenToQuery(String path, String operation,
AbfsUriQueryBuilder queryBuilder) throws SASTokenProviderException {
+try {
+ LOG.trace("Fetch SAS token for {}", path);
Review comment:
Added
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383182606
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
##
@@ -255,6 +283,11 @@ public AbfsRestOperation createPath(final String path,
final boolean isFile, fin
final AbfsUriQueryBuilder abfsUriQueryBuilder =
createDefaultUriQueryBuilder();
abfsUriQueryBuilder.addQuery(QUERY_PARAM_RESOURCE, isFile ? FILE :
DIRECTORY);
+if (authType == AuthType.SAS) {
+ String operation = isFile ? SASTokenProvider.CREATEFILE_OPERATION :
SASTokenProvider.MKDIR_OPERATION;
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r383182474 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java ## @@ -57,25 +59,30 @@ private final SharedKeyCredentials sharedKeyCredentials; private final String xMsVersion = "2018-11-09"; private final ExponentialRetryPolicy retryPolicy; + private final String accountName; private final String filesystem; + private final AuthType authType; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r383181656
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/SASTokenProvider.java
##
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.extensions;
+
+import java.io.IOException;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.security.AccessControlException;
+
+/**
+ * Interface to support SAS authorization.
+ */
+@InterfaceAudience.LimitedPrivate("authorization-subsystems")
+@InterfaceStability.Unstable
+public interface SASTokenProvider {
+
+ public static final String CONCAT_SOURCE_OPERATION = "concat-source";
Review comment:
Fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r382523121 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java ## @@ -38,8 +38,8 @@ import org.apache.hadoop.fs.azure.AzureNativeFileSystemStore; import org.apache.hadoop.fs.azure.NativeAzureFileSystem; import org.apache.hadoop.fs.azure.metrics.AzureFileSystemInstrumentation; -import org.apache.hadoop.fs.azurebfs.constants.FileSystemUriSchemes; -import org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException; +import org.apache.hadoop.fs.azurebfs.constants.*; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r382523369 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java ## @@ -39,6 +38,7 @@ import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Preconditions; +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.SASTokenProviderException; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382519485
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -578,35 +585,37 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
- public String getAbfsExternalAuthorizationClass() {
-return this.abfsExternalAuthorizationClass;
- }
-
- public AbfsAuthorizer getAbfsAuthorizer() throws IOException {
-String authClassName = getAbfsExternalAuthorizationClass();
-AbfsAuthorizer authorizer = null;
+ public SASTokenProvider getSASTokenProvider() throws
AzureBlobFileSystemException {
+AuthType authType = getEnum(FS_AZURE_ACCOUNT_AUTH_TYPE_PROPERTY_NAME,
AuthType.SharedKey);
+if (authType != AuthType.SAS) {
+ throw new SASTokenProviderException(String.format(
+"Invalid auth type: %s is being used, expecting SAS", authType));
+}
try {
- if (authClassName != null && !authClassName.isEmpty()) {
-@SuppressWarnings("unchecked")
-Class authClass = (Class)
rawConfig.getClassByName(authClassName);
-authorizer = authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
-LOG.trace("Initializing {}", authClassName);
-authorizer.init();
-LOG.trace("{} init complete", authClassName);
+ String configKey = FS_AZURE_SAS_TOKEN_PROVIDER_TYPE;
+ Class sasTokenProviderClass =
+ getClass(configKey, null, SASTokenProvider.class);
+ if (sasTokenProviderClass == null) {
Review comment:
Retaining the current code so that exception type thrown can be controlled.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r382512241 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/SASGenerator.java ## @@ -0,0 +1,126 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.azurebfs.utils; + +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; +import java.io.UnsupportedEncodingException; +import java.time.*; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382512112
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
##
@@ -59,6 +64,20 @@ public String toString() {
throw new IllegalArgumentException("Query string param is not
encode-able: " + entry.getKey() + "=" + entry.getValue());
}
}
+// append SAS Token
+if (sasToken != null) {
+ sasToken =
+ sasToken.startsWith(AbfsHttpConstants.QUESTION_MARK) ?
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r382511733 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java ## @@ -29,6 +30,8 @@ import java.util.Locale; import com.google.common.annotations.VisibleForTesting; +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.SASTokenProviderException; +import org.apache.hadoop.fs.azurebfs.extensions.SASTokenProvider; Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r382510841 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsAuthorizationException.java ## @@ -16,16 +16,12 @@ * limitations under the License. */ -package org.apache.hadoop.fs.azurebfs.extensions; - -import java.io.IOException; +package org.apache.hadoop.fs.azurebfs.contracts.exceptions; Review comment: Code is modified. Resolving this comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382510986
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java
##
@@ -294,18 +298,20 @@ private synchronized void writeCurrentBufferToService()
throws IOException {
waitForTaskToComplete();
}
-final Future job = completionService.submit(new Callable() {
+final Future job = completionService.submit(new
Callable() {
Review comment:
Code is modified. Resolving this comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382510664
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/AuthorizationStatus.java
##
@@ -0,0 +1,183 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.authentication;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResource;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResourceResult;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResult;
+
+import java.net.URI;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.HashMap;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.FileSystemConfigurations.DEFAULT_SAS_REFRESH_INTERVAL_BEFORE_EXPIRY;
+
+/**
+ * AuthorizationStatus maintains the status of Authorization and also SAS
+ * token if authorizer is providing it.
+ */
+public class AuthorizationStatus {
+ private boolean isAuthorized;
+ private HashMap sasTokenMap;
Review comment:
Code is modified. Resolving this comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382510394
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -578,21 +587,30 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
- public String getAbfsExternalAuthorizationClass() {
-return this.abfsExternalAuthorizationClass;
- }
+ public AbfsAuthorizer getAbfsAuthorizer() throws
AzureBlobFileSystemException {
+if (this.authorizer != null)
+{
+ return this.authorizer;
+}
- public AbfsAuthorizer getAbfsAuthorizer() throws IOException {
-String authClassName = getAbfsExternalAuthorizationClass();
-AbfsAuthorizer authorizer = null;
+String authClassName = this.abfsAuthorizerClass;
try {
if (authClassName != null && !authClassName.isEmpty()) {
@SuppressWarnings("unchecked")
Class authClass = (Class)
rawConfig.getClassByName(authClassName);
-authorizer = authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
+this.authorizer =
+authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
LOG.trace("Initializing {}", authClassName);
-authorizer.init();
+this.authorizer.init();
+if ((this.authorizer.getAuthType() != AuthType.SAS) && (
+this.authorizer.getAuthType() != AuthType.None)) {
+ throw new AbfsAuthorizationException(
Review comment:
Code is modified. Resolving this comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382510512
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -604,9 +622,10 @@ public AbfsAuthorizer getAbfsAuthorizer() throws
IOException {
| NoSuchMethodException
| SecurityException
| AbfsAuthorizationException e) {
- throw new IOException(e);
+ throw new AbfsAuthorizationException("Unable to initialize "
Review comment:
Code is modified. Resolving this comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r382510065
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsAuthorizerConstants.java
##
@@ -0,0 +1,90 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.constants;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.InvalidAbfsRestOperationException;
+import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperationType;
+
+/**
+ * Defines the string literals that need to be set to authorizer as action
+ */
+public class AbfsAuthorizerConstants {
+
+ public static final String RENAME_SOURCE_ACTION = "rename-source";
+ public static final String RENAME_DESTINATION_ACTION = "rename-destination";
+
+ public static final String CONCAT_SOURCE_ACTION = "concat-source";
+ public static final String CONCAT_TARGET_ACTION = "concat-target";
+
+ public static final String CHECKACCESS_ACTION_PREFIX_ACTION = "access-";
+
+ public static final String LISTSTATUS_ACTION = "liststatus";
+ public static final String DELETE_ACTION = "delete";
+ public static final String CREATEFILE_ACTION = "create";
+ public static final String MKDIR_ACTION = "mkdir";
+ public static final String GETACL_ACTION = "getaclstatus";
+ public static final String GETFILESTATUS_ACTION = "getfilestatus";
+ public static final String SETACL_ACTION = "setacl"; // Modify, removeacl,
+ // setacl
+ public static final String SETOWNER_ACTION = "setowner";
+ public static final String SETPERMISSION_ACTION = "setpermission";
+ public static final String APPEND_ACTION = "write";
+ public static final String READ_ACTION = "read";
+ public static final String EXECUTE_ACTION = "execute";
+
+ /**
+ * Converts AbfsRestOperation to Authorizer action
+ * @param opType
+ * @return
+ */
+ public static String getAction(AbfsRestOperationType opType)
+ throws InvalidAbfsRestOperationException {
+switch (opType) {
+case ListPaths:
+ return LISTSTATUS_ACTION;
+case RenamePath:
+ return RENAME_DESTINATION_ACTION;
+case GetAcl:
+ return GETACL_ACTION;
+case GetPathStatus:
+ return GETFILESTATUS_ACTION;
+case SetAcl:
+ return SETACL_ACTION;
+case SetOwner:
+ return SETOWNER_ACTION;
+case SetPermissions:
+ return SETPERMISSION_ACTION;
+case Append:
+case Flush:
+ return APPEND_ACTION;
+case ReadFile:
+ return READ_ACTION;
+case DeletePath:
+ return DELETE_ACTION;
+case CreatePath:
+ return CREATEFILE_ACTION;
+case Mkdir:
+ return MKDIR_ACTION;
+default:
+ throw new InvalidAbfsRestOperationException(
Review comment:
Code has changed since this version. Resolving the comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379628733
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/AuthorizationStatus.java
##
@@ -0,0 +1,183 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.authentication;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResource;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResourceResult;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResult;
+
+import java.net.URI;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.HashMap;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.FileSystemConfigurations.DEFAULT_SAS_REFRESH_INTERVAL_BEFORE_EXPIRY;
+
+/**
+ * AuthorizationStatus maintains the status of Authorization and also SAS
+ * token if authorizer is providing it.
+ */
+public class AuthorizationStatus {
+ private boolean isAuthorized;
+ private HashMap sasTokenMap;
Review comment:
AuthorizationStatus instance represents a single request response from
authorizer and a copy is saved only Abfs input or output stream, which will be
operating one I/O operation at a time.
As the instance is not shared at Abfs filesystem level across requests,
concurrency doesnt need handling,
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379641302
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
##
@@ -36,331 +40,485 @@
import static org.apache.hadoop.fs.permission.AclEntryScope.ACCESS;
import static org.apache.hadoop.fs.permission.AclEntryType.GROUP;
import static org.apache.hadoop.test.LambdaTestUtils.intercept;
-import static org.junit.Assume.assumeTrue;
/**
* Test Perform Authorization Check operation
*/
-public class ITestAzureBlobFileSystemAuthorization extends
AbstractAbfsIntegrationTest {
-
- private static final Path TEST_READ_ONLY_FILE_PATH_0 = new
Path(TEST_READ_ONLY_FILE_0);
- private static final Path TEST_READ_ONLY_FOLDER_PATH = new
Path(TEST_READ_ONLY_FOLDER);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_0 = new
Path(TEST_WRITE_ONLY_FILE_0);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_1 = new
Path(TEST_WRITE_ONLY_FILE_1);
- private static final Path TEST_READ_WRITE_FILE_PATH_0 = new
Path(TEST_READ_WRITE_FILE_0);
- private static final Path TEST_READ_WRITE_FILE_PATH_1 = new
Path(TEST_READ_WRITE_FILE_1);
- private static final Path TEST_WRITE_ONLY_FOLDER_PATH = new
Path(TEST_WRITE_ONLY_FOLDER);
- private static final Path TEST_WRITE_THEN_READ_ONLY_PATH = new
Path(TEST_WRITE_THEN_READ_ONLY);
- private static final String TEST_AUTHZ_CLASS =
"org.apache.hadoop.fs.azurebfs.extensions.MockAbfsAuthorizer";
+public class ITestAzureBlobFileSystemAuthorization
+extends AbstractAbfsIntegrationTest {
+
+ private static final String TEST_READ_ONLY_FILE_PATH_PREFIX_0 =
+ TEST_READ_ONLY_FILE_0;
+ private static final String TEST_READ_ONLY_FILE_PATH_PREFIX_1 =
+ TEST_READ_ONLY_FILE_1;
+ private static final String TEST_READ_ONLY_FOLDER_PATH_PREFIX =
+ TEST_READ_ONLY_FOLDER;
+ private static final String TEST_WRITE_ONLY_FILE_PATH_PREFIX_0 =
+ TEST_WRITE_ONLY_FILE_0;
+ private static final String TEST_WRITE_ONLY_FILE_PATH_PREFIX_1 =
+ TEST_WRITE_ONLY_FILE_1;
+ private static final String TEST_READ_WRITE_FILE_PATH_PREFIX_0 =
+ TEST_READ_WRITE_FILE_0;
+ private static final String TEST_READ_WRITE_FILE_PATH_PREFIX_1 =
+ TEST_READ_WRITE_FILE_1;
+ private static final String TEST_WRITE_ONLY_FOLDER_PATH_PREFIX =
+ TEST_WRITE_ONLY_FOLDER;
+ private static final String TEST_WRITE_THEN_READ_ONLY_PATH_PREFIX =
+ TEST_WRITE_THEN_READ_ONLY;
+ private static final String TEST_AUTHZ_CLASS =
+ "org.apache.hadoop.fs" + ".azurebfs.extensions.MockAbfsAuthorizer";
private static final String TEST_USER = UUID.randomUUID().toString();
private static final String TEST_GROUP = UUID.randomUUID().toString();
private static final String BAR = UUID.randomUUID().toString();
+ @Rule
+ public TestName name = new TestName();
public ITestAzureBlobFileSystemAuthorization() throws Exception {
}
@Override
public void setup() throws Exception {
-
this.getConfiguration().set(ConfigurationKeys.ABFS_EXTERNAL_AUTHORIZATION_CLASS,
TEST_AUTHZ_CLASS);
+boolean isHNSEnabled = this.getConfiguration().getBoolean(
+TestConfigurationKeys.FS_AZURE_TEST_NAMESPACE_ENABLED_ACCOUNT, false);
+Assume.assumeTrue(isHNSEnabled == true);
+this.getConfiguration().setAbfsAuthorizerClass(TEST_AUTHZ_CLASS);
+loadAuthorizer();
super.setup();
}
@Test
public void testOpenFileWithInvalidPath() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-intercept(IllegalArgumentException.class,
-()-> {
- fs.open(new Path("")).close();
+intercept(IllegalArgumentException.class, () -> {
+ fs.open(new Path("")).close();
});
}
@Test
public void testOpenFileAuthorized() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-fs.create(TEST_WRITE_THEN_READ_ONLY_PATH).close();
-fs.open(TEST_WRITE_THEN_READ_ONLY_PATH).close();
+Path testFilePath = new Path(
+TEST_WRITE_THEN_READ_ONLY_PATH_PREFIX + name.getMethodName());
+((MockAbfsAuthorizer) fs.getAbfsStore().getAuthorizer())
+.setwriteThenReadOnly(testFilePath.getName(),
WriteReadMode.WRITE_MODE);
+fs.create(testFilePath).close();
+((MockAbfsAuthorizer) fs.getAbfsStore().getAuthorizer())
+.setwriteThenReadOnly(testFilePath.getName(), WriteReadMode.READ_MODE);
+fs.open(testFilePath).close();
}
@Test
public void testOpenFileUnauthorized() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-fs.create(TEST_WRITE_ONLY_FILE_PATH_0).close();
-intercept(AbfsAuthorizationException.class,
-()-> {
- fs.open(TEST_WRITE_ONLY_FILE_PATH_0).close();
+Path testFilePath = new Path(
+TEST_WRITE_ONLY_FILE_PATH_PREFIX_0 +
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379640795 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java ## @@ -20,10 +20,21 @@ import java.io.IOException; import java.net.HttpURLConnection; +import java.net.MalformedURLException; +import java.net.URI; +import java.net.URISyntaxException; import java.net.URL; import java.net.UnknownHostException; import java.util.List; +import org.apache.hadoop.fs.azurebfs.authentication.AuthorizationStatus; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379639612
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsAuthorizerUnhandledException.java
##
@@ -0,0 +1,30 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.contracts.exceptions;
+
+/**
+ * Exception for all unhandled errors while communicating with authorizer
+ */
+public class AbfsAuthorizerUnhandledException
+extends AzureBlobFileSystemException {
+ public AbfsAuthorizerUnhandledException(Exception innerException) {
+super("An unhandled Authorizer request processing exception",
Review comment:
Updated the exception message
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379639316
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/SasTokenData.java
##
@@ -0,0 +1,27 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.authentication;
+
+import java.time.Instant;
+
+public class SasTokenData {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379628885
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/AuthorizationStatus.java
##
@@ -0,0 +1,183 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.authentication;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResource;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResourceResult;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResult;
+
+import java.net.URI;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.HashMap;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.FileSystemConfigurations.DEFAULT_SAS_REFRESH_INTERVAL_BEFORE_EXPIRY;
+
+/**
+ * AuthorizationStatus maintains the status of Authorization and also SAS
+ * token if authorizer is providing it.
+ */
+public class AuthorizationStatus {
+ private boolean isAuthorized;
+ private HashMap sasTokenMap;
+
+ public AuthorizationStatus() {
+sasTokenMap = new HashMap<>();
+ }
+
+ /**
+ * Fetch the SAS token
+ *
+ * @return
+ */
+ public String getSasTokenQuery(URI storePathUri) {
+if (sasTokenMap.containsKey(storePathUri)) {
+ SasTokenData sasTokenData = sasTokenMap.get(storePathUri);
+ if (isValidSas(sasTokenData)) {
+return sasTokenData.sasToken;
+ }
+}
+
+return null;
+ }
+
+ /**
+ * Update authTokenMap
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379629072
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/AuthorizationStatus.java
##
@@ -0,0 +1,183 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.authentication;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResource;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResourceResult;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResult;
+
+import java.net.URI;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.HashMap;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.FileSystemConfigurations.DEFAULT_SAS_REFRESH_INTERVAL_BEFORE_EXPIRY;
+
+/**
+ * AuthorizationStatus maintains the status of Authorization and also SAS
+ * token if authorizer is providing it.
+ */
+public class AuthorizationStatus {
+ private boolean isAuthorized;
+ private HashMap sasTokenMap;
+
+ public AuthorizationStatus() {
+sasTokenMap = new HashMap<>();
+ }
+
+ /**
+ * Fetch the SAS token
+ *
+ * @return
+ */
+ public String getSasTokenQuery(URI storePathUri) {
+if (sasTokenMap.containsKey(storePathUri)) {
+ SasTokenData sasTokenData = sasTokenMap.get(storePathUri);
+ if (isValidSas(sasTokenData)) {
+return sasTokenData.sasToken;
+ }
+}
+
+return null;
+ }
+
+ /**
+ * Update authTokenMap
+ * Also update the refresh interval for each SAS token.
+ *
+ * @param authorizationResource
+ * @param authResult - Authorizer AuthorizationResult
+ */
+ public void setSasToken(AuthorizationResource[] authorizationResource,
+ AuthorizationResult authResult) throws AbfsAuthorizationException {
+
+AuthorizationResourceResult[] resourceResult = authResult
+.getAuthResourceResult();
+
+int i = 0;
+for (AuthorizationResourceResult singleResourceAuth : resourceResult) {
+ // First check if the requested resource matches the resource
+ // for which authToken is returned
+ AuthorizationResource authorizationRequestedForResource =
+ authorizationResource[i];
+ if ((singleResourceAuth == null) || (singleResourceAuth.storePathUri
+ == null) || (singleResourceAuth.authorizerAction == null)) {
+throw new AbfsAuthorizationException("Invalid authorization response");
Review comment:
Have put in detailed checks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379628733
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/AuthorizationStatus.java
##
@@ -0,0 +1,183 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.authentication;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResource;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResourceResult;
+import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResult;
+
+import java.net.URI;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.HashMap;
+
+import static
org.apache.hadoop.fs.azurebfs.constants.FileSystemConfigurations.DEFAULT_SAS_REFRESH_INTERVAL_BEFORE_EXPIRY;
+
+/**
+ * AuthorizationStatus maintains the status of Authorization and also SAS
+ * token if authorizer is providing it.
+ */
+public class AuthorizationStatus {
+ private boolean isAuthorized;
+ private HashMap sasTokenMap;
Review comment:
AuthorizationStatus instance is saved at a time by the Abfs inpt or output
stream, which will be operating one I/O operation at a time.
As the instance is not shared at Abfs filesystem level across requests,
concurrency doesnt need handling,
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379627975
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
##
@@ -36,331 +40,485 @@
import static org.apache.hadoop.fs.permission.AclEntryScope.ACCESS;
import static org.apache.hadoop.fs.permission.AclEntryType.GROUP;
import static org.apache.hadoop.test.LambdaTestUtils.intercept;
-import static org.junit.Assume.assumeTrue;
/**
* Test Perform Authorization Check operation
*/
-public class ITestAzureBlobFileSystemAuthorization extends
AbstractAbfsIntegrationTest {
-
- private static final Path TEST_READ_ONLY_FILE_PATH_0 = new
Path(TEST_READ_ONLY_FILE_0);
- private static final Path TEST_READ_ONLY_FOLDER_PATH = new
Path(TEST_READ_ONLY_FOLDER);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_0 = new
Path(TEST_WRITE_ONLY_FILE_0);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_1 = new
Path(TEST_WRITE_ONLY_FILE_1);
- private static final Path TEST_READ_WRITE_FILE_PATH_0 = new
Path(TEST_READ_WRITE_FILE_0);
- private static final Path TEST_READ_WRITE_FILE_PATH_1 = new
Path(TEST_READ_WRITE_FILE_1);
- private static final Path TEST_WRITE_ONLY_FOLDER_PATH = new
Path(TEST_WRITE_ONLY_FOLDER);
- private static final Path TEST_WRITE_THEN_READ_ONLY_PATH = new
Path(TEST_WRITE_THEN_READ_ONLY);
- private static final String TEST_AUTHZ_CLASS =
"org.apache.hadoop.fs.azurebfs.extensions.MockAbfsAuthorizer";
+public class ITestAzureBlobFileSystemAuthorization
+extends AbstractAbfsIntegrationTest {
+
+ private static final String TEST_READ_ONLY_FILE_PATH_PREFIX_0 =
+ TEST_READ_ONLY_FILE_0;
+ private static final String TEST_READ_ONLY_FILE_PATH_PREFIX_1 =
+ TEST_READ_ONLY_FILE_1;
+ private static final String TEST_READ_ONLY_FOLDER_PATH_PREFIX =
+ TEST_READ_ONLY_FOLDER;
+ private static final String TEST_WRITE_ONLY_FILE_PATH_PREFIX_0 =
+ TEST_WRITE_ONLY_FILE_0;
+ private static final String TEST_WRITE_ONLY_FILE_PATH_PREFIX_1 =
+ TEST_WRITE_ONLY_FILE_1;
+ private static final String TEST_READ_WRITE_FILE_PATH_PREFIX_0 =
+ TEST_READ_WRITE_FILE_0;
+ private static final String TEST_READ_WRITE_FILE_PATH_PREFIX_1 =
+ TEST_READ_WRITE_FILE_1;
+ private static final String TEST_WRITE_ONLY_FOLDER_PATH_PREFIX =
+ TEST_WRITE_ONLY_FOLDER;
+ private static final String TEST_WRITE_THEN_READ_ONLY_PATH_PREFIX =
+ TEST_WRITE_THEN_READ_ONLY;
+ private static final String TEST_AUTHZ_CLASS =
+ "org.apache.hadoop.fs" + ".azurebfs.extensions.MockAbfsAuthorizer";
private static final String TEST_USER = UUID.randomUUID().toString();
private static final String TEST_GROUP = UUID.randomUUID().toString();
private static final String BAR = UUID.randomUUID().toString();
+ @Rule
+ public TestName name = new TestName();
public ITestAzureBlobFileSystemAuthorization() throws Exception {
}
@Override
public void setup() throws Exception {
-
this.getConfiguration().set(ConfigurationKeys.ABFS_EXTERNAL_AUTHORIZATION_CLASS,
TEST_AUTHZ_CLASS);
+boolean isHNSEnabled = this.getConfiguration().getBoolean(
+TestConfigurationKeys.FS_AZURE_TEST_NAMESPACE_ENABLED_ACCOUNT, false);
+Assume.assumeTrue(isHNSEnabled == true);
+this.getConfiguration().setAbfsAuthorizerClass(TEST_AUTHZ_CLASS);
+loadAuthorizer();
super.setup();
}
@Test
public void testOpenFileWithInvalidPath() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-intercept(IllegalArgumentException.class,
-()-> {
- fs.open(new Path("")).close();
+intercept(IllegalArgumentException.class, () -> {
+ fs.open(new Path("")).close();
});
}
@Test
public void testOpenFileAuthorized() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-fs.create(TEST_WRITE_THEN_READ_ONLY_PATH).close();
-fs.open(TEST_WRITE_THEN_READ_ONLY_PATH).close();
+Path testFilePath = new Path(
+TEST_WRITE_THEN_READ_ONLY_PATH_PREFIX + name.getMethodName());
+((MockAbfsAuthorizer) fs.getAbfsStore().getAuthorizer())
+.setwriteThenReadOnly(testFilePath.getName(),
WriteReadMode.WRITE_MODE);
+fs.create(testFilePath).close();
+((MockAbfsAuthorizer) fs.getAbfsStore().getAuthorizer())
+.setwriteThenReadOnly(testFilePath.getName(), WriteReadMode.READ_MODE);
+fs.open(testFilePath).close();
}
@Test
public void testOpenFileUnauthorized() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-fs.create(TEST_WRITE_ONLY_FILE_PATH_0).close();
-intercept(AbfsAuthorizationException.class,
-()-> {
- fs.open(TEST_WRITE_ONLY_FILE_PATH_0).close();
+Path testFilePath = new Path(
+TEST_WRITE_ONLY_FILE_PATH_PREFIX_0 +
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379627813
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
##
@@ -36,331 +40,485 @@
import static org.apache.hadoop.fs.permission.AclEntryScope.ACCESS;
import static org.apache.hadoop.fs.permission.AclEntryType.GROUP;
import static org.apache.hadoop.test.LambdaTestUtils.intercept;
-import static org.junit.Assume.assumeTrue;
/**
* Test Perform Authorization Check operation
*/
-public class ITestAzureBlobFileSystemAuthorization extends
AbstractAbfsIntegrationTest {
-
- private static final Path TEST_READ_ONLY_FILE_PATH_0 = new
Path(TEST_READ_ONLY_FILE_0);
- private static final Path TEST_READ_ONLY_FOLDER_PATH = new
Path(TEST_READ_ONLY_FOLDER);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_0 = new
Path(TEST_WRITE_ONLY_FILE_0);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_1 = new
Path(TEST_WRITE_ONLY_FILE_1);
- private static final Path TEST_READ_WRITE_FILE_PATH_0 = new
Path(TEST_READ_WRITE_FILE_0);
- private static final Path TEST_READ_WRITE_FILE_PATH_1 = new
Path(TEST_READ_WRITE_FILE_1);
- private static final Path TEST_WRITE_ONLY_FOLDER_PATH = new
Path(TEST_WRITE_ONLY_FOLDER);
- private static final Path TEST_WRITE_THEN_READ_ONLY_PATH = new
Path(TEST_WRITE_THEN_READ_ONLY);
- private static final String TEST_AUTHZ_CLASS =
"org.apache.hadoop.fs.azurebfs.extensions.MockAbfsAuthorizer";
+public class ITestAzureBlobFileSystemAuthorization
+extends AbstractAbfsIntegrationTest {
+
+ private static final String TEST_READ_ONLY_FILE_PATH_PREFIX_0 =
+ TEST_READ_ONLY_FILE_0;
+ private static final String TEST_READ_ONLY_FILE_PATH_PREFIX_1 =
+ TEST_READ_ONLY_FILE_1;
+ private static final String TEST_READ_ONLY_FOLDER_PATH_PREFIX =
+ TEST_READ_ONLY_FOLDER;
+ private static final String TEST_WRITE_ONLY_FILE_PATH_PREFIX_0 =
+ TEST_WRITE_ONLY_FILE_0;
+ private static final String TEST_WRITE_ONLY_FILE_PATH_PREFIX_1 =
+ TEST_WRITE_ONLY_FILE_1;
+ private static final String TEST_READ_WRITE_FILE_PATH_PREFIX_0 =
+ TEST_READ_WRITE_FILE_0;
+ private static final String TEST_READ_WRITE_FILE_PATH_PREFIX_1 =
+ TEST_READ_WRITE_FILE_1;
+ private static final String TEST_WRITE_ONLY_FOLDER_PATH_PREFIX =
+ TEST_WRITE_ONLY_FOLDER;
+ private static final String TEST_WRITE_THEN_READ_ONLY_PATH_PREFIX =
+ TEST_WRITE_THEN_READ_ONLY;
+ private static final String TEST_AUTHZ_CLASS =
+ "org.apache.hadoop.fs" + ".azurebfs.extensions.MockAbfsAuthorizer";
private static final String TEST_USER = UUID.randomUUID().toString();
private static final String TEST_GROUP = UUID.randomUUID().toString();
private static final String BAR = UUID.randomUUID().toString();
+ @Rule
+ public TestName name = new TestName();
public ITestAzureBlobFileSystemAuthorization() throws Exception {
}
@Override
public void setup() throws Exception {
-
this.getConfiguration().set(ConfigurationKeys.ABFS_EXTERNAL_AUTHORIZATION_CLASS,
TEST_AUTHZ_CLASS);
+boolean isHNSEnabled = this.getConfiguration().getBoolean(
+TestConfigurationKeys.FS_AZURE_TEST_NAMESPACE_ENABLED_ACCOUNT, false);
+Assume.assumeTrue(isHNSEnabled == true);
+this.getConfiguration().setAbfsAuthorizerClass(TEST_AUTHZ_CLASS);
+loadAuthorizer();
super.setup();
}
@Test
public void testOpenFileWithInvalidPath() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-intercept(IllegalArgumentException.class,
-()-> {
- fs.open(new Path("")).close();
+intercept(IllegalArgumentException.class, () -> {
+ fs.open(new Path("")).close();
});
}
@Test
public void testOpenFileAuthorized() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-fs.create(TEST_WRITE_THEN_READ_ONLY_PATH).close();
-fs.open(TEST_WRITE_THEN_READ_ONLY_PATH).close();
+Path testFilePath = new Path(
+TEST_WRITE_THEN_READ_ONLY_PATH_PREFIX + name.getMethodName());
+((MockAbfsAuthorizer) fs.getAbfsStore().getAuthorizer())
+.setwriteThenReadOnly(testFilePath.getName(),
WriteReadMode.WRITE_MODE);
+fs.create(testFilePath).close();
+((MockAbfsAuthorizer) fs.getAbfsStore().getAuthorizer())
+.setwriteThenReadOnly(testFilePath.getName(), WriteReadMode.READ_MODE);
+fs.open(testFilePath).close();
}
@Test
public void testOpenFileUnauthorized() throws Exception {
final AzureBlobFileSystem fs = this.getFileSystem();
-fs.create(TEST_WRITE_ONLY_FILE_PATH_0).close();
-intercept(AbfsAuthorizationException.class,
-()-> {
- fs.open(TEST_WRITE_ONLY_FILE_PATH_0).close();
+Path testFilePath = new Path(
Review comment:
Refactored
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379625885
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java
##
@@ -128,6 +134,46 @@ protected AbstractAbfsIntegrationTest() throws Exception {
}
+ public void loadAuthorizer() throws Exception {
+this.authorizer = abfsConfig.getAbfsAuthorizer();
+
+// if authorizerClass is configured, auto-creation of filesystem is
disabled
+if (authorizer == null) {
+ throw new InvalidConfigurationValueException(
+ "loadAuthorizer failed as " + "authorizer class is not configured");
+}
+
+// if authorizerClass is configured, auto-creation of filesystem is
disabled
+abfsConfig.setBoolean(AZURE_CREATE_REMOTE_FILESYSTEM_DURING_INITIALIZATION,
+false);
+
+// AbstractAbfsIntegrationTest always uses a new instance of FileSystem,
+// need to disable that and force filesystem provided in test configs.
+String[] authorityParts = authorityParts(
+new java.net.URI(rawConfig.get(FS_AZURE_CONTRACT_TEST_URI)));
+this.fileSystemName = authorityParts[0];
+
+// Reset URL with configured filesystem
+final String abfsUrl =
+this.getFileSystemName() + "@" + this.getAccountName();
+URI defaultUri = null;
+
+try {
+ defaultUri = new URI(abfsScheme, abfsUrl, null, null, null);
+} catch (Exception ex) {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379625630 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java ## @@ -21,6 +21,7 @@ import java.net.URL; import java.util.regex.Pattern; +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.AzureBlobFileSystemException; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379625337 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockAbfsAuthorizer.java ## @@ -41,47 +55,216 @@ public static final String TEST_WRITE_ONLY_FOLDER = "writeOnlyFolder"; public static final String TEST_READ_WRITE_FILE_0 = "readWriteFile0"; public static final String TEST_READ_WRITE_FILE_1 = "readWriteFile1"; - public static final String TEST_WRITE_THEN_READ_ONLY = "writeThenReadOnlyFile"; + public static final String TEST_WRITE_THEN_READ_ONLY = + "writeThenReadOnlyFile"; + private static final Set apiAuthorizerActions = new HashSet(); + public String accountName; private Configuration conf; - private Set readOnlyPaths = new HashSet(); - private Set writeOnlyPaths = new HashSet(); - private Set readWritePaths = new HashSet(); - private int writeThenReadOnly = 0; + private Set readOnlyPathsPrefixes = new HashSet(); Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379625266 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockAbfsAuthorizer.java ## @@ -41,47 +55,216 @@ public static final String TEST_WRITE_ONLY_FOLDER = "writeOnlyFolder"; public static final String TEST_READ_WRITE_FILE_0 = "readWriteFile0"; public static final String TEST_READ_WRITE_FILE_1 = "readWriteFile1"; - public static final String TEST_WRITE_THEN_READ_ONLY = "writeThenReadOnlyFile"; + public static final String TEST_WRITE_THEN_READ_ONLY = + "writeThenReadOnlyFile"; + private static final Set apiAuthorizerActions = new HashSet(); Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379625106 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockAbfsAuthorizer.java ## @@ -18,21 +18,35 @@ package org.apache.hadoop.fs.azurebfs.extensions; -import java.io.IOException; +import org.apache.hadoop.conf.Configuration; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379624955
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##
@@ -1131,20 +1142,38 @@ private void initializeClient(URI uri, String
fileSystemName, String accountName
SharedKeyCredentials creds = null;
AccessTokenProvider tokenProvider = null;
-if (abfsConfiguration.getAuthType(accountName) == AuthType.SharedKey) {
- LOG.trace("Fetching SharedKey credentials");
- int dotIndex = accountName.indexOf(AbfsHttpConstants.DOT);
- if (dotIndex <= 0) {
-throw new InvalidUriException(
-uri.toString() + " - account name is not fully qualified.");
- }
- creds = new SharedKeyCredentials(accountName.substring(0, dotIndex),
+AuthType authType = abfsConfiguration.getAuthType(accountName);
+AbfsAuthorizer authorizer = abfsConfiguration.getAbfsAuthorizer();
+
+// Create cred instance based on account level Auth config if
+// - authorizer is not configured
+// - or authroizer configured will not provide SAS
+if ((authorizer == null) || ((authorizer.getAuthType() != AuthType.SAS))) {
+ switch (authType) {
+ case SharedKey:
+LOG.trace("Fetching SharedKey credentials");
+int dotIndex = accountName.indexOf(AbfsHttpConstants.DOT);
+if (dotIndex <= 0) {
+ throw new InvalidUriException(
+ uri.toString() + " - account name is not fully qualified.");
+}
+creds = new SharedKeyCredentials(accountName.substring(0, dotIndex),
abfsConfiguration.getStorageAccountKey());
-} else {
- LOG.trace("Fetching token provider");
- tokenProvider = abfsConfiguration.getTokenProvider();
- ExtensionHelper.bind(tokenProvider, uri,
-abfsConfiguration.getRawConfiguration());
+break;
+ case Custom:
+ case OAuth:
+LOG.trace("Fetching token provider");
+tokenProvider = abfsConfiguration.getTokenProvider();
+ExtensionHelper
+.bind(tokenProvider, uri, abfsConfiguration.getRawConfiguration());
+break;
+ case SAS:
+throw new UnsupportedOperationException(
+"There is no " + "Authorizer configured");
+ case None:
+throw new UnsupportedOperationException(
+"No authentication means " + "configured");
Review comment:
Fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379624778
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##
@@ -1131,20 +1142,38 @@ private void initializeClient(URI uri, String
fileSystemName, String accountName
SharedKeyCredentials creds = null;
AccessTokenProvider tokenProvider = null;
-if (abfsConfiguration.getAuthType(accountName) == AuthType.SharedKey) {
- LOG.trace("Fetching SharedKey credentials");
- int dotIndex = accountName.indexOf(AbfsHttpConstants.DOT);
- if (dotIndex <= 0) {
-throw new InvalidUriException(
-uri.toString() + " - account name is not fully qualified.");
- }
- creds = new SharedKeyCredentials(accountName.substring(0, dotIndex),
+AuthType authType = abfsConfiguration.getAuthType(accountName);
+AbfsAuthorizer authorizer = abfsConfiguration.getAbfsAuthorizer();
+
+// Create cred instance based on account level Auth config if
+// - authorizer is not configured
+// - or authroizer configured will not provide SAS
Review comment:
Fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379624566
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##
@@ -1131,20 +1142,38 @@ private void initializeClient(URI uri, String
fileSystemName, String accountName
SharedKeyCredentials creds = null;
AccessTokenProvider tokenProvider = null;
-if (abfsConfiguration.getAuthType(accountName) == AuthType.SharedKey) {
- LOG.trace("Fetching SharedKey credentials");
- int dotIndex = accountName.indexOf(AbfsHttpConstants.DOT);
- if (dotIndex <= 0) {
-throw new InvalidUriException(
-uri.toString() + " - account name is not fully qualified.");
- }
- creds = new SharedKeyCredentials(accountName.substring(0, dotIndex),
+AuthType authType = abfsConfiguration.getAuthType(accountName);
+AbfsAuthorizer authorizer = abfsConfiguration.getAbfsAuthorizer();
+
+// Create cred instance based on account level Auth config if
+// - authorizer is not configured
+// - or authroizer configured will not provide SAS
+if ((authorizer == null) || ((authorizer.getAuthType() != AuthType.SAS))) {
+ switch (authType) {
+ case SharedKey:
+LOG.trace("Fetching SharedKey credentials");
+int dotIndex = accountName.indexOf(AbfsHttpConstants.DOT);
+if (dotIndex <= 0) {
+ throw new InvalidUriException(
+ uri.toString() + " - account name is not fully qualified.");
+}
+creds = new SharedKeyCredentials(accountName.substring(0, dotIndex),
abfsConfiguration.getStorageAccountKey());
-} else {
- LOG.trace("Fetching token provider");
- tokenProvider = abfsConfiguration.getTokenProvider();
- ExtensionHelper.bind(tokenProvider, uri,
-abfsConfiguration.getRawConfiguration());
+break;
+ case Custom:
+ case OAuth:
+LOG.trace("Fetching token provider");
+tokenProvider = abfsConfiguration.getTokenProvider();
+ExtensionHelper
+.bind(tokenProvider, uri, abfsConfiguration.getRawConfiguration());
+break;
+ case SAS:
+throw new UnsupportedOperationException(
+"There is no " + "Authorizer configured");
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379624400
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##
@@ -233,6 +234,16 @@ String decodeAttribute(byte[] value) throws
UnsupportedEncodingException {
public boolean getIsNamespaceEnabled() throws AzureBlobFileSystemException {
if (!isNamespaceEnabledSet) {
+ // If an authorizer is set, account needs to be HNS enabled. No need
+ // for server calls to check the same.
+ if (this.abfsConfiguration.getAbfsAuthorizer() != null)
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379624234
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java
##
@@ -112,10 +113,13 @@ public void initialize(URI uri, Configuration
configuration)
LOG.trace("AzureBlobFileSystemStore init complete");
final AbfsConfiguration abfsConfiguration =
abfsStore.getAbfsConfiguration();
+this.authorizer = abfsConfiguration.getAbfsAuthorizer();
this.setWorkingDirectory(this.getHomeDirectory());
-if (abfsConfiguration.getCreateRemoteFileSystemDuringInitialization()) {
+// auto-creation of container is disabled when authorizer is configured
+if (abfsConfiguration.getCreateRemoteFileSystemDuringInitialization() && (
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379623925
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -496,6 +497,14 @@ public boolean shouldTrackLatency() {
return this.trackLatency;
}
+ /**
+ * Return the accountName for the config generated
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379623593 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java ## @@ -18,8 +18,17 @@ package org.apache.hadoop.fs.azurebfs.utils; +import org.apache.http.client.utils.URIBuilder; Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379623419
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java
##
@@ -351,14 +358,21 @@ private synchronized void
flushWrittenBytesToServiceInternal(final long offset,
AbfsPerfTracker tracker = client.getAbfsPerfTracker();
try (AbfsPerfInfo perfInfo = new AbfsPerfInfo(tracker,
"flushWrittenBytesToServiceInternal", "flush")) {
- AbfsRestOperation op = client.flush(path, offset, retainUncommitedData,
isClose);
+ AbfsRestOperation op = client.flush(path, offset, retainUncommitedData,
+ isClose, this.authzStatus);
perfInfo.registerResult(op.getResult()).registerSuccess(true);
} catch (AzureBlobFileSystemException ex) {
if (ex instanceof AbfsRestOperationException) {
if (((AbfsRestOperationException) ex).getStatusCode() ==
HttpURLConnection.HTTP_NOT_FOUND) {
throw new FileNotFoundException(ex.getMessage());
}
}
+
+ if ((ex instanceof AbfsAuthorizationException)
Review comment:
Updated as suggested.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379623215 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java ## @@ -106,7 +110,7 @@ public AbfsOutputStream( 10L, TimeUnit.SECONDS, new LinkedBlockingQueue<>()); -this.completionService = new ExecutorCompletionService<>(this.threadExecutor); +this.completionService = new ExecutorCompletionService(this.threadExecutor); Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379622749 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AuthorizationResult.java ## @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.azurebfs.extensions; + +/** + * AuthorizationResult will be returned by Authorizer with store path URI and + * supported Auth token if the AuthorizationResource is found authorized for + * specified action Review comment: The intention was to make a copy of the values too. Have made the changes accordingly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379622912
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java
##
@@ -228,7 +230,9 @@ int readRemote(long position, byte[] b, int offset, int
length) throws IOExcepti
final AbfsRestOperation op;
AbfsPerfTracker tracker = client.getAbfsPerfTracker();
try (AbfsPerfInfo perfInfo = new AbfsPerfInfo(tracker, "readRemote",
"read")) {
- op = client.read(path, position, b, offset, length, tolerateOobAppends ?
"*" : eTag);
+ op = client.read(path, position, b, offset, length, tolerateOobAppends
+ ? "*" : eTag, this.authzStatus);
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379622525 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AuthorizationResource.java ## @@ -0,0 +1,29 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.azurebfs.extensions; + +import java.net.URI; + +/** + * AuthorizationResource with action and store path URI Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379622231
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizer.java
##
@@ -18,40 +18,50 @@
package org.apache.hadoop.fs.azurebfs.extensions;
-import java.io.IOException;
-
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;
-import org.apache.hadoop.fs.Path;
-import org.apache.hadoop.fs.permission.FsAction;
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException;
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizerUnhandledException;
+import org.apache.hadoop.fs.azurebfs.services.AuthType;
+
+import java.io.IOException;
/**
* Interface to support authorization in Azure Blob File System.
*/
@InterfaceAudience.LimitedPrivate("authorization-subsystems")
@InterfaceStability.Unstable
public interface AbfsAuthorizer {
-
/**
* Initialize authorizer for Azure Blob File System.
*
* @throws AbfsAuthorizationException if unable to initialize the authorizer.
* @throws IOException network problems or similar.
- * @throws IllegalArgumentException if the required parameters are not
provided.
*/
- void init() throws AbfsAuthorizationException, IOException;
+ void init()
+ throws AbfsAuthorizationException, AbfsAuthorizerUnhandledException;
/**
- * Checks if the provided {@link FsAction} is allowed on the provided {@link
Path}s.
+ * Get AuthType supported by Authorizer if Authorizer would be providing
+ * authentication token to ABFS server.
*
- * @param action the {@link FsAction} being requested on the provided {@link
Path}s.
- * @param absolutePaths The absolute paths of the storage being accessed.
- * @return true if authorized, otherwise false.
- * @throws AbfsAuthorizationException on authorization failure.
- * @throws IOException network problems or similar.
- * @throws IllegalArgumentException if the required parameters are not
provided.
+ * If Authorizer is not going to provide any auth tokens, return
AuthType.None
+ * @return
*/
- boolean isAuthorized(FsAction action, Path... absolutePaths)
- throws AbfsAuthorizationException, IOException;
+ AuthType getAuthType();
-}
+ /**
+ * Checks if the provided {@link AuthorizationResource} is allowed to
+ * perform requested action.
+ *
+ * @param authorizationResource which contains the action and store path URI
+ * @return AuthorizationResult with store URI and Auth token if Authorizer
+ * is providing any.
+ * @throws AbfsAuthorizationException on authorization failure.
+ * @throws IOException network problems or
+ * similarITestAzureBlobFileSystemListStatus.
Review comment:
Sorry. Fixed the comments.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379621742 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizer.java ## @@ -18,40 +18,50 @@ package org.apache.hadoop.fs.azurebfs.extensions; -import java.io.IOException; - import org.apache.hadoop.classification.InterfaceAudience; import org.apache.hadoop.classification.InterfaceStability; -import org.apache.hadoop.fs.Path; -import org.apache.hadoop.fs.permission.FsAction; +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException; +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizerUnhandledException; +import org.apache.hadoop.fs.azurebfs.services.AuthType; + +import java.io.IOException; Review comment: Updated as suggested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379621403 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsAuthorizerConstants.java ## @@ -0,0 +1,90 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.azurebfs.constants; + +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.InvalidAbfsRestOperationException; +import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperationType; + +/** + * Defines the string literals that need to be set to authorizer as action Review comment: Updated as suggested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379621067 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/authentication/AuthorizationStatus.java ## @@ -0,0 +1,183 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.fs.azurebfs.authentication; + +import org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsAuthorizationException; +import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResource; +import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResourceResult; +import org.apache.hadoop.fs.azurebfs.extensions.AuthorizationResult; + +import java.net.URI; Review comment: Updated as suggested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379620319
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
##
@@ -36,331 +40,485 @@
import static org.apache.hadoop.fs.permission.AclEntryScope.ACCESS;
import static org.apache.hadoop.fs.permission.AclEntryType.GROUP;
import static org.apache.hadoop.test.LambdaTestUtils.intercept;
-import static org.junit.Assume.assumeTrue;
/**
* Test Perform Authorization Check operation
*/
-public class ITestAzureBlobFileSystemAuthorization extends
AbstractAbfsIntegrationTest {
-
- private static final Path TEST_READ_ONLY_FILE_PATH_0 = new
Path(TEST_READ_ONLY_FILE_0);
- private static final Path TEST_READ_ONLY_FOLDER_PATH = new
Path(TEST_READ_ONLY_FOLDER);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_0 = new
Path(TEST_WRITE_ONLY_FILE_0);
- private static final Path TEST_WRITE_ONLY_FILE_PATH_1 = new
Path(TEST_WRITE_ONLY_FILE_1);
- private static final Path TEST_READ_WRITE_FILE_PATH_0 = new
Path(TEST_READ_WRITE_FILE_0);
- private static final Path TEST_READ_WRITE_FILE_PATH_1 = new
Path(TEST_READ_WRITE_FILE_1);
- private static final Path TEST_WRITE_ONLY_FOLDER_PATH = new
Path(TEST_WRITE_ONLY_FOLDER);
- private static final Path TEST_WRITE_THEN_READ_ONLY_PATH = new
Path(TEST_WRITE_THEN_READ_ONLY);
- private static final String TEST_AUTHZ_CLASS =
"org.apache.hadoop.fs.azurebfs.extensions.MockAbfsAuthorizer";
+public class ITestAzureBlobFileSystemAuthorization
Review comment:
Refactored.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379618345 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java ## @@ -22,13 +22,21 @@ import java.io.IOException; import java.io.UnsupportedEncodingException; import java.net.MalformedURLException; +import java.net.URI; +import java.net.URISyntaxException; import java.net.URL; import java.net.URLEncoder; import java.util.ArrayList; import java.util.List; import java.util.Locale; import com.google.common.annotations.VisibleForTesting; +import org.apache.hadoop.fs.Path; +import org.apache.hadoop.fs.azurebfs.authentication.AuthorizationStatus; Review comment: Moved as suggested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379618171
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AuthorizationResourceResult.java
##
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.extensions;
+
+import java.net.URI;
+
+/**
+ * AuthorizationResourceResult array will be included in the
+ * AuthorizationResult with results for each resource.
+ */
+public class AuthorizationResourceResult {
Review comment:
Fields made final.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379618015
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AuthorizationResource.java
##
@@ -0,0 +1,29 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.extensions;
+
+import java.net.URI;
+
+/**
+ * AuthorizationResource with action and store path URI
+ */
+public class AuthorizationResource {
Review comment:
Made the fields final as suggested.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379617782
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/ConfigurationKeys.java
##
@@ -128,7 +128,8 @@ public static String accountProperty(String property,
String account) {
public static final String FS_AZURE_ENABLE_DELEGATION_TOKEN =
"fs.azure.enable.delegation.token";
public static final String FS_AZURE_DELEGATION_TOKEN_PROVIDER_TYPE =
"fs.azure.delegation.token.provider.type";
- public static final String ABFS_EXTERNAL_AUTHORIZATION_CLASS =
"abfs.external.authorization.class";
+ /** External Authorizer **/
+ public static final String FS_AZURE_ABFS_AUTHORIZER =
"fs.azure.abfs.authorizer";
Review comment:
There were no consumers for abfs.external.authorization.class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379378008
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java
##
@@ -294,18 +298,20 @@ private synchronized void writeCurrentBufferToService()
throws IOException {
waitForTaskToComplete();
}
-final Future job = completionService.submit(new Callable() {
+final Future job = completionService.submit(new
Callable() {
Review comment:
Will retain current construct to support backporting actions to older hadoop
versions still on Java 7.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#discussion_r379375362 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsAuthorizationException.java ## @@ -16,16 +16,12 @@ * limitations under the License. */ -package org.apache.hadoop.fs.azurebfs.extensions; - -import java.io.IOException; +package org.apache.hadoop.fs.azurebfs.contracts.exceptions; Review comment: All ABFS related exceptions are in contracts, hence moved it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379374836
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsAuthorizerConstants.java
##
@@ -0,0 +1,90 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs.constants;
+
+import
org.apache.hadoop.fs.azurebfs.contracts.exceptions.InvalidAbfsRestOperationException;
+import org.apache.hadoop.fs.azurebfs.services.AbfsRestOperationType;
+
+/**
+ * Defines the string literals that need to be set to authorizer as action
+ */
+public class AbfsAuthorizerConstants {
+
+ public static final String RENAME_SOURCE_ACTION = "rename-source";
+ public static final String RENAME_DESTINATION_ACTION = "rename-destination";
+
+ public static final String CONCAT_SOURCE_ACTION = "concat-source";
+ public static final String CONCAT_TARGET_ACTION = "concat-target";
+
+ public static final String CHECKACCESS_ACTION_PREFIX_ACTION = "access-";
+
+ public static final String LISTSTATUS_ACTION = "liststatus";
+ public static final String DELETE_ACTION = "delete";
+ public static final String CREATEFILE_ACTION = "create";
+ public static final String MKDIR_ACTION = "mkdir";
+ public static final String GETACL_ACTION = "getaclstatus";
+ public static final String GETFILESTATUS_ACTION = "getfilestatus";
+ public static final String SETACL_ACTION = "setacl"; // Modify, removeacl,
+ // setacl
+ public static final String SETOWNER_ACTION = "setowner";
+ public static final String SETPERMISSION_ACTION = "setpermission";
+ public static final String APPEND_ACTION = "write";
+ public static final String READ_ACTION = "read";
+ public static final String EXECUTE_ACTION = "execute";
+
+ /**
+ * Converts AbfsRestOperation to Authorizer action
+ * @param opType
+ * @return
+ */
+ public static String getAction(AbfsRestOperationType opType)
+ throws InvalidAbfsRestOperationException {
+switch (opType) {
+case ListPaths:
+ return LISTSTATUS_ACTION;
+case RenamePath:
+ return RENAME_DESTINATION_ACTION;
+case GetAcl:
+ return GETACL_ACTION;
+case GetPathStatus:
+ return GETFILESTATUS_ACTION;
+case SetAcl:
+ return SETACL_ACTION;
+case SetOwner:
+ return SETOWNER_ACTION;
+case SetPermissions:
+ return SETPERMISSION_ACTION;
+case Append:
+case Flush:
+ return APPEND_ACTION;
+case ReadFile:
+ return READ_ACTION;
+case DeletePath:
+ return DELETE_ACTION;
+case CreatePath:
+ return CREATEFILE_ACTION;
+case Mkdir:
+ return MKDIR_ACTION;
+default:
+ throw new InvalidAbfsRestOperationException(
Review comment:
Seems like the right place to check for invalid operation type. Retaining it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379356660
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -604,9 +622,10 @@ public AbfsAuthorizer getAbfsAuthorizer() throws
IOException {
| NoSuchMethodException
| SecurityException
| AbfsAuthorizationException e) {
- throw new IOException(e);
+ throw new AbfsAuthorizationException("Unable to initialize "
Review comment:
Re-throwing as AbfsAuthorizerException with the actual exception as the
innerexception. Expect to quickly narrow down the issue at authorizer level.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add Authorizer Interface
snvijaya commented on a change in pull request #1842: HADOOP-16730 : ABFS: Add
Authorizer Interface
URL: https://github.com/apache/hadoop/pull/1842#discussion_r379356145
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
##
@@ -578,21 +587,30 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
}
}
- public String getAbfsExternalAuthorizationClass() {
-return this.abfsExternalAuthorizationClass;
- }
+ public AbfsAuthorizer getAbfsAuthorizer() throws
AzureBlobFileSystemException {
+if (this.authorizer != null)
+{
+ return this.authorizer;
+}
- public AbfsAuthorizer getAbfsAuthorizer() throws IOException {
-String authClassName = getAbfsExternalAuthorizationClass();
-AbfsAuthorizer authorizer = null;
+String authClassName = this.abfsAuthorizerClass;
try {
if (authClassName != null && !authClassName.isEmpty()) {
@SuppressWarnings("unchecked")
Class authClass = (Class)
rawConfig.getClassByName(authClassName);
-authorizer = authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
+this.authorizer =
+authClass.getConstructor(new Class[]
{Configuration.class}).newInstance(rawConfig);
LOG.trace("Initializing {}", authClassName);
-authorizer.init();
+this.authorizer.init();
+if ((this.authorizer.getAuthType() != AuthType.SAS) && (
+this.authorizer.getAuthType() != AuthType.None)) {
+ throw new AbfsAuthorizationException(
Review comment:
Currently the only supported AuthType for an external Authorizer is SAS.
Throwing the exception if AuthType is set to any other such as OAuth or
SharedKey right after the authorizer init.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
