[
https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811053#comment-17811053
]
ASF GitHub Bot commented on HADOOP-18610:
-----------------------------------------
steveloughran commented on code in PR #5953:
URL: https://github.com/apache/hadoop/pull/5953#discussion_r1467059052
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/TestWorkloadIdentityTokenProvider.java:
##########
@@ -18,10 +18,15 @@
package org.apache.hadoop.fs.azurebfs.oauth2;
+import org.apache.commons.io.FileUtils;
import org.apache.hadoop.fs.azurebfs.AbstractAbfsTestWithTimeout;
import org.junit.Test;
import org.mockito.Mockito;
+import java.io.File;
Review Comment:
nit: this block of imports should go up first
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/TestWorkloadIdentityTokenProvider.java:
##########
@@ -77,4 +83,45 @@ public void testTokenDoesNotExpireTooSoon() {
assertFalse(provider.hasEnoughTimeElapsedSinceLastRefresh());
}
+
+ /**
+ * Test that the correct token is read from the token file.
+ *
+ * @throws IOException if the token file is empty or file I/O fails.
+ */
+ @Test
+ public void testGetToken() throws IOException {
+ long startTime = System.currentTimeMillis();
+ File tokenFile = File.createTempFile("azure-identity-token", "txt");
+ FileUtils.write(tokenFile, TOKEN, StandardCharsets.UTF_8);
+ AzureADToken azureAdToken = new AzureADToken();
+ WorkloadIdentityTokenProvider tokenProvider = Mockito.spy(
+ new WorkloadIdentityTokenProvider(AUTHORITY, TENANT_ID, CLIENT_ID,
tokenFile.getPath()));
+ Mockito.doReturn(azureAdToken)
+ .when(tokenProvider).getTokenUsingJWTAssertion(TOKEN);
+ assertEquals(azureAdToken, tokenProvider.getToken());
+ assertTrue("token fetch time was not set correctly",
tokenProvider.getTokenFetchTime() > startTime);
Review Comment:
use AssertJ especially here.
make test >= so that if the start time and load happens in same millisecond
by clock granularity, no test failure
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java:
##########
@@ -884,6 +885,19 @@ public AccessTokenProvider getTokenProvider() throws
TokenAccessProviderExceptio
tokenProvider = new RefreshTokenBasedTokenProvider(authEndpoint,
clientId, refreshToken);
LOG.trace("RefreshTokenBasedTokenProvider initialized");
+ } else if (tokenProviderClass == WorkloadIdentityTokenProvider.class) {
+ String authority = getTrimmedPasswordString(
+ FS_AZURE_ACCOUNT_OAUTH_MSI_AUTHORITY,
+ AuthConfigurations.DEFAULT_FS_AZURE_ACCOUNT_OAUTH_MSI_AUTHORITY);
+ authority = appendSlashIfNeeded(authority);
+ String tenantId =
getPasswordString(FS_AZURE_ACCOUNT_OAUTH_MSI_TENANT);
Review Comment:
always good to trim this so if someone splits a value with newlines it is
trimmed properly
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/package-info.java:
##########
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
[email protected]
[email protected]
+package org.apache.hadoop.fs.azurebfs.oauth2;
Review Comment:
no need to worry about package files in test modules...is yetus complaining
about it?
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/TestWorkloadIdentityTokenProvider.java:
##########
@@ -77,4 +83,45 @@ public void testTokenDoesNotExpireTooSoon() {
assertFalse(provider.hasEnoughTimeElapsedSinceLastRefresh());
}
+
+ /**
+ * Test that the correct token is read from the token file.
+ *
+ * @throws IOException if the token file is empty or file I/O fails.
+ */
+ @Test
+ public void testGetToken() throws IOException {
+ long startTime = System.currentTimeMillis();
+ File tokenFile = File.createTempFile("azure-identity-token", "txt");
+ FileUtils.write(tokenFile, TOKEN, StandardCharsets.UTF_8);
+ AzureADToken azureAdToken = new AzureADToken();
+ WorkloadIdentityTokenProvider tokenProvider = Mockito.spy(
+ new WorkloadIdentityTokenProvider(AUTHORITY, TENANT_ID, CLIENT_ID,
tokenFile.getPath()));
+ Mockito.doReturn(azureAdToken)
+ .when(tokenProvider).getTokenUsingJWTAssertion(TOKEN);
+ assertEquals(azureAdToken, tokenProvider.getToken());
+ assertTrue("token fetch time was not set correctly",
tokenProvider.getTokenFetchTime() > startTime);
+ }
+
+ /**
+ * Test that an exception is thrown when the token file is empty.
+ *
+ * @throws IOException if file I/O fails.
+ */
+ @Test
+ public void testGetTokenThrowsWhenClientAssertionIsEmpty() throws
IOException {
+ File tokenFile = File.createTempFile("azure-identity-token", "txt");
+ AzureADToken azureAdToken = new AzureADToken();
+ WorkloadIdentityTokenProvider tokenProvider = Mockito.spy(
+ new WorkloadIdentityTokenProvider(AUTHORITY, TENANT_ID, CLIENT_ID,
tokenFile.getPath()));
+ Mockito.doReturn(azureAdToken)
+ .when(tokenProvider).getTokenUsingJWTAssertion(TOKEN);
+ boolean exception = false;
Review Comment:
Use LambdaTestUtils.intercept() here, ideally looking for the error string
as well as exception class.
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/TestWorkloadIdentityTokenProvider.java:
##########
@@ -77,4 +83,45 @@ public void testTokenDoesNotExpireTooSoon() {
assertFalse(provider.hasEnoughTimeElapsedSinceLastRefresh());
}
+
+ /**
+ * Test that the correct token is read from the token file.
+ *
+ * @throws IOException if the token file is empty or file I/O fails.
+ */
+ @Test
+ public void testGetToken() throws IOException {
+ long startTime = System.currentTimeMillis();
+ File tokenFile = File.createTempFile("azure-identity-token", "txt");
+ FileUtils.write(tokenFile, TOKEN, StandardCharsets.UTF_8);
+ AzureADToken azureAdToken = new AzureADToken();
+ WorkloadIdentityTokenProvider tokenProvider = Mockito.spy(
+ new WorkloadIdentityTokenProvider(AUTHORITY, TENANT_ID, CLIENT_ID,
tokenFile.getPath()));
+ Mockito.doReturn(azureAdToken)
+ .when(tokenProvider).getTokenUsingJWTAssertion(TOKEN);
+ assertEquals(azureAdToken, tokenProvider.getToken());
Review Comment:
Use assertJ assertions.
> ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS
> ---------------------------------------------------------------------
>
> Key: HADOOP-18610
> URL: https://issues.apache.org/jira/browse/HADOOP-18610
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools
> Affects Versions: 3.3.4
> Reporter: Haifeng Chen
> Priority: Critical
> Labels: pull-request-available
> Attachments: HADOOP-18610-preview.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In Jan 2023, Microsoft Azure AKS replaced its original pod-managed identity
> with with [Azure Active Directory (Azure AD) workload
> identities|https://learn.microsoft.com/en-us/azure/active-directory/develop/workload-identities-overview]
> (preview), which integrate with the Kubernetes native capabilities to
> federate with any external identity providers. This approach is simpler to
> use and deploy.
> Refer to
> [https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview|https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview.]
> and [https://azure.github.io/azure-workload-identity/docs/introduction.html]
> for more details.
> The basic use scenario is to access Azure cloud resources (such as cloud
> storage) from Kubernetes (such as AKS) workload using Azure managed identity
> federated with Kubernetes service account. The credential environment
> variables in pod projected by Azure AD workload identity are like following:
> AZURE_AUTHORITY_HOST: (Injected by the webhook,
> [https://login.microsoftonline.com/])
> AZURE_CLIENT_ID: (Injected by the webhook)
> AZURE_TENANT_ID: (Injected by the webhook)
> AZURE_FEDERATED_TOKEN_FILE: (Injected by the webhook,
> /var/run/secrets/azure/tokens/azure-identity-token)
> The token in the file pointed by AZURE_FEDERATED_TOKEN_FILE is a JWT (JASON
> Web Token) client assertion token which we can use to request to
> AZURE_AUTHORITY_HOST (url is AZURE_AUTHORITY_HOST + tenantId +
> "/oauth2/v2.0/token") for a AD token which can be used to directly access
> the Azure cloud resources.
> This approach is very common and similar among cloud providers such as AWS
> and GCP. Hadoop AWS integration has WebIdentityTokenCredentialProvider to
> handle the same case.
> The existing MsiTokenProvider can only handle the managed identity associated
> with Azure VM instance. We need to implement a WorkloadIdentityTokenProvider
> which handle Azure Workload Identity case. For this, we need to add one
> method (getTokenUsingJWTAssertion) in AzureADAuthenticator which will be used
> by WorkloadIdentityTokenProvider.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
