[
https://issues.apache.org/jira/browse/HADOOP-17725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17639946#comment-17639946
]
ASF GitHub Bot commented on HADOOP-17725:
-----------------------------------------
CLevasseur commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1328860096
I have pulled the latest changes. It now fails when running the tests. Note
that it also fails when I run the tests from the trunk branch of the
`apache/hadoop` repository.
I haven't setup OAuth2, is that alright if we just run those tests using the
Shared Key ?
Also, it looks like those tests create a lot of containers in the storage
account, is there an easy way to clean those ?
**Tests Output (Both apache/hadoop:trunk and CLevasseur/hadoop:trunk give
the same errors)**
**NonHNS-SharedKey**:
```
Choose action:
[Note - SET_ACTIVE_TEST_CONFIG will help activate the config for IDE/single
test class runs]
1) SET_ACTIVE_TEST_CONFIG
2) RUN_TEST
3) CLEAN_UP_OLD_TEST_CONTAINERS
4) SET_OR_CHANGE_TEST_ACCOUNT
5) PRINT_LOG4J_LOG_PATHS_FROM_LAST_RUN
#? 2
Enter parallel test run process count [default - 8]:
Set the active test combination to run the action:
1) HNS-OAuth
2) HNS-SharedKey
3) nonHNS-SharedKey
4) AppendBlob-HNS-OAuth
5) AllCombinationsTestRun
6) Quit
#? 3
Combination specific property setting: [ key=fs.azure.account.auth.type ,
value=SharedKey ]
Activated [src/test/resources/abfs-combination-test-configs.xml] - for
account: dataenginfraus3prod for combination NonHNS-SharedKey
Running test for combination NonHNS-SharedKey on account dataenginfraus3prod
[ProcessCount=8]
Test run report can be seen in
dev-support/testlogs/2022-11-28_10-12-22/Test-Logs-NonHNS-SharedKey.txt
pcregrep: pcre_exec() gave error -27 while matching text that starts:
[ERROR]
ITestAzureBlobFileSystemMainOperation>FSMainOperationsBaseTest.testGlobStatusFilterWithEmptyPathResults:492
? AbfsRestOperation
[ERROR] ITestAzureBlobF
pcregrep: Error -8, -21 or -27 means that a resource limit was exceeded.
pcregrep: Check your regex for nested unlimited loops.
---
> Improve error message for token providers in ABFS
> -------------------------------------------------
>
> Key: HADOOP-17725
> URL: https://issues.apache.org/jira/browse/HADOOP-17725
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/azure, hadoop-thirdparty
> Affects Versions: 3.3.0
> Reporter: Ivan Sadikov
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.2
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> It would be good to improve error messages for token providers in ABFS.
> Currently, when a configuration key is not found or mistyped, the error is
> not very clear on what went wrong. It would be good to indicate that the key
> was required but not found in Hadoop configuration when creating a token
> provider.
> For example, when running the following code:
> {code:java}
> import org.apache.hadoop.conf._
> import org.apache.hadoop.fs._
> val conf = new Configuration()
> conf.set("fs.azure.account.auth.type", "OAuth")
> conf.set("fs.azure.account.oauth.provider.type",
> "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
> conf.set("fs.azure.account.oauth2.client.id", "my-client-id")
> //
> conf.set("fs.azure.account.oauth2.client.secret.my-account.dfs.core.windows.net",
> "my-secret")
> conf.set("fs.azure.account.oauth2.client.endpoint", "my-endpoint")
> val path = new Path("abfss://[email protected]/")
> val fs = path.getFileSystem(conf)
> fs.getFileStatus(path){code}
> The following exception is thrown:
> {code:java}
> TokenAccessProviderException: Unable to load OAuth token provider class.
> ...
> Caused by: UncheckedExecutionException: java.lang.NullPointerException:
> clientSecret
> ...
> Caused by: NullPointerException: clientSecret {code}
> which does not tell what configuration key was not loaded.
>
> IMHO, it would be good if the exception was something like this:
> {code:java}
> TokenAccessProviderException: Unable to load OAuth token provider class.
> ...
> Caused by: ConfigurationPropertyNotFoundException: Configuration property
> fs.azure.account.oauth2.client.secret not found. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
