affo opened a new pull request, #2947:
URL: https://github.com/apache/fluss/pull/2947
<!--
*Thank you very much for contributing to Fluss - we are happy that you want
to help us improve Fluss. To help the community review your contribution in the
best possible way, please go through the checklist below, which will get the
contribution into a shape in which it can be best reviewed.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [GitHub
issue](https://github.com/apache/fluss/issues). Exceptions are made for typos
in JavaDoc or documentation files, which need no issue.
- Name the pull request in the format "[component] Title of the pull
request", where *[component]* should be replaced by the name of the component
being changed. Typically, this corresponds to the component label assigned to
the issue (e.g., [kv], [log], [client], [flink]). Skip *[component]* if you are
unsure about which is the best component.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean
verify` passes.
- Each pull request should address only one issue, not mix up code from
multiple issues.
**(The sections below can be removed for hotfixes or typos)**
-->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close [#xxx](https://github.com/apache/fluss/issues/2946)
<!-- What is the purpose of the change -->
Token delegation does not seem to work when the client does not configure
any entry of the Azure FS configuration:
1. FlussConnection filters client config:
FileSystem.initialize(
Configuration.fromMap(extractPrefix(conf.toMap(), "client.fs.")),
null);
1. extractPrefix filters to client.fs.* keys but does not strip the
prefix — so keys remain as client.fs.* in the resulting config.
2. AzureFileSystemPlugin.getHadoopConfiguration looks for keys with
prefix "fs.azure." — but the client config has no such keys (only client.fs.*).
The Hadoop config receives no Azure credentials.
3. setCredentialProvider (AzureFileSystemPlugin.java:64-84): since
fs.azure.account.key is absent from the Hadoop config, it falls back to
delegation tokens and calls
AzureDelegationTokenReceiver.updateHadoopConfig(hadoopConfig).
4. updateHadoopConfig (AzureDelegationTokenReceiver.java:41-71):
- Sets fs.azure.account.oauth.provider.type =
DynamicTemporaryAzureCredentialsProvider
- Sets additionInfos entries (just
fs.azure.account.oauth2.client.endpoint)
- DOES NOT set fs.azure.account.auth.type
5. ABFS driver (AzureBlobFileSystem.initialize →
AzureBlobFileSystemStore.<init> → initializeClient): since
fs.azure.account.auth.type is not set, defaults to SharedKey auth → calls
SimpleKeyProvider.getStorageAccountKey → key not found → "Failure to
initialize configuration".
### Brief change log
<!-- Please describe the changes made in this pull request and explain how
they address the issue -->
Simply `hadoopConfig.set("fs.azure.account.auth.type", "Custom");` in case
no account key is set (presumably, client side).
### Tests
<!-- List UT and IT cases to verify this change -->
- Fix existing UT test that was skipping an exception thrown on FS init to
actually check the exception is not due to misconfig
- add check for `"fs.azure.account.auth.type", "Custom"` to existing UTs
- add IT test to simulate the token delegation path (the previous IT test
only checked the "server path" by setting the complete config)
### API and Format
<!-- Does this change affect API or storage format -->
NA
### Documentation
<!-- Does this change introduce a new feature -->
NA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
