dwsmith1983 commented on PR #6059: URL: https://github.com/apache/datafusion-comet/pull/6059#issuecomment-6015905548
> Resolve the selected client endpoint using the runtime Hadoop container/account/global lookup before filtering and validating the principal. Add the regression below alongside the existing provider-specific tests. Done in 611a8b53a. Every key Hadoop 3.4.2 reads through `getPasswordString` (the client id, secret and endpoint, the MSI tenant, endpoint and authority, the token file, the refresh token, the user name and password, and `fs.azure.sas.fixed.token`) is now read as `<key>.<container>.<host>` first, then account-scoped, then global, under the same `containerConf` probe the SAS fixed token already used. The account key and the mechanism keys stay account-then-global: `SimpleKeyProvider` builds its `AbfsConfiguration` without a container, and `auth.type`, the provider classes and `keyprovider` go through `get` and `getEnum`, which have no container form (checked against the 3.4.1 and 3.4.2 bytecode). Your configuration is a test with the flag on: the store builds with the client secret, the tenant and authority host coming from the container-scoped endpoint. Beside it: the same configuration with the flag off fails as before, since Hadoop 3.4.1 does not read the form; a container-scoped secret wins over the account-level one when the flag is on; a blank container-scoped value is an error. One change beyond the finding: with the flag off, a container-scoped OAuth key that is the only credential now fails with an error naming the key, the way the SAS fixed token already did, instead of letting ambient `AZURE_*` credentials apply where Hadoop itself would fail; a named workload identity is exempt, since Hadoop 3.4.1 completes it with the default token path, and the three keys of unsupported mechanisms report that mechanism instead. The description and the data sources guide list which keys take the container form. That covers item 1 of #6605. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
