dwsmith1983 commented on PR #6059: URL: https://github.com/apache/datafusion-comet/pull/6059#issuecomment-5799952691
> Could we keep the account as written, the way `main` did, and update `mixed_case_host_resolves_account_scoped_key_and_borrows_no_env` and the sentence in `datasources.md` to match? `extract_account` keeps the first host label as written. For `abfss://[email protected]` the native scan now looks up `fs.azure.account.key.MyAcct.dfs.core.windows.net`, the key Hadoop reads, and a key spelled `myacct` is not found for that host, which matches Hadoop too. The test is renamed to say that, a second one pins the lowercase key not matching, and the sentence in `datasources.md` now says the account is taken as written. > Could those two be treated as absent under `MsiTokenProvider`, so the builder falls through to IMDS with no client id the way Hadoop does? `blank_value_problem` skips `fs.azure.account.oauth2.client.id` and `fs.azure.account.oauth2.msi.tenant` when the active provider is `MsiTokenProvider`, so empty strings there are absent and the builder goes to the managed identity endpoint with no client id. A blank client id under `ClientCredsTokenProvider` is still an error naming the key. The new test sets both to empty strings, with and without `fs.azure.account.auth.type=OAuth`, and checks the store builds and that `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_STORAGE_TOKEN` and `AZURE_FEDERATED_TOKEN_FILE` are all ignored. The user guide paragraph on blank values names the exception. Checked against both Hadoop versions Spark ships: 3.3.4 reads the two keys with `getMandatoryPasswordString`, which only rejects null, and 3.4.1 reads them with `getPasswordString`, so it does not even require them to be set. Hadoop's own `abfs.md` sample for `MsiTokenProvider` sets both to empty values. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
