sunchao commented on code in PR #6059:
URL: https://github.com/apache/datafusion-comet/pull/6059#discussion_r4178411296
##########
native/core/src/parquet/objectstore/azure.rs:
##########
@@ -170,9 +965,11 @@ fn translate_hadoop_configs(
}
}
- // SAS tokens are scoped to
`fs.azure.sas.<container>.<account>[.<endpoint-suffix>]`.
- if let (Some(container), Some(account)) = (container, account) {
- if let Some(sas) = sas_value(configs, container, account) {
+ // The account-level fixed token wins over the container-scoped SAS token.
A blank
+ // token is left out so it never reaches the builder; validation reports
it as an
+ // error before the store is built.
+ if let Some((_, sas)) = active_sas_token(configs, account, container) {
+ if !sas.trim().is_empty() {
Review Comment:
[P2] Normalize the fixed SAS token before forwarding it. With
`auth.type=SAS` and `fs.azure.sas.fixed.token=" ?sig=...&sv=2020-08-04&sp=r
"`, Hadoop trims the value and removes the leading `?`. This code checks the
trimmed value but passes the original string. `object_store` removes `?` before
trimming individual components, so the resulting credential contains `?sig`
instead of `sig` and cannot authenticate. A configuration also supplying the
clean legacy container token works on the base revision but breaks when this PR
gives the padded fixed token precedence. Could the fixed-token value be trimmed
as Hadoop’s `getTrimmedPasswordString` does, with a regression covering
whitespace before the optional `?`?
Evidence: Reproduced using verified exact base/head translation code and
actual object_store 0.13.2. URL:
abfss://[email protected]/path/file.parquet. Set
fs.azure.account.auth.type=SAS, fs.azure.sas.fixed.token='
?sig=synthetic&sv=2020-08-04&sp=r ', and
fs.azure.sas.data.myacct.dfs.core.windows.net='?sig=synthetic&sv=2020-08-04&sp=r'.
Both stores construct successfully. Inspecting the static
AzureCredential::SASToken yields first key 'sig' on base and '?sig' on head,
with no 'sig' entry on head. Removing only the surrounding whitespace restores
agreement. The same test reproduces 'sv' becoming '?sv'. Verified Hadoop 3.4.1
AbfsConfiguration.getSASTokenProvider calls getTrimmedPasswordString, and
AbfsClient.appendSASTokenToQuery removes the leading '?'. No network requests
were made.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
