benmatwil commented on issue #49972:
URL: https://github.com/apache/arrow/issues/49972#issuecomment-6125423371
I have now had Copilot have a look through and it seems to have identified
the issue.
Copilot mentions that there is a difference between how C++ and Python Azure
SDK generates SAS keys. The C++ SDK adds a `?` at the start of a generated SAS
key while Python does not. The pyarrow `AzureFileSystem` implementation assumes
that `?` will be passed with the sas_token string in the same way as the
generated token from the C++ Azure SDK. The Python SDK does not prepend the `?`
character. This means the `AzureFileSystem` implementation interprets the
sas_token without the `?` from the python SDK internally as part of the storage
account path instead of a query parameter and reorders it.
I have now tested by passing `sas_token=f"?{sas_token}"` to the
`AzureFileSystem` constructor and it now works (except for issues seen in
#46085).
Not sure how you want to align, i.e. should the Python filesystem interface
align with the Python Azure SDK for sas tokens. Or maybe docs need updating to
mention the `?` is required in Python.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
