Paul Balanca created ARROW-10675:
------------------------------------
Summary: [C++][Python] Support AWS S3 Web identity credentials
Key: ARROW-10675
URL: https://issues.apache.org/jira/browse/ARROW-10675
Project: Apache Arrow
Issue Type: Improvement
Affects Versions: 2.0.0, 1.0.1
Reporter: Paul Balanca
It seems to me that Arrow only supports at the moment the "AssumeRole" AWS STS
API, but not the other options offered:
[https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html#stsapi_comparison]
[https://sdk.amazonaws.com/cpp/api/LATEST/class_aws_1_1_auth_1_1_s_t_s_assume_role_web_identity_credentials_provider.html]
I am clearly no security/infra expert, but it seems that the configuration
"AssumeRoleWithWebIdentity" is used commonly in Kubernetes setups, and I
believe it would be beneficial for Arrow C++ & Python library to support.
At the moment, a work around is to call directly `aws sts` to generate a
temporary session, but it is a fairly paintful as the session expires: all
PyArrow objects with an S3 filesystem (datasets, ...) needs to be re-built with
new credentials.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
