[
https://issues.apache.org/jira/browse/ARROW-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315159#comment-17315159
]
Weston Pace commented on ARROW-8900:
------------------------------------
A PR would be awesome. CC [~apitrou] to verify what I'm about to say. There
is already a struct arrow::fs::S3Options in src/arrow/filesystem/s3fs.h which
would probably be the best place to put these new options. I don't think you
need to bother with any "mutator" methods like ConfigureAccessKey / FromUri
unless you plan on setting multiple options from one source of truth. Just add
the fields as new public properties to the struct. The ClientBuilder has
access to these options so it should be pretty easy to access them.
It looks like you are using python so you will also need to modify _s3fs.pyx
and add the options to the `S3FileSystem` class ([~jorisvandenbossche] any
other thoughts?)
Testing will be a trickier thing. We use minio for S3FS testing. I don't know
that we want to figure out how to get a proxy server running in CI. Maybe it
will be simple enough to create a ClientBuilder with the new S3Options and then
call config() and confirm the returned configuration has the appropriate AWS
SDK properties set.
> [C++] Respect HTTP(S)_PROXY for S3 Filesystems and/or expose proxy options as
> parameters
> ----------------------------------------------------------------------------------------
>
> Key: ARROW-8900
> URL: https://issues.apache.org/jira/browse/ARROW-8900
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Affects Versions: 0.17.0
> Reporter: Daniel Nugent
> Priority: Minor
>
> HTTP_PROXY and HTTPS_PROXY are not automatically respected by the
> Aws::Client::ClientConfiguration (see:
> https://github.com/aws/aws-sdk-cpp/issues/1049)
> Either Arrow should respect them or make them available as parameters when
> connecting to S3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
