pentschev commented on code in PR #38375:
URL: https://github.com/apache/arrow/pull/38375#discussion_r1369938151
##########
python/pyarrow/_s3fs.pyx:
##########
@@ -260,6 +271,26 @@ cdef class S3FileSystem(FileSystem):
load_frequency=900, proxy_options=None,
allow_bucket_creation=False, allow_bucket_deletion=False,
retry_strategy: S3RetryStrategy =
AwsStandardS3RetryStrategy(max_attempts=3)):
+ ensure_s3_initialized()
+
+ self._initialize_s3(access_key=access_key, secret_key=secret_key,
session_token=session_token,
+ anonymous=anonymous, region=region,
request_timeout=request_timeout,
+ connect_timeout=connect_timeout, scheme=scheme,
endpoint_override=endpoint_override,
+ background_writes=background_writes,
default_metadata=default_metadata,
+ role_arn=role_arn, session_name=session_name,
external_id=external_id,
+ load_frequency=load_frequency,
proxy_options=proxy_options,
+ allow_bucket_creation=allow_bucket_creation,
allow_bucket_deletion=allow_bucket_deletion,
+ retry_strategy=retry_strategy)
+
+ def _initialize_s3(self, *, access_key=None, secret_key=None,
session_token=None,
+ bint anonymous=False, region=None, request_timeout=None,
+ connect_timeout=None, scheme=None,
endpoint_override=None,
+ bint background_writes=True, default_metadata=None,
+ role_arn=None, session_name=None, external_id=None,
+ load_frequency=900, proxy_options=None,
+ allow_bucket_creation=False,
allow_bucket_deletion=False,
+ retry_strategy: S3RetryStrategy =
AwsStandardS3RetryStrategy(max_attempts=3)):
+
Review Comment:
Indeed they are, thanks for calling my attention to that. Now I see what
happened, I initially [tested this in the `12.0.1` tag where CS3Options was not
a
`std::optional`](https://github.com/apache/arrow/blob/6af660f48472b8b45a5e01b7136b9b040b185eb1/python/pyarrow/_s3fs.pyx#L256)
and that was causing the `CS3Options` default constructor to be called before
`ensure_s3_initialized()`. I can confirm it works as expected in `main` without
that, and fixed it now via
https://github.com/apache/arrow/pull/38375/commits/58097e91a305ea05aa7efc24db097a5289b514e4
.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
