gmcrocetti commented on code in PR #46078:
URL: https://github.com/apache/arrow/pull/46078#discussion_r2056846027
##########
python/pyarrow/tests/test_fs.py:
##########
@@ -1839,6 +1846,20 @@ def test_s3_real_aws_region_selection():
's3://x-arrow-nonexistent-bucket?region=us-east-3')
assert fs.region == 'us-east-3'
+ # Nonexistent bucket but "allow_delayed_open" delays failure until I/O is
performed
+ fs = S3FileSystem(allow_delayed_open=True, allow_bucket_creation=False)
+ with pytest.raises(IOError, match="AWS Error NO_SUCH_BUCKET"):
+ with fs.open_output_stream("x-arrow-nonexistent-bucket/test.txt") as
stream:
+ stream.write(b"test")
+
+ # Nonexistent bucket. from_uri is not delaying evaluation and raising
OSError
+ # S3FileSystem.from_uri also fails
+ fs, path = FileSystem.from_uri(
+ 's3://x-arrow-nonexistent-bucket/test.txt?allow_delayed_open=true')
+ with pytest.raises(IOError):
+ with fs.open_output_stream(path) as stream:
+ stream.write(b"test")
Review Comment:
@AlenkaF here is the test you asked.
One behavior I wasn't expecting is the immediate failure of (lines
1857-1858):
```python
fs, path = FileSystem.from_uri(
's3://x-arrow-nonexistent-bucket/test.txt?allow_delayed_open=true')
```
as explained in comments this line fails because the existence of the bucket
is checked when this is executed. maybe this is not what someone would expect
when passing `allow_delayed_open` as a query parameter ?
I thought it would be better to check double check and in case it doesn't
make sense revert the chances in `cpp/src/arrow/filesystem/s3fs.cc`. cc @pitrou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
