gmcrocetti commented on code in PR #46078:
URL: https://github.com/apache/arrow/pull/46078#discussion_r2056846027
##########
python/pyarrow/tests/test_fs.py:
##########
@@ -1839,6 +1846,20 @@ def test_s3_real_aws_region_selection():
's3://x-arrow-nonexistent-bucket?region=us-east-3')
assert fs.region == 'us-east-3'
+ # Nonexistent bucket but "allow_delayed_open" delays failure until I/O is
performed
+ fs = S3FileSystem(allow_delayed_open=True, allow_bucket_creation=False)
+ with pytest.raises(IOError, match="AWS Error NO_SUCH_BUCKET"):
+ with fs.open_output_stream("x-arrow-nonexistent-bucket/test.txt") as
stream:
+ stream.write(b"test")
+
+ # Nonexistent bucket. from_uri is not delaying evaluation and raising
OSError
+ # S3FileSystem.from_uri also fails
+ fs, path = FileSystem.from_uri(
+ 's3://x-arrow-nonexistent-bucket/test.txt?allow_delayed_open=true')
+ with pytest.raises(IOError):
+ with fs.open_output_stream(path) as stream:
+ stream.write(b"test")
Review Comment:
@AlenkaF here is the test you asked.
One behavior I wasn't expecting is the immediate failure of (lines
1857-1858):
```python
fs, path = FileSystem.from_uri(
's3://x-arrow-nonexistent-bucket/test.txt?allow_delayed_open=true')
```
as explained in comments this line fails because bucket's existence is
checked when this is executed, ignoring `allow_delayed_open`. Maybe this is
not the behavior one would expect when using this query parameter. WDYT ?
I thought it would be better to double check and in case it doesn't make
sense revert changes in `cpp/src/arrow/filesystem/s3fs.cc`. cc @pitrou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
