raulcd opened a new issue, #48565:
URL: https://github.com/apache/arrow/issues/48565
### Describe the bug, including details regarding any error messages,
version, and platform.
It seems that the following bucket `s3://mf-nwp-models/README.txt` used on a
couple of tests is not accessible anymore:
```
=================================== FAILURES
===================================
______________________ test_s3_real_aws_region_selection
_______________________
@pytest.mark.s3
def test_s3_real_aws_region_selection():
# Taken from a registry of open S3-hosted datasets
# at https://github.com/awslabs/open-data-registry
fs, path = FileSystem.from_uri('s3://mf-nwp-models/README.txt')
assert fs.region == 'eu-west-1'
> with fs.open_input_stream(path) as f:
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_fs.py:1935:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
pyarrow/_fs.pyx:857: in pyarrow._fs.FileSystem.open_input_stream
???
pyarrow/error.pxi:155: in pyarrow.lib.pyarrow_internal_check_status
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
> ???
E OSError: When reading information for key 'README.txt' in bucket
'mf-nwp-models': AWS Error ACCESS_DENIED during HeadObject operation: No
response body.
pyarrow/error.pxi:92: OSError
_______________________________ test_s3_finalize
_______________________________
@pytest.mark.s3
def test_s3_finalize():
# Once finalize_s3() was called, most/all operations on S3
filesystems
# should raise.
code = """if 1:
import pytest
from pyarrow.fs import (FileSystem, S3FileSystem,
ensure_s3_initialized, finalize_s3)
fs, path = FileSystem.from_uri('s3://mf-nwp-models/README.txt')
assert fs.region == 'eu-west-1'
f = fs.open_input_stream(path)
f.read(50)
finalize_s3()
with pytest.raises(ValueError, match="S3 .* finalized"):
f.read(50)
with pytest.raises(ValueError, match="S3 .* finalized"):
fs.open_input_stream(path)
with pytest.raises(ValueError, match="S3 .* finalized"):
S3FileSystem(anonymous=True)
with pytest.raises(ValueError, match="S3 .* finalized"):
FileSystem.from_uri('s3://mf-nwp-models/README.txt')
"""
> subprocess.check_call([sys.executable, "-c", code])
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_fs.py:2104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
popenargs = (['/opt/conda/envs/arrow/bin/python3.10', '-c', 'if 1:\n
import pytest\n from pyarrow.fs import (FileSys...ValueError, match="S3
.* finalized"):\n
FileSystem.from_uri(\'s3://mf-nwp-models/README.txt\')\n '],)
kwargs = {}, retcode = 1
cmd = ['/opt/conda/envs/arrow/bin/python3.10', '-c', 'if 1:\n import
pytest\n from pyarrow.fs import (FileSyst...s(ValueError, match="S3 .*
finalized"):\n
FileSystem.from_uri(\'s3://mf-nwp-models/README.txt\')\n ']
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command
'['/opt/conda/envs/arrow/bin/python3.10', '-c', 'if 1:\n import pytest\n
from pyarrow.fs import (FileSystem, S3FileSystem,\n
ensure_s3_initialized, finalize_s3)\n\n fs, path =
FileSystem.from_uri(\'s3://mf-nwp-models/README.txt\')\n assert
fs.region == \'eu-west-1\'\n f = fs.open_input_stream(path)\n
f.read(50)\n\n finalize_s3()\n\n with pytest.raises(ValueError,
match="S3 .* finalized"):\n f.read(50)\n with
pytest.raises(ValueError, match="S3 .* finalized"):\n
fs.open_input_stream(path)\n with pytest.raises(ValueError, match="S3 .*
finalized"):\n S3FileSystem(anonymous=True)\n with
pytest.raises(ValueError, match="S3 .* finalized"):\n
FileSystem.from_uri(\'s3://mf-nwp-models/README.txt\')\n ']' returned
non-zero exit status 1.
opt/conda/envs/arrow/lib/python3.10/subprocess.py:369: CalledProcessError
----------------------------- Captured stderr call
-----------------------------
Traceback (most recent call last):
File "<string>", line 8, in <module>
File "pyarrow/_fs.pyx", line 857, in
pyarrow._fs.FileSystem.open_input_stream
File "pyarrow/error.pxi", line 155, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
OSError: When reading information for key 'README.txt' in bucket
'mf-nwp-models': AWS Error ACCESS_DENIED during HeadObject operation: No
response body.
```
This has failed on several nightly jobs, I haven't double checked all of
them but at least the first ones are due to the same issue:
-
[test-conda-python-3.10](https://github.com/ursacomputing/crossbow/actions/runs/20287540566/job/58264656014)
-
[test-conda-python-3.10-pandas-1.3.4-numpy-1.21.2](https://github.com/ursacomputing/crossbow/actions/runs/20287538590/job/58264649982)
-
[test-conda-python-3.11](https://github.com/ursacomputing/crossbow/actions/runs/20287539559/job/58264652979)
-
[test-conda-python-3.11-pandas-latest-numpy-latest](https://github.com/ursacomputing/crossbow/actions/runs/20287537486/job/58264646041)
-
[test-conda-python-3.12](https://github.com/ursacomputing/crossbow/actions/runs/20287537454/job/58264646004)
-
[test-conda-python-3.12-pandas-latest-numpy-1.26](https://github.com/ursacomputing/crossbow/actions/runs/20287538710/job/58264650418)
-
[test-conda-python-3.12-pandas-latest-numpy-latest](https://github.com/ursacomputing/crossbow/actions/runs/20287540482/job/58264655747)
-
[test-conda-python-3.13](https://github.com/ursacomputing/crossbow/actions/runs/20287538674/job/58264650270)
-
[test-conda-python-3.13-pandas-nightly-numpy-nightly](https://github.com/ursacomputing/crossbow/actions/runs/20287538941/job/58264651184)
-
[test-conda-python-3.13-pandas-upstream_devel-numpy-nightly](https://github.com/ursacomputing/crossbow/actions/runs/20287539903/job/58264654127)
-
[test-conda-python-3.14](https://github.com/ursacomputing/crossbow/actions/runs/20287538473/job/58264649492)
### Component(s)
Python, Continuous Integration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
