Raúl Cumplido created ARROW-18392:
-------------------------------------
Summary: [CI][Python] Some nightly python tests fail due to ACCESS
DENIED to S3 bucket
Key: ARROW-18392
URL: https://issues.apache.org/jira/browse/ARROW-18392
Project: Apache Arrow
Issue Type: Bug
Components: Continuous Integration, Python
Reporter: Raúl Cumplido
Fix For: 11.0.0
Several nightly tests fail with:
{code:java}
=================================== FAILURES
===================================
____________________________ test_s3fs_wrong_region
____________________________ @pytest.mark.s3
def test_s3fs_wrong_region():
from pyarrow.fs import S3FileSystem
# wrong region for bucket
fs = S3FileSystem(region='eu-north-1')
msg = ("When getting information for bucket
'voltrondata-labs-datasets': "
r"AWS Error UNKNOWN \(HTTP status 301\) during HeadBucket "
"operation: No response body. Looks like the configured region
is "
"'eu-north-1' while the bucket is located in 'us-east-2'."
"|NETWORK_CONNECTION")
with pytest.raises(OSError, match=msg) as exc:
fs.get_file_info("voltrondata-labs-datasets")
# Sometimes fails on unrelated network error, so next call would also
fail.
if 'NETWORK_CONNECTION' in str(exc.value):
return
fs = S3FileSystem(region='us-east-2')
>
> fs.get_file_info("voltrondata-labs-datasets")opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_fs.py:1339:
>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pyarrow/_fs.pyx:571: in pyarrow._fs.FileSystem.get_file_info
???
pyarrow/error.pxi:144: in pyarrow.lib.pyarrow_internal_check_status
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E OSError: When getting information for bucket 'voltrondata-labs-datasets':
AWS Error ACCESS_DENIED during HeadBucket operation: No response body. {code}
I can't seem to be able to reproduce locally but is pretty consistent:
*
[test-conda-python-3.10|https://github.com/ursacomputing/crossbow/actions/runs/3528202639/jobs/5918051269]
*
[test-conda-python-3.11|https://github.com/ursacomputing/crossbow/actions/runs/3528201175/jobs/5918048135]
*
[test-conda-python-3.7|https://github.com/ursacomputing/crossbow/actions/runs/3528195566/jobs/5918035812]
*
[test-conda-python-3.7-pandas-latest|https://github.com/ursacomputing/crossbow/actions/runs/3528211334/jobs/5918069152]
*
[test-conda-python-3.8|https://github.com/ursacomputing/crossbow/actions/runs/3528193702/jobs/5918032370]
*
[test-conda-python-3.8-pandas-latest|https://github.com/ursacomputing/crossbow/actions/runs/3528213536/jobs/5918073481]
*
[test-conda-python-3.8-pandas-nightly|https://github.com/ursacomputing/crossbow/actions/runs/3528205157/jobs/5918056277]
*
[test-conda-python-3.9|https://github.com/ursacomputing/crossbow/actions/runs/3528202402/jobs/5918050613]
*
[test-conda-python-3.9-pandas-upstream_devel|https://github.com/ursacomputing/crossbow/actions/runs/3528210560/jobs/5918067302]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
