potiuk commented on PR #27823:
URL: https://github.com/apache/airflow/pull/27823#issuecomment-1336254238
> @ferruzzi
`tests/providers/amazon/aws/hooks/test_s3.py::TestAwsS3HookNoMock::test_check_for_bucket_raises_error_with_invalid_conn_id`
test fail locally because you have credentials in shared credentials file in
`default` profile. Actually we need to move this test into AwsBaseHook, if we
actually need this test. If I understand correctly we try to test that
boto3/botocore will raise an error if no credentials provided.
>
> `TestAwsS3Hook.test_create_bucket_no_region_regional_endpoint`: in this
test we check that if user want to use regional s3 endpoint for `us-east-1` but
do not specify any region than bucket creation will failed (Airflow prevent
this) because in this case region would set to `aws-global`
>
> It might happen by two things:
>
> 1. Some changes happen in recent version of `botocore` and this settings
ignored or do not return `aws-global` as region.
> 2. The settings `{"config_kwargs": {"s3": {"us_east_1_regional_endpoint":
"regional"}}}` somehow overwrites after changes in this PR. So other
`config_kwargs` might overwrites, e.g. retry strategy.
I think in this case the side-effect is different than DB:
```
INFO botocore.credentials:credentials.py:1251 Found credentials in
shared credentials file: ~/.aws/credentials
```
Looks like the tests you added are creating (and not deleting)
~/.aws/credentials - and the failing test expected to raise "No Credentials"
error.
If my hypothesis is right, the fix is two fold:
* make sure your new tests clean-up the ~/.aws/credentials in
tearDown/fixture
* make sure to cleanup ~/.aws/credentials in setup before the
`test_check_for_bucket_raises_error_with_invalid_conn_id` test.
On one hand the test did not have the proper "Setup" clearing the state
before the state to one that was expected from the test, but on the other hand
- it would be difficult to foresee when the test was being written that someone
else will create and leave such credential file. Hard to blame anyone in
particular, other than it's totally not feasible to run thousands of tests each
in completely isolated and pristine environment without side-effects like that.
This would be another side-effect example that you might stumble upon and it
is something that will happen occasionally unfortunately. Can't imagine how to
prevent those kind of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
