cmlad commented on issue #8212:
URL: https://github.com/apache/airflow/issues/8212#issuecomment-656254411
The issue seems to be infinite recursion due to an interaction from gevent's
monkey patching and the botocore library used by S3TaskHandler:
```
Traceback (most recent call last):
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/airflow/utils/log/s3_task_handler.py",
line 131, in s3_log_exists
found_log = self.hook.get_key(remote_log_location) is not None
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/airflow/hooks/S3_hook.py",
line 224, in get_key
obj = self.get_resource_type('s3').Object(bucket_name, key)
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/airflow/contrib/hooks/aws_hook.py",
line 186, in get_resource_type
config=config, verify=self.verify)
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/boto3/session.py",
line 389, in resource
aws_session_token=aws_session_token, config=config)
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/boto3/session.py",
line 263, in client
aws_session_token=aws_session_token, config=config)
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/session.py",
line 823, in create_client
credentials = self.get_credentials()
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/session.py",
line 428, in get_credentials
'credential_provider').load_credentials()
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/session.py",
line 919, in get_component
self._components[name] = factory()
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/session.py",
line 149, in _create_credential_resolver
self, region_name=self._last_client_region_used
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/credentials.py",
line 70, in create_credential_resolver
container_provider = ContainerProvider()
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/credentials.py",
line 1803, in __init__
fetcher = ContainerMetadataFetcher()
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/utils.py",
line 1578, in __init__
timeout=self.TIMEOUT_SECONDS
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/httpsession.py",
line 180, in __init__
self._manager = PoolManager(**self._get_pool_manager_kwargs())
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/httpsession.py",
line 188, in _get_pool_manager_kwargs
'ssl_context': self._get_ssl_context(),
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/httpsession.py",
line 197, in _get_ssl_context
return create_urllib3_context()
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/site-packages/botocore/httpsession.py",
line 72, in create_urllib3_context
context.options |= options
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/ssl.py", line
507, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/ssl.py", line
507, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File
"/usr/local/Caskroom/miniconda/base/envs/airflow/lib/python3.7/ssl.py", line
507, in options
super(SSLContext, SSLContext).options.__set__(self, value)
[Previous line repeated 409 more times]
RecursionError: maximum recursion depth exceeded
```
I still need to dive deeper into this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
