Bjorn Olsen created AIRFLOW-6822:
------------------------------------
Summary: AWS hooks dont always cache the boto3 client
Key: AIRFLOW-6822
URL: https://issues.apache.org/jira/browse/AIRFLOW-6822
Project: Apache Airflow
Issue Type: Bug
Components: aws
Affects Versions: 1.10.9
Reporter: Bjorn Olsen
Assignee: Bjorn Olsen
Implementation of the Amazon AWS hooks (eg S3 hook, Glue hook etc) varies with
how they call the underlying aws_hook.get_client_type(X) method.
Most of the time the client that gets returned is cached by the superclass, but
not always. The client should always be cached for performance reasons -
creating a client is a time consuming process.
Example of how to do it (athena.py):
{code:java}
def get_conn(self):
"""
check if aws conn exists already or create one and return it
:return: boto3 session
"""
if not self.conn:
self.conn = self.get_client_type('athena')
return self.conn{code}
Example of how not to do it: (s3.py):
{code:java}
def get_conn(self):
return self.get_client_type('s3'){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
