[
https://issues.apache.org/jira/browse/AIRFLOW-6822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038270#comment-17038270
]
Ash Berlin-Taylor commented on AIRFLOW-6822:
--------------------------------------------
The cachedproperty module we also use elsewhere could be good here:
{code:python}
@cachedproperty
def conn(self):
return self.get_client_type('athena')
def get_conn(self):
# Compat shim
return self.conn
{code}
> AWS hooks dont always cache the boto3 client
> --------------------------------------------
>
> Key: AIRFLOW-6822
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6822
> Project: Apache Airflow
> Issue Type: Bug
> Components: aws
> Affects Versions: 1.10.9
> Reporter: Bjorn Olsen
> Assignee: Bjorn Olsen
> Priority: Minor
>
> Implementation of the Amazon AWS hooks (eg S3 hook, Glue hook etc) varies
> with how they call the underlying aws_hook.get_client_type(X) method.
> Most of the time the client that gets returned is cached by the superclass,
> but not always. The client should always be cached for performance reasons -
> creating a client is a time consuming process.
> Example of how to do it (athena.py):
>
> {code:java}
> def get_conn(self):
> """
> check if aws conn exists already or create one and return it
> :return: boto3 session
> """
> if not self.conn:
> self.conn = self.get_client_type('athena')
> return self.conn{code}
>
>
> Example of how not to do it: (s3.py):
>
> {code:java}
> def get_conn(self):
> return self.get_client_type('s3'){code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
