[
https://issues.apache.org/jira/browse/AIRFLOW-6822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075299#comment-17075299
]
ASF subversion and git services commented on AIRFLOW-6822:
----------------------------------------------------------
Commit 68d1714f296989b7aad1a04b75dc033e76afb747 in airflow's branch
refs/heads/master from Bjorn Olsen
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=68d1714 ]
[AIRFLOW-6822] AWS hooks should cache boto3 client (#7541)
> AWS hooks dont always cache the boto3 client
> --------------------------------------------
>
> Key: AIRFLOW-6822
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6822
> Project: Apache Airflow
> Issue Type: Bug
> Components: aws
> Affects Versions: 1.10.9
> Reporter: Bjorn Olsen
> Assignee: Bjorn Olsen
> Priority: Minor
> Fix For: 2.0.0
>
>
> Implementation of the Amazon AWS hooks (eg S3 hook, Glue hook etc) varies
> with how they call the underlying aws_hook.get_client_type(X) method.
> Most of the time the client that gets returned is cached by the superclass,
> but not always. The client should always be cached for performance reasons -
> creating a client is a time consuming process.
> Example of how to do it (athena.py):
>
> {code:java}
> def get_conn(self):
> """
> check if aws conn exists already or create one and return it
> :return: boto3 session
> """
> if not self.conn:
> self.conn = self.get_client_type('athena')
> return self.conn{code}
>
>
> Example of how not to do it: (s3.py):
>
> {code:java}
> def get_conn(self):
> return self.get_client_type('s3'){code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
