[
https://issues.apache.org/jira/browse/AIRFLOW-6083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
George K Ong reassigned AIRFLOW-6083:
-------------------------------------
Assignee: George K Ong (was: Daumantas Pagojus)
> AwsLambdaHook is not accepting non-default configuration
> --------------------------------------------------------
>
> Key: AIRFLOW-6083
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6083
> Project: Apache Airflow
> Issue Type: Bug
> Components: aws
> Affects Versions: 1.10.6
> Reporter: Daumantas Pagojus
> Assignee: George K Ong
> Priority: Blocker
>
> Hello.
> While using Airflow we have come across a problem with AwsLambdaHook.
> We are using this hook to launch Lambda function which takes a while to
> complete (around 5-6 minutes). However, since lambda is invoked through boto3
> client, which has default timeout set to [60
> seconds|https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/],
> our Airflow interface shows this lambda failing, even though Lambda finishes
> successfully checking at AWS console. This also causes another side effect:
> Since boto3 thinks that Lambda has timed out, it automatically spawns another
> instance, which also times out and this chain lasts 5 times, spawning 5
> Lambdas and all these Lambdas show as failed in Airflow interface, while they
> actually succeed.
>
> This can be solved by passing in custom configuration when creating a boto3
> client, however, it is not possible to do that when creating AwsLambdaHook as
> it does not take in this parameter.
> However, we see that AwsLambdaHook inherits and uses AwsHook's function
> (get_client_type) to get the boto3 client and this function accepts
> configuration parameter (which defaults to None), but it is never passed to
> it from the Lambda's hook, which could be easily achieved and would fix the
> bug we are facing at the moment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
