[
https://issues.apache.org/jira/browse/AIRFLOW-6083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16998319#comment-16998319
]
ASF subversion and git services commented on AIRFLOW-6083:
----------------------------------------------------------
Commit c4c635df6906f56e01724573923e19763bb0da62 in airflow's branch
refs/heads/master from Daumis102
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=c4c635d ]
[AIRFLOW-6083] Adding ability to pass custom configuration to lambda client.
(#6678)
There was no way to pass custom configuration from AwsLambdaHook to lambda
client. This was an issue when for example default lambda timeout was too short.
> AwsLambdaHook is not accepting non-default configuration
> --------------------------------------------------------
>
> Key: AIRFLOW-6083
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6083
> Project: Apache Airflow
> Issue Type: Bug
> Components: aws
> Affects Versions: 1.10.6
> Reporter: Daumantas Pagojus
> Assignee: Daumantas Pagojus
> Priority: Blocker
>
> Hello.
> While using Airflow we have come across a problem with AwsLambdaHook.
> We are using this hook to launch Lambda function which takes a while to
> complete (around 5-6 minutes). However, since lambda is invoked through boto3
> client, which has default timeout set to [60
> seconds|https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/],
> our Airflow interface shows this lambda failing, even though Lambda finishes
> successfully checking at AWS console. This also causes another side effect:
> Since boto3 thinks that Lambda has timed out, it automatically spawns another
> instance, which also times out and this chain lasts 5 times, spawning 5
> Lambdas and all these Lambdas show as failed in Airflow interface, while they
> actually succeed.
>
> This can be solved by passing in custom configuration when creating a boto3
> client, however, it is not possible to do that when creating AwsLambdaHook as
> it does not take in this parameter.
> However, we see that AwsLambdaHook inherits and uses AwsHook's function
> (get_client_type) to get the boto3 client and this function accepts
> configuration parameter (which defaults to None), but it is never passed to
> it from the Lambda's hook, which could be easily achieved and would fix the
> bug we are facing at the moment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)