[
https://issues.apache.org/jira/browse/AIRFLOW-6083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16987709#comment-16987709
]
Bjorn Olsen edited comment on AIRFLOW-6083 at 12/4/19 10:15 AM:
----------------------------------------------------------------
This should be resolved with AIRFLOW-6072
After this PR gets merged you could use the "Extra" part of the Airflow
Connection to pass additional config to the boto3 client / resource / session
as you please.
Edit: You should also be able to change the timeout this way, if desired.
was (Author: bjorn.ols...@gmail.com):
This should be resolved with AIRFLOW-6072
After this PR gets merged you could use the "Extra" part of the Airflow
Connection to pass additional config to the boto3 client / resource / session
as you please.
> AwsLambdaHook is not accepting non-default configuration
> --------------------------------------------------------
>
> Key: AIRFLOW-6083
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6083
> Project: Apache Airflow
> Issue Type: Bug
> Components: aws
> Affects Versions: 1.10.6
> Reporter: Daumantas Pagojus
> Assignee: Daumantas Pagojus
> Priority: Blocker
>
> Hello.
> While using Airflow we have come across a problem with AwsLambdaHook.
> We are using this hook to launch Lambda function which takes a while to
> complete (around 5-6 minutes). However, since lambda is invoked through boto3
> client, which has default timeout set to [60
> seconds|https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/],
> our Airflow interface shows this lambda failing, even though Lambda finishes
> successfully checking at AWS console. This also causes another side effect:
> Since boto3 thinks that Lambda has timed out, it automatically spawns another
> instance, which also times out and this chain lasts 5 times, spawning 5
> Lambdas and all these Lambdas show as failed in Airflow interface, while they
> actually succeed.
>
> This can be solved by passing in custom configuration when creating a boto3
> client, however, it is not possible to do that when creating AwsLambdaHook as
> it does not take in this parameter.
> However, we see that AwsLambdaHook inherits and uses AwsHook's function
> (get_client_type) to get the boto3 client and this function accepts
> configuration parameter (which defaults to None), but it is never passed to
> it from the Lambda's hook, which could be easily achieved and would fix the
> bug we are facing at the moment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
