[
https://issues.apache.org/jira/browse/AIRFLOW-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991471#comment-16991471
]
ASF subversion and git services commented on AIRFLOW-6072:
----------------------------------------------------------
Commit 4fb498f87ef89acc30f2576ebc5090ab0653159e in airflow's branch
refs/heads/master from Bjorn Olsen
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=4fb498f ]
[AIRFLOW-6072] aws_hook: Outbound http proxy setting and other enhancements
(#6686)
> aws_hook: Ability to set outbound proxy
> ---------------------------------------
>
> Key: AIRFLOW-6072
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6072
> Project: Apache Airflow
> Issue Type: Improvement
> Components: aws
> Affects Versions: 1.10.6
> Reporter: Bjorn Olsen
> Assignee: Bjorn Olsen
> Priority: Minor
> Fix For: 2.0.0
>
>
> The boto3 connection used by aws_hook does not respect outbound http_proxy
> settings (even if these are set in system wide).
>
> The way to configure a proxy is to pass a botocore.config.Config object to
> boto3 when creating a client (according to this SO post).
> [https://stackoverflow.com/questions/33480108/how-do-you-use-an-http-https-proxy-with-boto3]
> While the aws_hook get_client_type() method is used extensively by AWS
> Operators, the "config" argument is not used by any operator.
> Adding a check to aws_hook for "config" in the "extra_config" of the Airflow
> Connection, could allow us to pass kwargs there that build the Config object
> automatically by the hook.
> Otherwise we have to update every AWS Operator to also take a "config"
> parameter.
>
> To set an outbound proxy is then as simple as adding this to your
> extra_config:
> {code:java}
> { .. ,
> "config":{ "proxies": {
> "http": "http://myproxy:8080";,
> "https": "http://myproxy:8080"; }},
> .. }
> {code}
>
> This needs to work both for the main boto3 clients that do task work, but
> also during the assume_role process which also uses a boto3 client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
