Juraj Šanko created AIRFLOW-7070:
------------------------------------
Summary: SimpleHttpOperator does not consider extra attributes
from Airflow connection
Key: AIRFLOW-7070
URL: https://issues.apache.org/jira/browse/AIRFLOW-7070
Project: Apache Airflow
Issue Type: Bug
Components: operators
Affects Versions: 1.10.9
Reporter: Juraj Šanko
I have noticed this problem with SimpleHttpOperator in Airflow 1.10.9.
When we set it up with conn_id, it does not take into account 'extra'
parameters from connection.
Example:
Airflow -> Admin -> Connection -> Extra:
{"proxies":"https":"http://xxxxxxxx:[email protected]:80"}}
My python skill is very bad, but I think there are 2 things happening:
*1.*
http_hook.py - function *get_conn* - line 78
Context:
{code:java}
if conn.extra:
try:
session.headers.update(conn.extra_dejson)
{code}
This is updating session header, however header can't accept type dictionary,
only string. So first error with this is that the header is incorrectly
formatted (type error).
To me, it makes no sense to use extra field as header source, but if this have
to be kept for compatibility reason, I would suggest excluding python-request
parameters from pushing to headers. Otherwise It may results also in insecure
behavior.
{code:java}
session.proxies.update(extra.get("proxies"))
// This works for me if I disable header code above. (don't use it in
solution){code}
*2.*
**http_hook.py - function *run_and_check* - ** lines 176-184
Extra options here are only taken into consideration from extra_option object,
but they they should be taken from session object as well and overwritten by
connection extra_option object afterwards.
{code:java}
proxies=session.proxies
// replaced this code, as brute force solution.(don't use it in solution){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
