lolloberga opened a new issue #15655:
URL: https://github.com/apache/airflow/issues/15655
Hi everyone,
I install my Airflow system in an environment that needs to use an http
proxy.
How can i setup proxy parameters in Airflow system or in a single DAG?
I report my error log:
```
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=test_http_dag
AIRFLOW_CTX_TASK_ID=is_api_available
AIRFLOW_CTX_EXECUTION_DATE=2021-05-04T14:45:14.656646+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-05-04T14:45:14.656646+00:00
[2021-05-04 16:45:15,983] {http.py:102} INFO - Poking: api/
[2021-05-04 16:45:15,988] {base.py:69} INFO - Using connection to: id:
test_api_connection. Host: https://randomusers.me/, Port: None, Schema: ,
Login: , Password: None, extra: None
[2021-05-04 16:45:15,989] {http.py:140} INFO - Sending 'GET' to url:
https://randomusers.me/api/
[2021-05-04 16:47:26,572] {http.py:195} WARNING -
HTTPSConnectionPool(host='randomusers.me', port=443): Max retries exceeded with
url: /api/ (Caused by
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at
0x7fdeb3faaca0>: Failed to establish a new connection: [Errno 110] Connection
timed out')) Tenacity will retry to execute the operation
[2021-05-04 16:47:26,573] {taskinstance.py:1482} ERROR - Task failed with
exception
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in
_new_conn
conn = connection.create_connection(
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 84,
in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 74,
in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665,
in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 376,
in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 996,
in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 314, in
connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 171, in
_new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError:
<urllib3.connection.VerifiedHTTPSConnection object at 0x7fdeb3faaca0>: Failed
to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in
send
resp = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719,
in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 436, in
increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='randomusers.me',
port=443): Max retries exceeded with url: /api/ (Caused by
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at
0x7fdeb3faaca0>: Failed to establish a new connection: [Errno 110] Connection
timed out'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line
1138, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File
"/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line
1311, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File
"/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line
1341, in _execute_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.8/dist-packages/airflow/sensors/base.py",
line 228, in execute
while not self.poke(context):
File
"/usr/local/lib/python3.8/dist-packages/airflow/providers/http/sensors/http.py",
line 104, in poke
response = self.hook.run(
File
"/usr/local/lib/python3.8/dist-packages/airflow/providers/http/hooks/http.py",
line 141, in run
return self.run_and_check(session, prepped_request, extra_options)
File
"/usr/local/lib/python3.8/dist-packages/airflow/providers/http/hooks/http.py",
line 196, in run_and_check
raise ex
File
"/usr/local/lib/python3.8/dist-packages/airflow/providers/http/hooks/http.py",
line 180, in run_and_check
response = session.send(
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in
send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in
send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError:
HTTPSConnectionPool(host='randomusers.me', port=443): Max retries exceeded with
url: /api/ (Caused by
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at
0x7fdeb3faaca0>: Failed to establish a new connection: [Errno 110] Connection
timed out'))
[2021-05-04 16:47:26,578] {taskinstance.py:1525} INFO - Marking task as
FAILED. dag_id=test_http_dag, task_id=is_api_available,
execution_date=20210504T144514, start_date=20210504T144515,
end_date=20210504T144726
[2021-05-04 16:47:26,649] {local_task_job.py:146} INFO - Task exited with
return code 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
