litalkat commented on issue #95:
URL:
https://github.com/apache/airflow-client-python/issues/95#issuecomment-1732325895
@ferruzzi sure.
the request is for specific airflow deployment. in general- the airflow
deployment is running ~100K airflow tasks per day.
the process already run at least 5000 times In the last 3 months.
I already countered this "504 Service Exception" response when I made an API
call for long-range date but I saw that it can handle a 7000-9000 records in
the response.
in the last two days I get this 504 exception almost for any call (even for
a range of 3 minutes -> ~100 tasks response)
its important to mention that I am monitoring a lot of other airflow
instances in the same time but I didn't say any thing about rate limit
the exception is returning EXACTLY 1 min after the API call is sent
the traceback looks like:
(504)
Reason: Gateway Timeout
HTTP response headers: HTTPHeaderDict({'x-powered-by': 'Express', .....
HTTP response body: Error accured while trying to proxy: <some private env
name>/api/v1/dags/~/dagRuns/~/taskInstances/list
the request is using the
airflow_client.get_task_instances_batch(ListTaskInstancesForm(...),_check_return_type=False)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
