GitHub user potiuk added a comment to the discussion: Intermittent Airflow 3.1.2 API Timeouts Under High Load Causing Task Failures (KubernetesExecutor)
Two possible issues (both with your deployment): * your DNS seems to be not responding fast enough. Likely your deployment does not give it enough CPU or memory to operate - this is most likely root cause of your problem * You als might have wrong scaling setup.I think this **might be** because of different scaling patterns for Airlfow API server. With Fast API, it's better to scale by having 1 worker for API server and N replicas to handle multiple requests better under high load - see https://fastapi.tiangolo.com/deployment/docker/#one-load-balancer-multiple-worker-containers Can you please try that and increase number of replicas while decreasing number of workers to 1 as explained in Fast API docs explhttps://fastapi.tiangolo.com/deployment/docker/#one-load-balancer-multiple-worker-containers GitHub link: https://github.com/apache/airflow/discussions/58453#discussioncomment-15004737 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
