varaprasadregani opened a new issue, #69968: URL: https://github.com/apache/airflow/issues/69968
### Description If Airflow could provide native controls for limiting excessive Airflow UI/API usage per user or session. In a production Airflow 3.x environment, we observed an incident where multiple users had many Airflow UI tabs open at the same time. Several tabs were repeatedly loading DAG grid, task instance, and log views. This generated repeated API requests and expensive metadata DB queries, including calls such as: ``` GET /api/v2/dags/~/dagRuns/~/taskInstances GET /ui/grid/structure/... GET /ui/grid/ti_summaries/... ``` Some API requests remained active for a very long time, with one task-instance API request taking approximately 75 minutes. During the incident, the API server became overloaded and the UI became extremely slow. Since Airflow 3.x workers also communicate task execution state through the API server, this also affected task-state transitions and caused task start/status updates to time out. ### Use case/motivation Airflow administrators need a way to protect the API server and metadata DB from excessive UI/API usage by a single user or a small group of users. Common examples: A user opens 10-20 Airflow UI tabs. Multiple tabs refresh the same DAG grid or task-instance views. Broad task-instance queries run across many DAGs or DAG runs. Long-running UI/API requests consume API server workers and metadata DB sessions. Worker execution API traffic is affected because UI/API traffic consumes shared API server capacity. Today, this must be handled externally through ingress/proxy/database controls. Those controls are useful, but they are not Airflow-user-aware in many deployments and do not provide a clear UI message to the user. Why This Matters In Airflow 3.x, the API server is more central to both UI and task execution flows. If expensive UI/API requests overload the API server, task execution communication can also be impacted. Native user-aware limits would help operators protect production environments without relying only on external load balancers, ingress rules, or database timeouts. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
