mshalak-nix opened a new issue #8760: URL: https://github.com/apache/airflow/issues/8760
**Apache Airflow version**: 1.10.9 **Kubernetes version: 1.15.11 **Environment**: - **Cloud provider or hardware configuration**: Google Cloud **What happened**: I started to notice that Airflow webserver performance degrades over time. At one moment all my webserver pods had 100% CPU used, so UI and API became too slow, even unusable. I started to investigate this issue and looks like it depends on amount of task instances stored in DB. I had 100,000+ of them in the task_instance table. When I checked what process took full CPU on a webserver pods, that was a gunicorn. After I cleaned up task_instance table, CPU usage dropped to nothing. Right now I have 30,000 tasks completed and see CPU spikes again.  Also this issue seems to go away if I disable all my DAGs. Looks like webserver has some query which executes from time to time and consumes all the CPU. **What you expected to happen**: Amount of completed tasks do not influence the webserver performance. **How to reproduce it**: Generate 100,000 tasks and have 10 enabled DAGs in Airflow. Webserver CPU usage will be high, web becomes unusable. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
