awesomescot opened a new issue, #41806: URL: https://github.com/apache/airflow/issues/41806
### Apache Airflow version 2.10.0 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? When I have over about 1000 dag files the standalone processor seems to stop functioning properly. I see CPU drop to almost zero. Parsing processes is also around 0. The dagbag never fills up. Logs are unhelpful. I can't seem to figure out what the dag processor is doing, seems as though it's silently crashing. ### What you think should happen instead? I think the standalone dag processor should process in the same or less time than the scheduler dag processor. ### How to reproduce I'm not sure I can share our dag files, but I will post my values file and would love to see if others can reproduce. ### Operating System kubernetes helm chart ### Versions of Apache Airflow Providers The ones in the helm chart. ### Deployment Official Apache Airflow Helm Chart ### Deployment details We are connecting to an RDS postgres instance(also very low cpu usage). ### Anything else? I've been trying to play around with settings to see if I can figure out what is happening, but no luck so far. I'm happy to post any logs that would be helpful. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
