tirkarthi commented on issue #56879: URL: https://github.com/apache/airflow/issues/56879#issuecomment-3422383062
Thanks @wjddn279 for the detailed report. We have been facing similar issue in our environment with around 4 dag processors and 11k dags with each dag processor parsing around 3k dags using MySQL 8.0 . I was initially under the assumption that this was due to high connections and CPU load in MySQL dropping connections but this issue could be the case. Some relevant discussions for mysql that I found after reading this issue. https://github.com/sqlalchemy/sqlalchemy/issues/7815 https://docs.sqlalchemy.org/en/20/core/pooling.html https://docs.sqlalchemy.org/en/20/core/pooling.html#using-connection-pools-with-multiprocessing-or-os-fork -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
