potiuk commented on issue #19192: URL: https://github.com/apache/airflow/issues/19192#issuecomment-954129734
BTW. Yeah, checking the limits of connections opened in your DB might be a good idea. Especially if you are using variables in your DAGs at top level, it **MIGHT** lead to a significant number of connections opened, which **might* eventually cause scheduler to try to open a new connection and patiently wait until the DB server will have any connection free. It might simply be that your long running tasks are written in the way that they (accidentally) open a lot of those connections and do not close them until the task completes. I think PGBouncer **might** help with that, but if too many connections are opened by a single long running task and they are not closed, that might also not help too much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
