notatallshaw-gts opened a new issue, #30335: URL: https://github.com/apache/airflow/issues/30335
### What do you see as an issue? Similar to how `sql_alchemy_pool_recycle` defaults to 1800 seconds for the Airflow metastore: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#config-database-sql-alchemy-pool-recycle If users are using celery as their backend it provides extra stability to set `pool_recycle`. This problem is particularly acute for users who are using MySQL as backend for tasks because MySQL disconnects connections after 8 hours of being idle. While Airflow can usually force celery to retry connecting it does not always work and tasks can fail. This is specifically reccomended by the SqlAlchemy docs: * https://docs.sqlalchemy.org/en/14/core/pooling.html#setting-pool-recycle * https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_recycle * https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_wait_timeout ### Solving the problem We currently have a file which looks like this: ```python from airflow.config_templates.default_celery import DEFAULT_CELERY_CONFIG database_engine_options = DEFAULT_CELERY_CONFIG.get( "database_engine_options", {} ) # Use pool_pre_ping to detect stale db connections # https://github.com/apache/airflow/discussions/22113 database_engine_options["pool_pre_ping"] = True # Use pool recyle due to MySQL disconnecting sessions after 8 hours # https://docs.sqlalchemy.org/en/14/core/pooling.html#setting-pool-recycle # https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine.params.pool_recycle # https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_wait_timeout database_engine_options["pool_recycle"] = 1800 DEFAULT_CELERY_CONFIG["database_engine_options"] = database_engine_options ``` And we point the env var `AIRFLOW__CELERY__CELERY_CONFIG_OPTIONS` to this object, not sure if this is best practise? ### Anything else Maybe just change the default options to include this? ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
