[
https://issues.apache.org/jira/browse/AIRFLOW-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aizhamal Nurmamat kyzy updated AIRFLOW-1559:
--------------------------------------------
Component/s: (was: db)
database
> MySQL warnings about aborted connections, missing engine disposal
> -----------------------------------------------------------------
>
> Key: AIRFLOW-1559
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1559
> Project: Apache Airflow
> Issue Type: Bug
> Components: database
> Reporter: Daniel Huang
> Assignee: Daniel Huang
> Priority: Minor
> Fix For: 1.10.0
>
>
> We're seeing a flood of warnings about aborted connections in our MySQL logs.
> {code}
> Aborted connection 56720 to db: 'airflow' user: 'foo' host: 'x.x.x.x' (Got an
> error reading communication packets)
> {code}
> It appears this is because we're not performing [engine
> disposal|http://docs.sqlalchemy.org/en/latest/core/connections.html#engine-disposal].
> The most common source of this warning is from the scheduler, when it kicks
> off new processes to process the DAG files. Calling dispose in
> https://github.com/apache/incubator-airflow/blob/master/airflow/jobs.py#L403
> greatly reduced these messages. However, the worker is still causing some of
> these, I assume from when we spin up processes to run tasks. We do call
> dispose in
> https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L1394-L1396,
> but I think it's a bit early. Not sure if there's a place we can put this
> cleanup to ensure it's done everywhere.
> Quick script to reproduce this warning message:
> {code}
> from airflow import settings
> from airflow.models import Connection
> session = settings.Session()
> session.query(Connection).count()
> session.close()
> # not calling settings.engine.dispose()
> {code}
> Reproduced with Airflow 1.8.1, MySQL 5.7, and SQLAlchemy 1.1.13.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
