Daniel Huang created AIRFLOW-1559:
-------------------------------------
Summary: MySQL warnings about aborted connections, missing engine
disposal
Key: AIRFLOW-1559
URL: https://issues.apache.org/jira/browse/AIRFLOW-1559
Project: Apache Airflow
Issue Type: Bug
Components: db
Reporter: Daniel Huang
Priority: Minor
We're seeing a flood of warnings about aborted connections in our MySQL logs.
{code}
Aborted connection 56720 to db: 'airflow' user: 'foo' host: 'x.x.x.x' (Got an
error reading communication packets)
{code}
It appears this is because we're not performing [engine
disposal|http://docs.sqlalchemy.org/en/latest/core/connections.html#engine-disposal].
The most common source of this warning is from the scheduler, when it kicks
off new processes to process the DAG files. Calling dispose in
https://github.com/apache/incubator-airflow/blob/master/airflow/jobs.py#L403
greatly reduced these messages. However, the worker is still causing some of
these, I assume from when we spin up processes to run tasks. We do call dispose
in
https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L1394-L1396,
but I think it's a bit early. Not sure if there's a place we can put this
cleanup to ensure it's done everywhere.
Quick script to reproduce this warning message:
{code}
from airflow import settings
from airflow.models import Connection
session = settings.Session()
session.query(Connection).count()
session.close()
# not calling settings.engine.dispose()
{code}
Reproduced with Airflow 1.8.1, MySQL 5.7, and SQLAlchemy 1.1.13.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
