[
https://issues.apache.org/jira/browse/AIRFLOW-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747054#comment-16747054
]
ASF subversion and git services commented on AIRFLOW-2703:
----------------------------------------------------------
Commit b3918a658c358352348ecb3cb52532e605e7ecf3 in airflow's branch
refs/heads/v1-10-test from mishikaSingh
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=b3918a6 ]
[AIRFLOW-2703] Catch transient DB exceptions from scheduler's heartbeat it does
not crash (#3650)
If there is any issue in DB connection then rest of the functions take care of
those exceptions but in heartbeat of scheduler, there is no handling for this
kind of situation.
Airflow Scheduler should not crash if a "transient" DB exception occurs in the
heartbeat of scheduler.
> Scheduler crashes if Mysql Connectivity is lost
> -----------------------------------------------
>
> Key: AIRFLOW-2703
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2703
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.9.0, 2.0.0
> Reporter: raman
> Assignee: Mishika Singh
> Priority: Major
> Fix For: 2.0.0
>
>
> Airflow scheduler crashes if connectivity to Mysql is lost.
> Below is the stack Trace
> Traceback (most recent call last): File
> "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 371,
> in helper pickle_dags) File
> "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line
> 50, in wrapper result = func(*args, **kwargs) File
> "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1762,
> in process_file dag.sync_to_db() File
> "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line
> 50, in wrapper result = func(*args, **kwargs) File
> "/usr/src/venv/local/lib/python2.7/site-packages/airflow/models.py", line
> 3816, in sync_to_db session.commit() File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 943, in commit self.transaction.commit() File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 471, in commit t[1].commit() File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 1643, in commit self._do_commit() File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 1674, in _do_commit self.connection._commit_impl() File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 726, in _commit_impl self._handle_dbapi_exception(e, None, None, None,
> None) File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 1413, in _handle_dbapi_exception exc_info File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
> line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb,
> cause=cause) File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File
> "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/mysql/base.py",
> line 1784, in do_commit dbapi_connection.commit() OperationalError:
> (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server
> during query') (Background on this error at: http://sqlalche.me/e/e3q8)
> Process DagFileProcessor141318-Process:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
