Jeremiah Lowin created AIRFLOW-174:
--------------------------------------
Summary: Add --debug option to scheduler
Key: AIRFLOW-174
URL: https://issues.apache.org/jira/browse/AIRFLOW-174
Project: Apache Airflow
Issue Type: Improvement
Components: scheduler
Affects Versions: Airflow 1.7.1
Reporter: Jeremiah Lowin
Assignee: Bolke de Bruin
Priority: Minor
{{airflow webserver}} has a {{--debug}} param which enables the use of
interactive debuggers like {{ipdb}} (among other side effects). Unfortunately
the {{airflow scheduler}} process does not respect debugger instructions, which
makes tracing errors very difficult. It just prints the following error and
resumes operation:
{code}
Traceback (most recent call last):
File "/Users/jlowin/git/airflow/airflow/jobs.py", line 690, in _do_dags
self.process_dag(dag, tis_out)
File "/Users/jlowin/git/airflow/airflow/jobs.py", line 521, in process_dag
run.update_state()
File "/Users/jlowin/git/airflow/airflow/utils/db.py", line 53, in wrapper
result = func(*args, **kwargs)
File "/Users/jlowin/git/airflow/airflow/models.py", line 3471, in update_state
all_deadlocked = (has_unfinished_tasks and no_dependencies_met)
File "/Users/jlowin/git/airflow/airflow/models.py", line 3471, in update_state
all_deadlocked = (has_unfinished_tasks and no_dependencies_met)
File "/Users/jlowin/anaconda3/lib/python3.5/bdb.py", line 48, in
trace_dispatch
return self.dispatch_line(frame)
File "/Users/jlowin/anaconda3/lib/python3.5/bdb.py", line 67, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
{code}
[~bolke] I'm assigning this to you for now because I suspect it's related to
the subprocess/daemonizing changes you made though I'm not sure. If we can
enable {{ipdb}} it will make future scheduler work so much easier!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
