Re: Broken DAG message won't go away in webserver

2018-08-10 Thread Dan Davydov
The scheduler should clear import errors for DAGfiles that no longer exist or DAGs that have fixed the import errors. Take a look at https://github.com/apache/incubator-airflow/blob/master/airflow/jobs.py#L761 and where it is called. If it's not working there might be a bug. On Fri, Aug 10, 2018

Re: Broken DAG message won't go away in webserver

2018-08-10 Thread Ben Laird
What is the desired behavior then? If I fix an error in a DAG, there should be a way to clear the state in the error table. Since these errors pop up in the webserver logs, and restarting the webserver with no error signals that the DAGs are healthy, should restarting the webserver clear the error

Re: Broken DAG message won't go away in webserver

2018-08-09 Thread Alex Guziel
IIRC the scheduler sets these messages in the error table in the db. On Thu, Aug 9, 2018 at 2:13 PM, Ben Laird wrote: > The messages persist even after restarting the webserver. I've verified > with other airflow users in the office that they'd have to manually delete > records from the 'import_

Re: Broken DAG message won't go away in webserver

2018-08-09 Thread Ben Laird
The messages persist even after restarting the webserver. I've verified with other airflow users in the office that they'd have to manually delete records from the 'import_error' table. When you say 'sync your DAGs', what do you mean exactly? When we fix a DAG, we'd normally kill the webserver pro

Re: Broken DAG message won't go away in webserver

2018-08-09 Thread Taylor Edmiston
Yeah, you definitely shouldn't need to do a resetdb for that. Did you try restarting the webserver? How do you sync your DAGs to the webserver? Is it possible the fixed DAG didn't get synced there? For me, IIRC, the error stops persisting once the DAG is fixed and synced. *Taylor Edmiston* Blo