Hi Ali, I was able to replicate your scenario above.
The executor maintains a cache of running task instances. When a task instance finishes, it is removed from the `running` cache of the executor. When you hard-delete the DAG Run and the task instances via the UI, the corresponding task instances are still in the 'running' cache. So when the scheduler is iterating through the dags, it is told by the executor that this particular task instance is already running, and does not need to be rescheduled. If you bounce the scheduler, the task should be picked up again as the cache will be regenerated. I believe this is a bug in the scheduler. So feel free to report it as a JIRA issue on this. Although in general, I try to avoid hard-deleting items from the DB when they are in `running` state. Hope this helps! Joy On Mon, Sep 11, 2017 at 2:38 PM, Ali Uz <[email protected]> wrote: > Hello, > > I have a DAG that has an ExternalTaskSensor that waits for another DAG to > complete before running. Here's my scenario: I run this DAG and while the > external task sensor is running, I pause it, then delete all its tasks and > the DAG Run from the Admin panel. Then I turn it on again, and the DAG > displays as "running" in the UI, but none of its tasks get queued. > > Also in the Jobs UI there seems to be a Job stuck in "running" which seems > to be related to the DAG I just deleted. > > How can I ensure that my DAG will always run no matter what? > > Thanks, > > - Ali >
