That error message usually means that an error took place inside Airflow before the task ran -- maybe something with setting up the task? The task's state is NONE, meaning it never even started, but the executor is reporting that it successfully sent the command to start the task (SUCCESS)... the culprit is some failure in between.
The error message seems to say that the DAG itself couldn't be loaded from the .py file: airflow.utils.AirflowException: DAG [Pipeline_DEVTEST_CDB_DEVTEST_00_B10C8DBE1CFA89C1F274B] could not be found in /usr/local/airflow/dags/pipeline.py However a DAG with such a complicated name isn't referenced in the example code (just "Pipeline" + i). My guess is that the DAG id is being generated in a non-deterministic or time-based way, and therefore the run command can't find it once the generation criteria change. But hard to say without more detail. On Fri, Apr 29, 2016 at 3:11 PM Bolke de Bruin <[email protected]> wrote: > I would really like to know what the use case is for a depends_on_past on > the *task* level. What past are you trying to depend on? > > What I am currently assuming from just reading the example and replying on > my phone is that the depends_on_past prevents execution. Have t4 and t5 > ever run? > > Bolke > > Sent from my iPhone > > > On 29 apr. 2016, at 21:01, Chris Riccomini <[email protected]> > wrote: > > > > @Bolke/@Jeremiah, do you guys think this is related? Full thread is here: > > https://groups.google.com/forum/?pli=1#!topic/airbnb_airflow/y7wt3I24Rmw > > > >> On Fri, Apr 29, 2016 at 11:57 AM, Chris Riccomini <[email protected]> > wrote: > >> > >> Please subscribe to the dev@ mailing list. Sorry to make you jump > through > >> hoops--I know it's annoying--but it's for a good cause. ;) > >> > >> This looks like a bug. I'm wondering if it's related to > >> https://issues.apache.org/jira/browse/AIRFLOW-20. Perhaps the backfill > is > >> causing a mis-alignment between the dag runs, and depends_on_past logic > >> isn't seeing the prior execution? > >> >
