We do something like that here here in this way, and I'd love feedback from anyone else if they do it a different way. Basically we use TriggerDagRunOperators and a task in the "Producer" that cares what day of the week it is being run, and a "Consumer" DAG with no schedule interval that depends on being triggers when the right conditions of the "Producer" DAG are met.
In more detail, we have a daily DAG like your "Producer" that ingests data every day. We want to run a series of tasks including generating a report in a "Consumer" DAG after after Monday's data has been processed; which would be on a Tuesday, after a run of "Producer" off the execution date of Monday. The last task in "Producer" is a TriggerDagRunOperator that only triggers IF it notices that now() is Tuesday. This triggers an instance at the right time of a child DAG of the "Consumer" type. On Mon, Jun 13, 2016 at 11:50 AM, Rafael Barbosa <[email protected]> wrote: > Hi everyone, > > I tried looking around, but I cannot find an archive of this mailing list, > so I apologize if this question has been asked before. > > I am looking for a why to set the dependency between two DAGs: a "Producer" > runs every day, appending data in a DB and a "Consumer" that runs weekly, > generating a summary report of the new data. > > I would like to run the "Consumer" every Monday (for instance), but only > after the Monday instance of the daily "Producer" is complete. What would > be the best way to accomplish this? > > Thanks, > Rafael >
