By default the TriggerDagRunOperator creates a DagRun with execution_date of `utcnow()`, it doesn't inherit the execution_date of the triggering Dag.
Unless you are passing a non default value to TriggerDagRunOperator then you will get the behavior you are seeing. So in your case the following happened: - At '2020-03-05 05:00:00' the scheduler created a DagRun for the triggering Dag with an execution_date = '2020-03-04 05:00:00' - The TriggerDagRunOperator tasks ran creating a DagRun for the triggered Dag with execution_date = nowutc() (which could be very close to '2020-03-05 05:00:00' if the triggering Dag is fast enough. Chris On Fri, Mar 6, 2020, 10:11 PM Reed Villanueva <[email protected]> wrote: > Interesting. > You say that "A Triggered DAG has the execution date of when it is actual > run or it’s given time to execute." > In my case, the triggering dag has cron schedule of... > 0 5 1,3-31 * * > <http://airflowetl.ucera.local:8080/admin/dagrun/?flt2_dag_id_equals=hph_clarity_etl_trigger> > > and the triggered dag has cron schedule... > None > <http://airflowetl.ucera.local:8080/admin/dagrun/?flt2_dag_id_equals=hph_clarity_etl> > So the triggered dag has no given time to automatically execute. > Could you explain a bit more about how these schedules and the > TriggerDagRunOperator are interacting to create a logged time of +1 day > from the triggering dag's last run? > > On Fri, Mar 6, 2020 at 5:06 AM Shaw, Damian P. < > [email protected]> wrote: > >> Firstly, you can use other times other than UTC, you set the start date >> of the dag to a different time zone: >> https://airflow.apache.org/docs/stable/timezone.html#time-zone-aware-dags . >> Note that the UI and ExecutionDate will still be in UTC, if you need to get >> the ExecutionDate in your local timezone this is currently best achieved >> through Macros: >> https://stackoverflow.com/questions/44855949/make-custom-airflow-macros-expand-other-macros/54922776#54922776 >> . I run dozens of DAGs in about more than 5 different timezones this way. >> >> >> >> Secondly the execution date of Triggered DAGs has a different logic to >> Scheduled DAGs (I’m crossing my fingers this inconsistency is resolved in >> Airflow 2.0 by getting rid of the weird Scheduled DAG logic): >> >> * A Triggered DAG has the execution date of when it is actual run or it’s >> given time to execute. For example say you set it to run at 2020-03-06 >> 12:00:00 then that is it’s execution date. >> >> * A Scheduled DAG has an execution date that subscribes to the logic that >> the DAG is processing a “period” of data. So for example if you schedle it >> to run at midday every day then the run at 2020-03-06 12:00:00 considers >> that period of scheduling to be from 2020-03-05 12:00:00 (inclusive) to >> 2020-03-06 12:00:00 (not inclusive), it then gives you an execution date of >> 2020-03-05 12:00:00 >> >> >> >> As you can see even though their actual execution time is the same, their >> execution date is 1 day apart. >> >> >> >> A Triggered DAG can not provide the same logic as a Scheduled DAG because >> it has no schedule to determine the “period”. I have created Macros that >> gives me the same execution date regardless of whether I am using Scheduled >> or Triggered DAGs, you can use the above SO answer as a template if you >> want to do the same, and the thing to remember is that for a Scheduled DAG >> the next execution date is equivelent to the Triggered DAG execution date. >> >> >> >> Hope this helps >> >> Damian >> >> >> >> >> >> >> >> *From:* Reed Villanueva <[email protected]> >> *Sent:* Thursday, March 5, 2020 21:35 >> *To:* [email protected] >> *Subject:* Dag triggered via TriggerDagRunOperator logs start date as 1 >> day ahead >> >> >> >> Seeing a weird problem where when running a dag on cron schedule >> >> *0 5 1,3-31 * ** >> <http://airflowetl.ucera.local:8080/admin/dagrun/?flt2_dag_id_equals=hph_clarity_etl_trigger> >> >> >> (note I am still using UTC time (since I do not believe that I can change >> it in v1.10.7)) that ends by running a TriggerDagRunOperator task. >> >> When the "triggering" dag runs, the Last Run date shows the correct date >> and is reflected in the run_id. Eg. >> >> 2020-03-04 05:00 >> <http://airflowetl.ucera.local:8080/admin/airflow/graph?dag_id=hph_clarity_etl_trigger&execution_date=2020-03-04+05%3A00%3A00%2B00%3A00> >> >> >> However, the dag that gets triggered shows the Last Run date as 1 day >> ahead and this unexpected behavior is also reflected in the triggered dag's >> run_id >> >> 2020-03-05 05:00 >> <http://airflowetl.ucera.local:8080/admin/airflow/graph?dag_id=hph_clarity_etl&execution_date=2020-03-05+05%3A00%3A47.975014%2B00%3A00> >> >> >> yet the triggering dag only takes about 1min. to run. >> >> >> >> Does anyone know what could be happening here? >> >> Is there a time difference that I am not recognizing here (note that I am >> not living in a UTC time zone (I am in HST and the cron is intended to run >> the triggering dag at 7:00PM each night excluding the 1st of each month))? >> >> >> This electronic message is intended only for the named >> recipient, and may contain information that is confidential or >> privileged. If you are not the intended recipient, you are >> hereby notified that any disclosure, copying, distribution or >> use of the contents of this message is strictly prohibited. If >> you have received this message in error or are not the named >> recipient, please notify us immediately by contacting the >> sender at the electronic mail address noted above, and delete >> and destroy all copies of this message. Thank you. >> >> >> >> >> ============================================================================== >> Please access the attached hyperlink for an important electronic >> communications disclaimer: >> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html >> >> ============================================================================== >> > > This electronic message is intended only for the named > recipient, and may contain information that is confidential or > privileged. If you are not the intended recipient, you are > hereby notified that any disclosure, copying, distribution or > use of the contents of this message is strictly prohibited. If > you have received this message in error or are not the named > recipient, please notify us immediately by contacting the > sender at the electronic mail address noted above, and delete > and destroy all copies of this message. Thank you. >
