For the longest time, we have been discussing supporting full cron
functionality in Airflow. Currently, Airflow supports cron syntax with one
pretty visible issue.

For example, if you wanted cron to schedule a job every day at midnight,
you would specify 0 0 * * *. In Airflow, if you were to pause your dag for
a week and then unpause it, your dag would run the past 7 days' worth of
dag runs. However, a common use-case for cron schedules is to use it for
some routine job scheduling (e.g. full db backup, log file cleanup,
minutely alert condition checking) that only needs to run at midnight, but
doesn't need to "backfill" the days missed.

We had originally envisioned building this into the scheduler, but that is
unnecessarily complex. Instead, we have had the ShortCircuitOperator around
for a while. I plan to extend this to make a
ShortCircuitIfNotCurrentOperator. This will skip downstream for dag runs
that are not current.

https://issues.apache.org/jira/browse/AIRFLOW-198

FYI.. alternate names for this operator are welcome.
-s

Reply via email to