yuqian90 commented on issue #7038: [AIRFLOW-4495] Allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-581713200 > @yuqian90 I see your point - since our use case is a one-off, this extra communication of a failed pipeline could be okay. However, for others I wonder if this might not be a one-off case; it might be something a team does once a week, or once a month, or every time a particular ticket comes in, etc. The extra step of handling the failure at re-triggering time could be burdensome and yet another thing that goes into an oncall handbook or something that causes an oncall page. > > Just to clarify as well - the suggestion isn't to make `allow_trigger_in_future` be _enabled_ for all DAGs, the suggestion is to make `allow_trigger_in_future` _respected_ by all DAGs. > > An alternative to changing the default behavior of the scheduler is adding another configuration option, something like `handle_duplicate_runs_gracefully`, that would allow for handling the re-triggering scenario for those that opt-in. My naive feeling is that this isn't a very intuitive option and would be difficult to explain, but open to alternatives as well. > > Thoughts? You mentioned some DAGs that are usually scheduled daily at a specific time, but need to be triggered earlier than the scheduled time once every week regularly. For complicated triggering logic like this, did you consider making another DAG that triggers this DAG with `TriggerDagRunOperator` ? Then the DAG itself can be set to `schedule_interval=None` and relies on another DAG to do the triggering. It'll also respect `allow_trigger_in_future` once `schedule_interval=None` is set. Having Airflow silently handling trigger dag with duplicated key sounds a bit dangerous to me. But maybe others have different opinions.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
