Hey Alexander, Airflow does not work like that for "regular" use - but you can use DebugExecutor which does what you want https://airflow.apache.org/docs/apache-airflow/stable/executor/debug.html?highlight=debugexecutor
But note it's just for debugging. Airflow is not supposed to be used to trigger dags immediately - it's a scheduler and it's optimized for that. We do have discussions about enabling such cases in the future but this not were Airflow use case is today. It's used for scheduling. J On Thu, Apr 14, 2022 at 8:02 PM Daniel Standish <[email protected]> wrote: > > I see. Ok well yeah, the scheduler won't register the dag immediately. > There are settings which control how frequently it checks for new files. > > You might try shortening the time for this. I don't know the setting off top > of head but look at the config. > > You could consider using the API to check whether the dag was picked up > before triggering: > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_dag > > You could also consider not writing out the dag file dynamically in this way > and instead use "dag factory" functions, or task groups -- these are > sometimes used for similar goals. And in 2.3.0 (beta coming soon) there will > be support for task mapping which can also achieve similar things.
