Hi All,

Relative newbie here.

Is it possible (or is there a known pattern) for allowing atomic
updates of DAGs?

By atomic I mean, if a DAG is in progress running some version of the
code, then it should continue to use it even if the DAG code is
changed. The change should be used up only for new DAG runs started
after the update.

Eg I have a dag setup as:
A ---> B ---> C
Each step produces an intermediate build artifact which is used as
input by the next stage. If the DAG step is currently executing "B"
and I update the DAG code, I dont want the changes to be picked up
until "C" completes.

Does airflow controller provide any such feature or is this something
that each developer handles on their own?

Thanks for your time.

-mandeep

Reply via email to