> No. There is no such feature yet. Assumption is that DAGs change "slowly" and > without heavy structural changes that would do any harm in those situations. > > If you need "heavy" change to a DAG, you need to create a new DAG with new > dag_id - this is current recommendation when it comes to "dag versioning"
Thanks for the suggestion. I'll look into deploying DAG with a new version instead of updating the existing one. > > There is an Airflow Improvement Proposal > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-36+DAG+Versioning > about DAG versioning where such atomic changes will likely become a reality - > but it has not been worked on for quite some time (but this is is likely to > change in the coming months and work on DAG Versioning will resume - or so I > hear). I'll keep an eye on its progress. Thanks for the link! -mandeep > > J. > > > On Thu, May 12, 2022 at 11:57 PM Mandeep Sandhu <[email protected]> > wrote: >> >> Hi All, >> >> Relative newbie here. >> >> Is it possible (or is there a known pattern) for allowing atomic >> updates of DAGs? >> >> By atomic I mean, if a DAG is in progress running some version of the >> code, then it should continue to use it even if the DAG code is >> changed. The change should be used up only for new DAG runs started >> after the update. >> >> Eg I have a dag setup as: >> A ---> B ---> C >> Each step produces an intermediate build artifact which is used as >> input by the next stage. If the DAG step is currently executing "B" >> and I update the DAG code, I dont want the changes to be picked up >> until "C" completes. >> >> Does airflow controller provide any such feature or is this something >> that each developer handles on their own? >> >> Thanks for your time. >> >> -mandeep
