Also I have a feeling that with AST / smart conversion, we should be able to get an almost 100% accurate automated conversion of that one. It is "almost" (of-course there will be edge cases), but it looks like a pretty automate'able conversion.
Maybe - I am not sure how possible it is due to contracts etc. - but maybe also the managed service teams could run such conversion on the vast number of DAGs people have out there and "perfect" the conversion mechanism. I think - again - what's really crucial here is that it does not have to be a big-bang. If we will have prolonged period of time, where compatible approach will happily coexist with new approach - for Airflow 2.11 and Airflow 3 and when you will still be able to run part of your DAGS with "old ways" and gradually converting your DAGs to "new ways", ability to track the progress and "nagging warnings" - and you have a converter that works "out-of-the-box" for "almost" everything - this is the maximum support we can give to our users. J. On Mon, Aug 5, 2024 at 8:52 AM Tzu-ping Chung <t...@astronomer.io.invalid> wrote: > I don’t know how to compare the efforts since honestly I have very little > idea how many templated fields people tend to have. I can outline what each > group needs to do though. > > Airflow maintainers > - Develop the compatibility layer > - One time work before 3.0 > - Maintenance afterwards, at least until we drop 2.x support from all > providers > - Apply compatibility layer to all providers > - Drop compatibility layer from providers when they drop 2.x support > - Not a hard requirement > > Third-party operator authors > - Apply compatibility layer to all operators > - Drop compatibility layer from operators when dropping 2.x support > - Not a hard requirement > > DAG authors (that want to upgrade to Airflow 3) > - Same as third-party operator authors if the real authors don’t want to > upgrade > - Convert DAGs to use the new syntax > > > > On 30 Jul 2024, at 04:55, Scheffler Jens (XC-AS/EAE-ADA-T) < > jens.scheff...@de.bosch.com.INVALID> wrote: > > > > Thanks TP for the rework. > > > > I added some comments (iteration 2) on the migration ideas. I think > these details make it clearer, still i have partial doubts how much burden > we add to users to migrate DAGs to get to version 3. I very much favor the > new templating but am not sure how many DAG authors we leave with migration > problems behind. > > Do we have a guess or estimation how much burden we as airflow > developers need to keep as compatability compared to the amount of DAG > templates that people neet to adjust? > > > > Sent from Outlook for iOS<https://aka.ms/o0ukef> > > ________________________________ > > From: Tzu-ping Chung <t...@astronomer.io.INVALID> > > Sent: Monday, July 29, 2024 8:54:58 PM > > To: dev@airflow.apache.org <dev@airflow.apache.org> > > Cc: michalmod...@google.com <michalmod...@google.com> > > Subject: Re: [VOTE] AIP-80: Explicit Template Fields in Operator > Arguments > > > > I have updated the AIP to include the additional compatibility > discussions in this thread. Please take a look again. > > > > Specifically (although by no means exclusively) it would be awesome if > Michał you could have a look and see if it addresses more of the concerns > and could be viable for you. Although the vote is non-binding, I still > would like to be more confident I tried to address the real concerns from > the community, which is the real problem when it comes to migration. > > > > Link to document for convenience > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fx%2F2grOEg&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C9318251bd2fe4490b64308dcafffffe5%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638578761240102546%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=vVICot9p06VQJ56QmP9dwvEaWfpR7nCcMVqUexl6B3w%3D&reserved=0 > <https://cwiki.apache.org/confluence/x/2grOEg> > > > > TP > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
