Just as a comment - I think a number of people expressed that they want to comment on AIP-43 / AIP-44. Mateusz mentioned that also maybe some people will have more time during the slower period around Xmas - so feel free to make your comments still - we aim - if possible - to have the discussions conclude ~ mid Jan and that should give enough time for different contributors. Do not wait till the last moment though and feel free to comment on both AIPs as soon as you can. I will be around also to answer those.
On Wed, Dec 15, 2021 at 9:47 PM Mateusz Henc <[email protected]> wrote: > > Hi, > FYI I start my vacation on Monday and I am off for 3 weeks (back on 01/10) > with limited access to my laptop. > > If you have any comments please add by the end of this week, so I could > address them while I am still there. > If you are fine then maybe we could start voting soon? > > // Leaving more time for AIP-44 as it is longer and more complicated. > > Best regards, > Mateusz Henc > > > On Tue, Dec 14, 2021 at 1:09 PM Jarek Potiuk <[email protected]> wrote: >> >> Comment on that one too - we also provided more details (including >> possible deployment options). Similarly as in case of AIP-44 we had a >> few goals - which also resulted from earlier discussions: >> >> * Keep current behavior practically untouched >> * Allow for different deployments "no separation", "dag processor >> separation", "multiple dag processors for tenant isolation" >> * the "multiple dag processor" case does NOT change the ways dags are >> stored or parsed or where dags are stored. They are all parts of >> "DAG_FOLDER". The change is merely there to allow each dag processor >> to be "responsible" for parsing only selected part of the "dag" tree. >> This is purely to isolate code execution between tenants and has no >> impact whatsoever on other components (and it has no impact on other >> ideas there - such as DAG fetcher or DAG submission). It's purely to >> allow the "deployment" to better isolate code execution of user code >> coming from different tenants. >> >> I hope this one will be faster to discuss and raise less >> controversies/doubts - but feel absolutely free to comment/criticize >> etc. >> >> J. >> >> On Thu, Dec 9, 2021 at 4:36 PM Mateusz Henc <[email protected]> wrote: >> > >> > Hi, >> > We just updated the AIP-43 adding some details and deployment options. >> > Please take a look. Any comments are appreciated. >> > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-43+Dag+processor+separation >> > >> > BTW Update for AIP-44 is coming soon. >> > >> > Best regards, >> > Mateusz Henc >> > >> > >> > On Thu, Dec 2, 2021 at 3:04 PM Jarek Potiuk <[email protected]> wrote: >> >> >> >> Excited :) >> >> >> >> On Thu, Dec 2, 2021 at 2:20 PM Mateusz Henc <[email protected]> >> >> wrote: >> >> > >> >> > Hi, >> >> > I just submitted my proposal for separating the DagProcessor from >> >> > Scheduler and running it as an independent component. This is part of >> >> > AIP-1 Airflow multi-tenancy effort. >> >> > >> >> > PTAL >> >> > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-43+Dag+processor+separation >> >> > >> >> > Best regards, >> >> > Mateusz Henc
