Re: [HUGE DISCUSSION] Airflow3 and tactical (Airflow 2) vs strategic (Airflow 3) approach

2024-05-10 Thread Pierre Jeambrun
Thank you for the work and proposal made around airflow 3. I also strongly believe that those important changes need to happen for airflow to stay relevant and it would be the right moment to start discussing and planning for airflow 3. As mentioned some features are hard to develop within

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Jarek Potiuk
> Just for clarification, this is only related to the provider's tests, right? Absolutely. On Fri, May 10, 2024 at 11:21 AM Andrey Anshin wrote: > > "enable" tests for 2.8 and 2.7 separately > > Just for clarification, this is only related to the provider's tests, > right? > > > > On Fri, 10

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Andrey Anshin
> "enable" tests for 2.8 and 2.7 separately Just for clarification, this is only related to the provider's tests, right? On Fri, 10 May 2024 at 13:15, Jarek Potiuk wrote: > > Regarding Airflow 2.7 and Airflow 2.8 in the time we are ready to move > forward to the initial version of Airflow 3

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Jarek Potiuk
> Regarding Airflow 2.7 and Airflow 2.8 in the time we are ready to move forward to the initial version of Airflow 3 providers might already drop support of these versions in providers. Airflow 2.7 in the mid of August 2024 Airflow 2.8 in the mid of December 2024 Yep. But also "here and now"

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Andrey Anshin
BTW, forget to mention that we should also check Pytest: Good Integration Practices from https://docs.pytest.org/en/stable/explanation/goodpractices.html On Fri, 10 May 2024 at 13:07, Andrey Anshin wrote: > I think the current solution with run tests against installed packages > might help

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Andrey Anshin
I think the current solution with run tests against installed packages might help with future modifications and develop new dev experience. And what is more important is help to find problems and incompatibilities of providers with the previous version of Airflow "here and now". Regarding Airflow

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Jarek Potiuk
And yes - as we get down to 2.8 and 2.7 it might be possible that we will already implement some of the simplifications you mentioned as it might be easier than adding back-compatiblity to the current ways. I assume it will be `quite` a bit harder to make our test suite work with Airflow 2.8 and

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Jarek Potiuk
Yep. I think these are all good ideas, and I think this should be part of our big Airflow 2 vs. Airflow 3 discussion. Almost as important as what is in and what is out is where and how development of different components happen. Same repo? Different repos? Different branches? Single monorepo for

Re: [PROPOSAL] Improved compatiblity checks for Providers (running unit tests for multiple airflow versions)

2024-05-10 Thread Andrey Anshin
Great job, Jarek! I would have some proposals, which should be considered as a long term We should rework our test structure to fully run provider tests without touching the Core tests. The main problem here is that we configure a lot of things into the root conftest.py which might be a problem