> Just wondernig... would an optional dependency not be the right place to describe that apache-airflow-providers-google[tests] would have an dependency to the common_tests subproject? > Would mean you would need to install via > pip install -e . -e ./task_sdk[tests] -e. ./providers/google[tests]
Something like that. This is more of a details of workspace but we are going to use "dev" dependency group for that - rather than extra. Details to be worked out how pip interaction will look like (pip does not have support for dependency groups yet - they are coming in 25.1 - they are already merged in main) With `uv` dependency groups can be used today and "dev" dependency group is installed automatically when you run uv sync or `uv pip install -e.` -> so we will follow this along See details about "development dependencies" in uv here https://docs.astral.sh/uv/concepts/projects/dependencies/#development-dependencies Dependency groups are already approved via PEP-735 https://peps.python.org/pep-0735/ and as mentioned - we are a release away from having them released in `pip`, so for now we will have to emulate it with extras for pip case I think but we will be able to remove it when pip 25.1 is released and matures a bit. J. On Sun, Feb 16, 2025 at 2:23 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > I would love to see some airflow_testing package which will be useful for > testing airflow-related projects and involve independently. > > > Certainly, it's not a good thing to have tests import something from > tests. > New packages as projects are cheap and provide more flexibility and are > useful from outside of the project. > > I already explained in the PR, but let me repeat my opinion here: > > IMHO It's extremely unlikely we are going to release and publish the > common test code / fixtures in any way. They will continue to be in > development-only-distribution and they will be treated as "internal detail". > > If we decide to release and publish them, we will have to maintain > backwards compatibility and account for our users (like you) using them for > their own purpose. That would block us or make it very difficult to make > breaking changes in them. > > So while you will be free to continue copying the whole distribution and > use it in your tests as you want (our licence allows that) - I seriously > doubt we will ever release and publish it in "reusable form" with > "compatibility guarantees". It's far more efficient if people like you just > copy them and are aware that they can change any time. > > J. > > On Sun, Feb 16, 2025 at 2:21 PM Alexander Shorin <kxe...@apache.org> > wrote: > >> I would love to see some airflow_testing package which will be useful for >> testing airflow-related projects and involve independently. >> >> Certainly, it's not a good thing to have tests import something from >> tests. >> New packages as projects are cheap and provide more flexibility and are >> useful from outside of the project. >> >> Also this project could have a future for testing, compatibility, quality >> and rest of measuring. >> >> -- >> ,,,^..^,,, >> >> >> On Sun, Feb 16, 2025 at 4:12 PM Jarek Potiuk <ja...@potiuk.com> wrote: >> >> > Hello here, >> > >> > Next phase of the cleanup - it's been sped up by the comment from >> @kxepal >> > - https://github.com/apache/airflow/pull/46801#issuecomment-2661415731 >> - >> > I >> > have planned to do it a bit later this week, but maybe indeed it's a >> good >> > idea to start a discussion now so that people are not confused. >> > >> > Currently we are using "from tests_common" to reuse code in various >> > providers and airflow, and this is fine, with the exception that >> > "tests_common" is currently just a package in "airflow" main project. >> But >> > it does not have to be (or rather it should not be). >> > >> > We should have it in a separate sub-project - similarly as providers/* >> and >> > task_sdk are now separate projects - and we should add dependency to the >> > common test code distribution from within all the projects that use it >> (via >> > workspace feature). >> > >> > My proposal is to name it "common_test_code" (but I am open to any >> > suggestions): It will look like this >> > >> > . >> > |- airflow >> > |- common_test_code >> > | pyproject.toml >> > | src >> > | tests_common >> > >> > The code in airflow and providers will not change, they will continue to >> > use "from tests_common import ...". >> > >> > * uv will work as it used to work, no changes will be needed - uv sync >> will >> > automatically install the common_test_code as editable project >> > >> > * additionally - (and that's a big plus) after this change you should be >> > able to run `uv sync` in a provider folder and have provider-specific >> > separate venv, and basically you should be able to run tests for a >> provider >> > using that .venv and generally treat provider project as a "standalone" >> one >> > - this is what I hinted at in the previous mail. >> > >> > * people who do not use uv but pip for example will have to manually add >> > the `common_test_code` as an editable install in their venv. For >> example: >> > >> > pip install -e . -e ./task_sdk -e. ./providers/google -e >> ./common_test_code >> > >> > This is the next step in standardizing the layout of the Airflow project >> > using workspaces. >> > >> > J. >> > >> >
