Hello here,
Next phase of the cleanup - it's been sped up by the comment from @kxepal
- https://github.com/apache/airflow/pull/46801#issuecomment-2661415731 - I
have planned to do it a bit later this week, but maybe indeed it's a good
idea to start a discussion now so that people are not confused.
Currently we are using "from tests_common" to reuse code in various
providers and airflow, and this is fine, with the exception that
"tests_common" is currently just a package in "airflow" main project. But
it does not have to be (or rather it should not be).
We should have it in a separate sub-project - similarly as providers/* and
task_sdk are now separate projects - and we should add dependency to the
common test code distribution from within all the projects that use it (via
workspace feature).
My proposal is to name it "common_test_code" (but I am open to any
suggestions): It will look like this
.
|- airflow
|- common_test_code
| pyproject.toml
| src
| tests_common
The code in airflow and providers will not change, they will continue to
use "from tests_common import ...".
* uv will work as it used to work, no changes will be needed - uv sync will
automatically install the common_test_code as editable project
* additionally - (and that's a big plus) after this change you should be
able to run `uv sync` in a provider folder and have provider-specific
separate venv, and basically you should be able to run tests for a provider
using that .venv and generally treat provider project as a "standalone" one
- this is what I hinted at in the previous mail.
* people who do not use uv but pip for example will have to manually add
the `common_test_code` as an editable install in their venv. For example:
pip install -e . -e ./task_sdk -e. ./providers/google -e ./common_test_code
This is the next step in standardizing the layout of the Airflow project
using workspaces.
J.
