potiuk commented on issue #42632:
URL: https://github.com/apache/airflow/issues/42632#issuecomment-2449654038
Hey Here. I would like to start a discussion and propose some redesign of
our CI - to address some of the issues mentioned here - and involve others so
that we come up together with a better design and then we get it implemented
together.
First let me state the problems and what is wrong currently. And I will
write a separte comment on what would be the first proposal to address it.
* Before #42505 and #43319 all the tests we had were in "tests" folder. Now
we have three test folders for unit tests: "tests", "providers/tests",
"task_sdk/tests"
* Those tests cannot be run together in single pytest command - because all
of them have a conftest.py file at the top folder and pytest refuses to run
them together because it does not know which `conftest.py` is the "top-level"
one.
* Currently we have "non-db-tests" that runs all the "non-db" tests together
in a single xdist/parallel fashion and it currenlty runs together "tests" and
"providers/tests" together - but this is really thanks to a workaround for it
in "providers" (there is an `__init__.py` in "providers" folder that makes the
test belong to package "providers.tests" - but this is really a hack because
"providers" is where the whole "providers" project is, it's not a python
"package".
* Both "airflow" and "providers" tests are currently run with `breeze
testing tests` command (which by default used to run ALL tests ("airflow +
providers") and only when you specified particular packages or modules to run
as additional arguments woudl run only those tests you specified. This was a
bit "hacky" approach how it was done and one of the reasons why running system
tests were broken after provider's separation (i *temporarily* fixed it in
#43529 but this is really a band-aid).
* Also the same `breeze testing tests` command is used to run "helm" unit
tests - and there is another "hack" implemented - we are also using breeze
container to run the tests but "helm" tests do not need to be run in breeze
container at all - they need a small local venv we already have from k8s where
`helm` is installed - and we can run them there
* Actually `airflow` and `task_sdk` tests technically (eventually when we
fully remove all provider references from airflow core and tests) will not have
to use breeze container as well. With UV we could technicaly run the tests in a
venv created dynamically in CI (with main constraints / `uv lock` for
reproducibility) rather than using the CI image - because we do not need alll
the 700+ deps - they should be "reproducible-enough" without the CI imaage of
ours.
* We alredy have separate "TaskSDK tests" that are running in the same
environment as "Non-DB tests" - but only for "task_sdk" and they have separate
`breeze testing task-sdk-tests` - it looks like a good direction,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
