> I'd put them rather in core as an integration for this... but as said, there might be more options and much more opinions in the room. But capability is more important so "praise" to all who have contributed to this! (Hope Edge will be added there soon as well...)
We can also put them to core, yes, but my reasoning why `chart` is that they **actually** use the chart and they **mainly** test if Airflow installed with the chart "works". So they are (at least the main ones that work with all executor+k8s matrix) are pretty muc testing the chart e2e. But there are indeed some tests that are testing k8s pod operators, and maybe a good compromise will be to split those and put the "chart" tests in chart, but the "k8s" Pod operator ones in "cncf.kubernetes" provider. Those tests are anyhow quite a bit different and we won't touch them for a while - so we can decide later. Also I think the k8s POD operators might be anyhow eventually separated and done differently - Kalyan currently works on making the `breeze --executor Kubernetes` to work and once it does, we might want to separate the K8SPod Operators anyway to use this rather than setting up airflow with the chart - as it's not needed there at all. I would just defer the decision on k8s_tests for now. J. On Mon, Nov 17, 2025 at 11:03 PM Jens Scheffler <[email protected]> wrote: > > Very cool! > > Not sure whether they are really placed well in "chart" because we do > not want to use K8s tests to check the chart but the full integration of > Airflow with Kubernetes. I tincludes core, K8s provider, Task SDK... so > for K8s tests we need at least one exception to the rule. > > I'd put them rather in core as an integration for this... but as said, > there might be more options and much more opinions in the room. But > capability is more important so "praise" to all who have contributed to > this! (Hope Edge will be added there soon as well...) > > Jens > > On 11/17/25 19:41, Jarek Potiuk wrote: > >> I have one question about the k8s tests in the future: should they live > > under airflow-core, under task-sdk, or in a separate dedicated location? > > > > This is an excellent question. And as surprising as it might be I think > > there should be in .... chart > > > > chart > > tests > > unit <- current "helm_tests" > > e2e <- current "k8s_tests" > > > > > > > > On Mon, Nov 17, 2025 at 7:09 PM Zhe-You Liu <[email protected]> wrote: > > > >> Hi Jarek, > >> > >> Thanks for proposing the new e2e test structure, the task-sdk e2e setup > >> looks great to me! > >> I have one question about the k8s tests in the future: should they live > >> under airflow-core, under task-sdk, or in a separate dedicated location? > >> > >> Best regards, > >> Jason > >> > >> > >> On Mon, Nov 17, 2025 at 11:56 PM Jarek Potiuk <[email protected]> wrote: > >> > >>> Hello here, > >>> > >>> *TL;DR; We want to have better organized, and much improved new `e2e` > >> test > >>> type, a common thing in Airflow - one that others will be able to add > >> tests > >>> to and add their own similar tests following the blueprint.* > >>> > >>> Over the last few weeks quite a few of us were working on something we > >> have > >>> not fully realise we will end up and it was mostly bottom -up grass root > >>> ideas that simply "clicked" together and we would like to propose adding > >>> (or more formalising) a new type of tests in airflow - e2e tests. The > >>> people who worked on it were mostly Amogh, Zhe You lIu, Bugra, myself. > >>> > >>> Wa already have a few attempts to do it (in various stages - as top-level > >>> folders: *docker-tests, k8s-tests, airflow-e2e-tests, airflow-ctl-tests, > >>> task-sdk-integration-tests* - yes the lack of consistency in names is > >> very > >>> confusing) but with recent improvements in task-sdk-integration-tests - > >> we > >>> think we have a good proposal on how we can implement those tests and > >>> standardise it across airflow distributions. > >>> > >>> Current (best) approach is > >>> https://github.com/apache/airflow/tree/main/task-sdk-integration-tests > >>> > >>> And it has the following properties: > >>> > >>> 1) uses uv pytest (standard pytest tests), docker-compose and > >>> python-on-whales together, also has nice breeze wrapper for CI > >>> > >>> 2) automatically sets-up (and keeps running during iteration) > >>> docker-compose based airflow deployment - with components (any - > >>> api-server, dag processor, scheduler, triggerer) necessary during the > >> tests > >>> and allows to interact with it, including triggering local dags and the > >>> like > >>> > >>> 3) pytest tests are run in a local venv controlled by uv sync > >>> > >>> 4) it allows for extremely fast (almost like pytest-native in venv) > >>> iteration speed with tests - it uses Zhe's hot-reload functionality > >> added a > >>> week ago in order to reload components inside the docker-containers, and > >>> local sources are mounted to those - which means that just saving a file > >> in > >>> your IDE or local env will make automated (sub-second, really) reload of > >>> the components you interact with. Basically at the time you press > >> Shift-F10 > >>> (or whatever shortcut you have) to re-run your tests, your just > >> auto-saved > >>> modified Airflow code already runs in those docker-compose components (!) > >>> - *this is the most important feature of it.* > >>> > >>> 5) All those tests are automatically run in CI - in relevant PRs (driven > >> by > >>> selective checks) - but also they are very easily runnable locally in > >> local > >>> venv (no breeze CI image needed). Basically: > >>> > >>> cd task-sdk-integration-tests > >>> uv run pytest > >>> > >>> 6) Last minute addition - we've been literally asked by Kacper an hour > >> ago > >>> about having "something like that" for open lineage - every current > >>> distribution is able to have their own set of tests like that if the > >>> stewards of that part want it > >>> > >>> Enough bragging.... > >>> > >>> *Our proposal:* > >>> > >>> * we consistently rename those tests as `*e2e*` tests > >>> > >>> * we adapt all of them (including *k8s* tests eventually - where we have > >> to > >>> hack kind a bit) - to follow the same patterns and principles > >>> > >>> * we extract common code for that to `*devel-common*` and reuse across > >>> those tests > >>> > >>> * we get rid of all the top-level `*SMTH-test*` folders we have now and > >>> move those different kinds of tests to `*tests/e2e*` (next to `unit`, > >>> `system`, `integration` we already have) of distributions that the tests > >>> are relevant to > >>> > >>> For example - instead of: > >>> > >>> task-sdk > >>> src > >>> tests > >>> task_sdk > >>> some_unit_test.py > >>> task-sdk-integration-tests > >>> dags <- here e2e test dags are stored > >>> logs <- here logs from all components are kept (.gitignored) > >>> tests > >>> task_sdk_tests > >>> some_end_2_end_test.py > >>> > >>> We propose: > >>> > >>> > >>> task-sdk > >>> src > >>> tests > >>> unit > >>> task_sdk > >>> some_unit_test.py > >>> e2e > >>> dags > >>> logs > >>> task_sdk > >>> some_end_2_end_test.py > >>> > >>> Of course - this is a proposal - and if there are other ideas on how to > >>> restructure the test folders, that might be a good idea to do it now. > >> *Only > >>> serious and complete offers are going to be considered :)*, so I propose > >>> constructive, complete proposals rather than criticising the current one. > >>> > >>> *Let us know what you think,* > >>> > >>> Jarek, Zhe, Bugra, Amogh > >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
