Thanks for sharing this. Some / Most providers already follow a similar structure: https://github.com/apache/airflow/tree/main/providers/amazon/tests and having related things closer will provide better maintainability as well as discoverability. Any effort in that area is highly appreciated.
That said, for k8s_tests, I think the best course of action would be to separate it out and place it where it belongs. What I mean by that is, the k8s_tests/ has tests varying from testing running dags using the KubernetesExecutor to testing KPO and also testing any executor / general execution with K8s as the environment. A lot of this can be cleaned up, move KE / KPO tests to cncf provider and general execution to helm_tests as e2e tests or the likes of it. Although this is a separate discussion that can be taken up later. Thanks & Regards, Amogh Desai On Tue, Nov 18, 2025 at 3:47 AM Jarek Potiuk <[email protected]> wrote: > > I'd put them rather in core as an integration for this... but as said, > there might be more options and much more opinions in the room. But > capability is more important so "praise" to all who have contributed to > this! (Hope Edge will be added there soon as well...) > > We can also put them to core, yes, but my reasoning why `chart` is > that they **actually** use the chart and they **mainly** test if > Airflow installed with the chart "works". So they are (at least the > main ones that work with all executor+k8s matrix) are pretty muc > testing the chart e2e. > > But there are indeed some tests that are testing k8s pod operators, > and maybe a good compromise will be to split those and put the "chart" > tests in chart, but the "k8s" Pod operator ones in "cncf.kubernetes" > provider. Those tests are anyhow quite a bit different and we won't > touch them for a while - so we can decide later. Also I think the k8s > POD operators might be anyhow eventually separated and done > differently - Kalyan currently works on making the `breeze --executor > Kubernetes` to work and once it does, we might want to separate the > K8SPod Operators anyway to use this rather than setting up airflow > with the chart - as it's not needed there at all. > > I would just defer the decision on k8s_tests for now. > > > J. > > On Mon, Nov 17, 2025 at 11:03 PM Jens Scheffler <[email protected]> > wrote: > > > > Very cool! > > > > Not sure whether they are really placed well in "chart" because we do > > not want to use K8s tests to check the chart but the full integration of > > Airflow with Kubernetes. I tincludes core, K8s provider, Task SDK... so > > for K8s tests we need at least one exception to the rule. > > > > I'd put them rather in core as an integration for this... but as said, > > there might be more options and much more opinions in the room. But > > capability is more important so "praise" to all who have contributed to > > this! (Hope Edge will be added there soon as well...) > > > > Jens > > > > On 11/17/25 19:41, Jarek Potiuk wrote: > > >> I have one question about the k8s tests in the future: should they > live > > > under airflow-core, under task-sdk, or in a separate dedicated > location? > > > > > > This is an excellent question. And as surprising as it might be I think > > > there should be in .... chart > > > > > > chart > > > tests > > > unit <- current "helm_tests" > > > e2e <- current "k8s_tests" > > > > > > > > > > > > On Mon, Nov 17, 2025 at 7:09 PM Zhe-You Liu <[email protected]> > wrote: > > > > > >> Hi Jarek, > > >> > > >> Thanks for proposing the new e2e test structure, the task-sdk e2e > setup > > >> looks great to me! > > >> I have one question about the k8s tests in the future: should they > live > > >> under airflow-core, under task-sdk, or in a separate dedicated > location? > > >> > > >> Best regards, > > >> Jason > > >> > > >> > > >> On Mon, Nov 17, 2025 at 11:56 PM Jarek Potiuk <[email protected]> > wrote: > > >> > > >>> Hello here, > > >>> > > >>> *TL;DR; We want to have better organized, and much improved new > `e2e` > > >> test > > >>> type, a common thing in Airflow - one that others will be able to add > > >> tests > > >>> to and add their own similar tests following the blueprint.* > > >>> > > >>> Over the last few weeks quite a few of us were working on something > we > > >> have > > >>> not fully realise we will end up and it was mostly bottom -up grass > root > > >>> ideas that simply "clicked" together and we would like to propose > adding > > >>> (or more formalising) a new type of tests in airflow - e2e tests. The > > >>> people who worked on it were mostly Amogh, Zhe You lIu, Bugra, > myself. > > >>> > > >>> Wa already have a few attempts to do it (in various stages - as > top-level > > >>> folders: *docker-tests, k8s-tests, airflow-e2e-tests, > airflow-ctl-tests, > > >>> task-sdk-integration-tests* - yes the lack of consistency in names is > > >> very > > >>> confusing) but with recent improvements in > task-sdk-integration-tests - > > >> we > > >>> think we have a good proposal on how we can implement those tests and > > >>> standardise it across airflow distributions. > > >>> > > >>> Current (best) approach is > > >>> > https://github.com/apache/airflow/tree/main/task-sdk-integration-tests > > >>> > > >>> And it has the following properties: > > >>> > > >>> 1) uses uv pytest (standard pytest tests), docker-compose and > > >>> python-on-whales together, also has nice breeze wrapper for CI > > >>> > > >>> 2) automatically sets-up (and keeps running during iteration) > > >>> docker-compose based airflow deployment - with components (any - > > >>> api-server, dag processor, scheduler, triggerer) necessary during the > > >> tests > > >>> and allows to interact with it, including triggering local dags and > the > > >>> like > > >>> > > >>> 3) pytest tests are run in a local venv controlled by uv sync > > >>> > > >>> 4) it allows for extremely fast (almost like pytest-native in venv) > > >>> iteration speed with tests - it uses Zhe's hot-reload functionality > > >> added a > > >>> week ago in order to reload components inside the docker-containers, > and > > >>> local sources are mounted to those - which means that just saving a > file > > >> in > > >>> your IDE or local env will make automated (sub-second, really) > reload of > > >>> the components you interact with. Basically at the time you press > > >> Shift-F10 > > >>> (or whatever shortcut you have) to re-run your tests, your just > > >> auto-saved > > >>> modified Airflow code already runs in those docker-compose > components (!) > > >>> - *this is the most important feature of it.* > > >>> > > >>> 5) All those tests are automatically run in CI - in relevant PRs > (driven > > >> by > > >>> selective checks) - but also they are very easily runnable locally in > > >> local > > >>> venv (no breeze CI image needed). Basically: > > >>> > > >>> cd task-sdk-integration-tests > > >>> uv run pytest > > >>> > > >>> 6) Last minute addition - we've been literally asked by Kacper an > hour > > >> ago > > >>> about having "something like that" for open lineage - every current > > >>> distribution is able to have their own set of tests like that if the > > >>> stewards of that part want it > > >>> > > >>> Enough bragging.... > > >>> > > >>> *Our proposal:* > > >>> > > >>> * we consistently rename those tests as `*e2e*` tests > > >>> > > >>> * we adapt all of them (including *k8s* tests eventually - where we > have > > >> to > > >>> hack kind a bit) - to follow the same patterns and principles > > >>> > > >>> * we extract common code for that to `*devel-common*` and reuse > across > > >>> those tests > > >>> > > >>> * we get rid of all the top-level `*SMTH-test*` folders we have now > and > > >>> move those different kinds of tests to `*tests/e2e*` (next to `unit`, > > >>> `system`, `integration` we already have) of distributions that the > tests > > >>> are relevant to > > >>> > > >>> For example - instead of: > > >>> > > >>> task-sdk > > >>> src > > >>> tests > > >>> task_sdk > > >>> some_unit_test.py > > >>> task-sdk-integration-tests > > >>> dags <- here e2e test dags are stored > > >>> logs <- here logs from all components are kept (.gitignored) > > >>> tests > > >>> task_sdk_tests > > >>> some_end_2_end_test.py > > >>> > > >>> We propose: > > >>> > > >>> > > >>> task-sdk > > >>> src > > >>> tests > > >>> unit > > >>> task_sdk > > >>> some_unit_test.py > > >>> e2e > > >>> dags > > >>> logs > > >>> task_sdk > > >>> some_end_2_end_test.py > > >>> > > >>> Of course - this is a proposal - and if there are other ideas on how > to > > >>> restructure the test folders, that might be a good idea to do it now. > > >> *Only > > >>> serious and complete offers are going to be considered :)*, so I > propose > > >>> constructive, complete proposals rather than criticising the current > one. > > >>> > > >>> *Let us know what you think,* > > >>> > > >>> Jarek, Zhe, Bugra, Amogh > > >>> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
