Rahul, Thank you for starting this thread. Strong +1 from me as well. I not only agree with this, but I believe this needs to be a high priority going forward and before 3.2.
I also agree with the comments on the "test pyramid" approach, of there needing to be FAR more unit tests as compared to end to end tests. I also believe that a certain level of rigor around this i.e. some form of enforcement to ensure that all new UI changes come with a unit test. I would also add the Core API changes (for UI) to the mix here, since a fair number of UI changes requires underlying API changes. Vikram On Thu, Sep 25, 2025 at 12:04 AM Amogh Desai <[email protected]> wrote: > Valid points. > > I was just expressing concern from a CI/CD failure perspective but in no > way > was I objecting to usage of typescript. > > Anything that works for the team that owns most of the tests works by me! > > > Thanks & Regards, > Amogh Desai > > > On Thu, Sep 25, 2025 at 1:39 AM Jens Scheffler <[email protected]> > wrote: > > > Echoing a full agreement to all statement before. > > > > I also would emphasize the test pyramid. As we really have a low > > coverage we should maybe also have a metric. I would also propose that > > we focus more on testing coverage == hopefully also improved quality > > preventing regression in 3.2. One option could a bit of force to force > > all new changes applied to UI to request a unit test being added (all > > changes TS/x files need to have a corresponding .test.ts/x file) except > > for cases where it does not make sense we allow exclusion of the rule by > > adding a label by maintainers. (just want to drop the idea, might be > > many will hate it) > > > > For language also have no strong preference but even though I am a TS > > noob I see that testing UI is closer to UI development, to using Python > > for TS UI testing might make a knowledge gap also for potential utility > > code. So as most voices would propose TS (maybe then finally I get to > > understand it) > > > > Jens > > > > On 24.09.25 18:33, Jarek Potiuk wrote: > > > Small poking at important things from holidays while having a coffee > at a > > > place with internet :) > > > > > > +1 on having both. I have no strong opinion on the language and > > Playwright. > > > > > > But there are few things for consideration: > > > > > > We should have many, many more unit tests than e2e tests. This is the > > > classic test pyramid case: The Practical Test Pyramid > > > https://share.google/YHVIt8EBptT1yYFur > > > > > > The e2e tests are slow, brittle and difficult to maintain and we should > > > have rather small number of those - mostly green path on all major > > > functionalities. Possibly with some cross browser checks - but those > are > > > very difficult to get right. > > > > > > It will be a pain to maintain those. Like our k8s e2e tests - when they > > > break (rarely) people will have hard time to create and run the tests > so > > > the tooling around that has to drive people who will be reproducing > the > > > failures by hand - how to set it up and how to reproduce it and how to > > > iterate as fast as possible. > > > > > > Keeping the tests stable and side-effect free will be a challenge - so > > > likely a good 'reset the environment' before each test is a must. Also > > they > > > need to be run with all UI changes in CI which will make UI PR run > > longer. > > > It's an imperative to get them run not only in canary builds but in all > > PRs > > > for ui and API and likely whenever any core model change. It will slow > > down > > > UI development and create frustration for contributors unfamiliar with > > the > > > test framework so we need to have at least a few people proficient in > > > intricacies of the test framework dedicated and willing to help those > who > > > will struggle with issues. > > > > > > Language is secondary - e2e tests will require all stack - Python > > (backend) > > > and Typescript and react experiences anyway. Most likely people who > write > > > UI should be involved so Typescript is fine, and also even those who do > > not > > > know TypeScript will be able to fix some obvious cases. > > > > > > J. > > > > > > śr., 24 wrz 2025, 09:50 użytkownik Shubham Raj <[email protected]> > > > napisał: > > > > > >> This addition will be really nice. > > >> Also, at this point of 3.x, I think UI has been relatively stable and > > >> writing e2e would be a good starting point. Unit tests can go > parallel. > > >> It's right that people maintaining CI/CD would be first persons to see > > any > > >> breakage, but I feel writing in ts will be easier for folks working on > > >> frontend, and also for the new contributors who would like to > > contribute in > > >> this UI e2e tests coming from cypress or similar experience, since > > majority > > >> of other choices do come with ts/js support. > > >> I would be happy to help with the e2e tests. We should probably go in > > >> phases with P0/1/2. > > >> > > >> Thanks. > > >> > > >> > > >> On Wed, Sep 24, 2025 at 9:01 PM Pierre Jeambrun < > [email protected]> > > >> wrote: > > >> > > >>> I also think that having a good unit test coverage + e2e tests would > > >> really > > >>> help maintaining quality of releases and prevent regressions early, > > >> making > > >>> all that much more manageable. > > >>> > > >>> My two cents regarding the questions above: > > >>> > > >>> - > > >>> Should we prioritize expanding unit test coverage first, or work > on > > >> unit > > >>> and E2E tests in parallel? > > >>> > > >>> Both in parallel sounds good to me. They do not depend on each > > other > > >> (At > > >>> least in my mind) > > >>> - > > >>> For E2E testing, is Playwright a good choice, or are there other > > >>> preferred tools we should explore? > > >>> > > >>> I like playwright and I feel this is the modern solution to go > to. > > >>> - > > >>> If Playwright, should we use TypeScript or Python as language for > > >> tests? > > >>> Good arguments mentioned above for both sides. I personally think > > >> (maybe > > >>> wrongly), that ideally people maintaining / developing e2e tests will > > be > > >>> people with > > >>> front-end knowledge. (events, identifiers, selectors, css > styling, > > >>> browser local states, etc..), so I tend to think that those people > > will > > >>> most likely come from the front-end community and > > >>> having that in typescript would probably help them get started. > > >>> > > >>> > > >>> On Wed, Sep 24, 2025 at 10:01 AM Amogh Desai <[email protected]> > > >>> wrote: > > >>> > > >>>> Thanks for starting this discussion, Rahul. > > >>>> > > >>>> I am aligned with your thoughts, we have seen many many UI issues > > >> lately > > >>>> which I believe we should prioritize fixing or at least reducing > going > > >>>> forward. > > >>>> > > >>>> To answer your question(s) -- > > >>>> > > >>>> * I think the unit tests and e2e / user flow tests as orthogonal, so > > >> they > > >>>> can be > > >>>> developed in parallel. > > >>>> > > >>>> * While I do not have real experience with Playwright but from what > I > > >>> hear > > >>>> / read on reddit, > > >>>> github, and related forums, it looks like Playwright can do similar > > job > > >>> as > > >>>> alternatives such as > > >>>> Selenium, Cypress, while being more beginner friendly, which could > > play > > >>>> better in our favour. > > >>>> > > >>>> * Again, not very strong on this but I feel that having Typescript > as > > >> the > > >>>> language of choice would limit > > >>>> the number of contributors we get to help on this. Python would be > my > > >>>> weapon of choice, given > > >>>> the "testing" is not limited by it and it potentially could > encourage > > >>>> broader participation. > > >>>> > > >>>> (Folks active on CI/CD(atleast me), are not very comfortable with > > >>>> Typescript tests compared to > > >>>> Python) > > >>>> > > >>>> One additional point: It would be valuable if we extract the e2e / > > user > > >>>> flow testing into a GH job that > > >>>> a release manager could run as "sort" of last validation before > > >> releasing > > >>>> to avoid any last > > >>>> minute issues. > > >>>> > > >>>> Thanks & Regards, > > >>>> Amogh Desai > > >>>> > > >>>> > > >>>> On Tue, Sep 23, 2025 at 9:15 PM Brent Bovenzi via dev < > > >>>> [email protected]> wrote: > > >>>> > > >>>>> I agree that we need to improve UI tests. Happy to make that a > > >> priority > > >>>>> during 3.1.x and 3.2 development. > > >>>>> > > >>>>> Personally, I will start with expanding unit tests first. But that > > >>>> doesn't > > >>>>> mean others can't get started on E2E tests. > > >>>>> > > >>>>> I lean towards using typescript for playwright. With E2E we are > > >> testing > > >>>> the > > >>>>> UI more than we are testing the API so I think the code should sit > > >>> closer > > >>>>> to the UI code. > > >>>>> > > >>>>> On Tue, Sep 23, 2025 at 9:38 AM Vincent Beck <[email protected]> > > >>>> wrote: > > >>>>>> Not answering any of these questions (sorry) but strong +1 on > > >> adding > > >>>>>> end-to-end (E2E) tests. This will significantly make our UI more > > >>> robust > > >>>>> but > > >>>>>> also will speed up some of our work. An example is front-end > > >>> dependency > > >>>>>> upgrades by dependabot. Today we have no way of knowing whether > > >> these > > >>>> PRs > > >>>>>> break the UI, in order to merge these PRs we need to compile the > > >>>>> front-end > > >>>>>> assets and test manually whether the UI work. Having end-to-end > > >> tests > > >>>> on > > >>>>>> the front-end and making them run in the CI would speed up this > > >> very > > >>>>>> example (on top of increasing code robustness, increase our > > >>> confidence > > >>>>> etc). > > >>>>>> On 2025/09/23 11:03:40 Rahul Vats wrote: > > >>>>>>> Hi all, > > >>>>>>> > > >>>>>>> With Airflow 3.0, the UI has undergone a complete revamp, and new > > >>>>> changes > > >>>>>>> are naturally more prone to regressions. We've already seen more > > >> UI > > >>>>>>> regressions that are hard to catch with our current limited test > > >>>>>> coverage. > > >>>>>>> Manual testing is also challenging given the wide variety of > > >>>> workflows > > >>>>>> and > > >>>>>>> screens in the UI. > > >>>>>>> > > >>>>>>> Currently, the Airflow UI has a few unit tests using Vitest and > > >>> React > > >>>>>>> Testing Library, but coverage is quite limited — only about 4 > > >> test > > >>>>> files > > >>>>>>> covering major pages like DagsList and TaskInstance. Many > > >> critical > > >>>>>>> pages (Connections, > > >>>>>>> Variables, Pools, Dashboard, etc.) have no test coverage at all. > > >>>>>>> > > >>>>>>> I’m exploring a two-pronged approach to improve UI testing: > > >>>>>>> > > >>>>>>> 1. > > >>>>>>> > > >>>>>>> Expand unit test coverage for individual components and pages > > >>>>>>> > > >>>>>>> 2. > > >>>>>>> > > >>>>>>> Add end-to-end (E2E) tests using Playwright for complete user > > >>>>>> workflows > > >>>>>>> > > >>>>>>> This would give us both component-level safety and full > > >>> user-journey > > >>>>>>> testing across browsers, helping catch regressions early. > > >>>>>>> > > >>>>>>> A few questions: > > >>>>>>> > > >>>>>>> - > > >>>>>>> > > >>>>>>> Should we prioritize expanding unit test coverage first, or > > >> work > > >>>> on > > >>>>>> unit > > >>>>>>> and E2E tests in parallel? > > >>>>>>> > > >>>>>>> - > > >>>>>>> > > >>>>>>> For E2E testing, is Playwright a good choice, or are there > > >> other > > >>>>>>> preferred tools we should explore? > > >>>>>>> > > >>>>>>> - > > >>>>>>> > > >>>>>>> If Playwright, should we use TypeScript or Python as language > > >>> for > > >>>>>> tests? > > >>>>>>> - > > >>>>>>> > > >>>>>>> Any other suggestions to strengthen our UI testing strategy? > > >>>>>>> > > >>>>>>> > > >>>>>>> I’d love to hear your thoughts on whether this would be valuable > > >>> for > > >>>>> the > > >>>>>>> project before diving into implementation details. > > >>>>>>> > > >>>>>>> Regards, > > >>>>>>> Rahul Vats > > >>>>>>> > > >>>>>> > > >> --------------------------------------------------------------------- > > >>>>>> To unsubscribe, e-mail: [email protected] > > >>>>>> For additional commands, e-mail: [email protected] > > >>>>>> > > >>>>>> > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
