Thanks for starting this discussion, Rahul. I am aligned with your thoughts, we have seen many many UI issues lately which I believe we should prioritize fixing or at least reducing going forward.
To answer your question(s) -- * I think the unit tests and e2e / user flow tests as orthogonal, so they can be developed in parallel. * While I do not have real experience with Playwright but from what I hear / read on reddit, github, and related forums, it looks like Playwright can do similar job as alternatives such as Selenium, Cypress, while being more beginner friendly, which could play better in our favour. * Again, not very strong on this but I feel that having Typescript as the language of choice would limit the number of contributors we get to help on this. Python would be my weapon of choice, given the "testing" is not limited by it and it potentially could encourage broader participation. (Folks active on CI/CD(atleast me), are not very comfortable with Typescript tests compared to Python) One additional point: It would be valuable if we extract the e2e / user flow testing into a GH job that a release manager could run as "sort" of last validation before releasing to avoid any last minute issues. Thanks & Regards, Amogh Desai On Tue, Sep 23, 2025 at 9:15 PM Brent Bovenzi via dev < [email protected]> wrote: > I agree that we need to improve UI tests. Happy to make that a priority > during 3.1.x and 3.2 development. > > Personally, I will start with expanding unit tests first. But that doesn't > mean others can't get started on E2E tests. > > I lean towards using typescript for playwright. With E2E we are testing the > UI more than we are testing the API so I think the code should sit closer > to the UI code. > > On Tue, Sep 23, 2025 at 9:38 AM Vincent Beck <[email protected]> wrote: > > > Not answering any of these questions (sorry) but strong +1 on adding > > end-to-end (E2E) tests. This will significantly make our UI more robust > but > > also will speed up some of our work. An example is front-end dependency > > upgrades by dependabot. Today we have no way of knowing whether these PRs > > break the UI, in order to merge these PRs we need to compile the > front-end > > assets and test manually whether the UI work. Having end-to-end tests on > > the front-end and making them run in the CI would speed up this very > > example (on top of increasing code robustness, increase our confidence > etc). > > > > On 2025/09/23 11:03:40 Rahul Vats wrote: > > > Hi all, > > > > > > With Airflow 3.0, the UI has undergone a complete revamp, and new > changes > > > are naturally more prone to regressions. We've already seen more UI > > > regressions that are hard to catch with our current limited test > > coverage. > > > Manual testing is also challenging given the wide variety of workflows > > and > > > screens in the UI. > > > > > > Currently, the Airflow UI has a few unit tests using Vitest and React > > > Testing Library, but coverage is quite limited — only about 4 test > files > > > covering major pages like DagsList and TaskInstance. Many critical > > > pages (Connections, > > > Variables, Pools, Dashboard, etc.) have no test coverage at all. > > > > > > I’m exploring a two-pronged approach to improve UI testing: > > > > > > 1. > > > > > > Expand unit test coverage for individual components and pages > > > > > > 2. > > > > > > Add end-to-end (E2E) tests using Playwright for complete user > > workflows > > > > > > > > > This would give us both component-level safety and full user-journey > > > testing across browsers, helping catch regressions early. > > > > > > A few questions: > > > > > > - > > > > > > Should we prioritize expanding unit test coverage first, or work on > > unit > > > and E2E tests in parallel? > > > > > > - > > > > > > For E2E testing, is Playwright a good choice, or are there other > > > preferred tools we should explore? > > > > > > - > > > > > > If Playwright, should we use TypeScript or Python as language for > > tests? > > > > > > - > > > > > > Any other suggestions to strengthen our UI testing strategy? > > > > > > > > > I’d love to hear your thoughts on whether this would be valuable for > the > > > project before diving into implementation details. > > > > > > Regards, > > > Rahul Vats > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
