Echo'ing Pierre. I don't really have an opinion on the tech stack itself, but feels it comes down to who we expect to own most of the tests. If it's mostly front-end folks, then TypeScript probably makes more sense. If it's mostly core Airflow engineers writing end-to-end flow tests, then Python would be more natural. Sounds like we're leaning toward the former?
On Wed, Sep 24, 2025 at 11:31 AM Pierre Jeambrun <[email protected]> wrote: > I also think that having a good unit test coverage + e2e tests would really > help maintaining quality of releases and prevent regressions early, making > all that much more manageable. > > My two cents regarding the questions above: > > - > Should we prioritize expanding unit test coverage first, or work on unit > and E2E tests in parallel? > > Both in parallel sounds good to me. They do not depend on each other (At > least in my mind) > - > For E2E testing, is Playwright a good choice, or are there other > preferred tools we should explore? > > I like playwright and I feel this is the modern solution to go to. > - > If Playwright, should we use TypeScript or Python as language for tests? > > Good arguments mentioned above for both sides. I personally think (maybe > wrongly), that ideally people maintaining / developing e2e tests will be > people with > front-end knowledge. (events, identifiers, selectors, css styling, > browser local states, etc..), so I tend to think that those people will > most likely come from the front-end community and > having that in typescript would probably help them get started. > > > On Wed, Sep 24, 2025 at 10:01 AM Amogh Desai <[email protected]> > wrote: > > > Thanks for starting this discussion, Rahul. > > > > I am aligned with your thoughts, we have seen many many UI issues lately > > which I believe we should prioritize fixing or at least reducing going > > forward. > > > > To answer your question(s) -- > > > > * I think the unit tests and e2e / user flow tests as orthogonal, so they > > can be > > developed in parallel. > > > > * While I do not have real experience with Playwright but from what I > hear > > / read on reddit, > > github, and related forums, it looks like Playwright can do similar job > as > > alternatives such as > > Selenium, Cypress, while being more beginner friendly, which could play > > better in our favour. > > > > * Again, not very strong on this but I feel that having Typescript as the > > language of choice would limit > > the number of contributors we get to help on this. Python would be my > > weapon of choice, given > > the "testing" is not limited by it and it potentially could encourage > > broader participation. > > > > (Folks active on CI/CD(atleast me), are not very comfortable with > > Typescript tests compared to > > Python) > > > > One additional point: It would be valuable if we extract the e2e / user > > flow testing into a GH job that > > a release manager could run as "sort" of last validation before releasing > > to avoid any last > > minute issues. > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Tue, Sep 23, 2025 at 9:15 PM Brent Bovenzi via dev < > > [email protected]> wrote: > > > > > I agree that we need to improve UI tests. Happy to make that a priority > > > during 3.1.x and 3.2 development. > > > > > > Personally, I will start with expanding unit tests first. But that > > doesn't > > > mean others can't get started on E2E tests. > > > > > > I lean towards using typescript for playwright. With E2E we are testing > > the > > > UI more than we are testing the API so I think the code should sit > closer > > > to the UI code. > > > > > > On Tue, Sep 23, 2025 at 9:38 AM Vincent Beck <[email protected]> > > wrote: > > > > > > > Not answering any of these questions (sorry) but strong +1 on adding > > > > end-to-end (E2E) tests. This will significantly make our UI more > robust > > > but > > > > also will speed up some of our work. An example is front-end > dependency > > > > upgrades by dependabot. Today we have no way of knowing whether these > > PRs > > > > break the UI, in order to merge these PRs we need to compile the > > > front-end > > > > assets and test manually whether the UI work. Having end-to-end tests > > on > > > > the front-end and making them run in the CI would speed up this very > > > > example (on top of increasing code robustness, increase our > confidence > > > etc). > > > > > > > > On 2025/09/23 11:03:40 Rahul Vats wrote: > > > > > Hi all, > > > > > > > > > > With Airflow 3.0, the UI has undergone a complete revamp, and new > > > changes > > > > > are naturally more prone to regressions. We've already seen more UI > > > > > regressions that are hard to catch with our current limited test > > > > coverage. > > > > > Manual testing is also challenging given the wide variety of > > workflows > > > > and > > > > > screens in the UI. > > > > > > > > > > Currently, the Airflow UI has a few unit tests using Vitest and > React > > > > > Testing Library, but coverage is quite limited — only about 4 test > > > files > > > > > covering major pages like DagsList and TaskInstance. Many critical > > > > > pages (Connections, > > > > > Variables, Pools, Dashboard, etc.) have no test coverage at all. > > > > > > > > > > I’m exploring a two-pronged approach to improve UI testing: > > > > > > > > > > 1. > > > > > > > > > > Expand unit test coverage for individual components and pages > > > > > > > > > > 2. > > > > > > > > > > Add end-to-end (E2E) tests using Playwright for complete user > > > > workflows > > > > > > > > > > > > > > > This would give us both component-level safety and full > user-journey > > > > > testing across browsers, helping catch regressions early. > > > > > > > > > > A few questions: > > > > > > > > > > - > > > > > > > > > > Should we prioritize expanding unit test coverage first, or work > > on > > > > unit > > > > > and E2E tests in parallel? > > > > > > > > > > - > > > > > > > > > > For E2E testing, is Playwright a good choice, or are there other > > > > > preferred tools we should explore? > > > > > > > > > > - > > > > > > > > > > If Playwright, should we use TypeScript or Python as language > for > > > > tests? > > > > > > > > > > - > > > > > > > > > > Any other suggestions to strengthen our UI testing strategy? > > > > > > > > > > > > > > > I’d love to hear your thoughts on whether this would be valuable > for > > > the > > > > > project before diving into implementation details. > > > > > > > > > > Regards, > > > > > Rahul Vats > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > >
