Hi, I'd like to talk about applying https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests to application testing.
Now, the executive summary on InstalledTests is "improved make check framework". Its focus is on rigorous automation. At the moment in GNOME, many shared libraries like glib and libsoup have test suites via "make check" historically, and a growing number are additionally implementing InstalledTests. But few applications have test suites. What application testing exists tends to live in external downstream repositories. Having implemented one of the fastest, most efficient and reliable continuous deployment systems in the world[1] for GNOME, I'd really like it if more applications had tests to take advantage of this =) https://bugzilla.gnome.org/show_bug.cgi?id=721349 is a bug for Epiphany, which in turn links to some tests Matthias wrote for gnome-software. These tests happen to be implemented using a mixture of Dogtail and DBus; they work by launching the application, talking to it via DBus, and checking the results. Your tests don't even necessarily need to spawn your app - you can also install test binaries for internal components. For example, Epiphany could install a small test binary that checked the behavior of the HTTP cache directly. What gnome-continuous will take care of is automating execution of the tests. But of course you can also use "gnome-desktop-testing-runner" inside jhbuild, as intended. The reason I want to see more application tests is because applications are what people actually use. A good example is that while GLib has some basic tests for GSocket, if we run through the Epiphany tests, we're *also* testing GSocket, as well as libsoup, and gtk+, etc. Application testing thus creates a powerful virtuous loop, testing everything at once. So please take a look at the tests Matthias wrote for gnome-software, and consider writing something similar yourself to test your applications! If you have questions about implementation, you can always catch me and several others on IRC in #gnome-hackers (EST time), or just reply to this mail. Thanks! Footnotes: [1] If you know of any other project that comes close, please send me a link _______________________________________________ desktop-devel-list mailing list [email protected] https://mail.gnome.org/mailman/listinfo/desktop-devel-list
