Hi, I think a few stuff were left in the air about the huge testing challenge we face, and how to put the outreach program to play in that direction. I would like to organize a few of my own thoughts, and get other people's opinions on this. I'm not appending this to the existing thread in desktop-devel, because I think this might be less specific than that conversation. If you are in a rush, jump to the conclusion at the bottom.
In my mind there are a few things we mean when we say "testing", here they are: UNIT TESTS These are small non-runtime tests that are usually invoked by "make check". They are ideal for shared libraries. There are plenty of GNOME modules out there that take advantage of autotools integration of these kinds of tests, and the GNOME build brigade provides test results during it's (nightly?) automated builds. In my mind we have, or potentially have, good coverage of these kinds of tests. It's very much like tinderbox, you could see it in action here: http://build.gnome.org AUTOMATED GUI TESTS These are runtime tests that automate user interaction, and have different methods of asserting the test results. This is what the talk about is in desktop-devel[1]. I think everyone there is in agreement that - the status quo sucks, we need an open and collaborative space for automated testing, maybe something like build.gnome.org. And a blessed single testing toolkit. Coincidentally, automated GUI testing usually relies heavily on our accessibility framework. But let's not get confused, the main purpose of most of these tests is not accessibility, a developer will put these tests in place to squash conventional bugs. Will has a hidden agenda when he pushes for these kinds of tests in other GNOME modules, we get to exercise an application's accessibility "for free". ACCESSIBILITY TESTS Unlike the top two categories, accessibility tests don't have a specific method, it's just any kind of test that tests for accessibility, they could be unit tests, or GUI tests. They could also be a third kind of test, and that is external poking, like at-poke and accerciser do. They could be automated, and they could be manual. I think this is a very raw but fertile ground. A good example of a start is Peter Parente's validation plugin[2] for Accerciser, it will give a lint-style accessibility test. CONCLUSION In my opinion, our outreach program should focus on the third meaning of "testing" - accessibility testing. Besides the obvious test schemas and accerciser extensions, I think this also entails work inside other modules, and adding to their testing story a sleuth of tests that test nothing other than accessibility. It could be as simple as writing a test that searches for unrelated labels in a glade file, this script could be duplicated across every module that uses glade, and run during "make check". Or a dogtail sequence that simply listens for the right events as buttons get clicked. Think about it - I haven't. [3] Eitan. 1. http://mail.gnome.org/archives/desktop-devel-list/2008-February/msg00103.html 2. http://pithy.tumblr.com/post/23941873 3. Stephen Colbert _______________________________________________ gnome-accessibility-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/gnome-accessibility-list
