Hi, I've been offered a spot to present keytest at LinuxConf 2011. There has been previous discussion of monkey testing as a cost-effective way of finding bugs. I was thinking of presenting an argument that keytest generated bugs reports can also be of higher quality than typical user generated bug reports, because e.g. - Keytest can automatically add bisection information etc. - Keytest can provide bug reports in trunk when the code is fresh and before the bugs have been fossilized in Ubuntu packages, whereas user generated bug reports tend to come long after the code was written. - The main downside of Keytest seems to be its tenancy to flood trac, but that can be somewhat worked around by not marking obscure keytest bugs as being highest priority.
I was wondering if the people who actually dealt with the bug reports agreed with this sentiment, and also if anyone wanted to voice any view points that they would like me to include in the presentation. Abstract: We present a new tool Mon-Keytest for performing "Monkey Testing" on GUI software. Monkey testing involves sending randomly generated user input to an application until it crashes or otherwise generates an obvious error. It is well known that Monkey Testing can be an effective way of finding bugs. However, many projects have no shortage of bug reports. For example, at time of writing Ubuntu is up to bug number 682551. We argue that Monkey Testing can still be useful in such cases. Mon-Keytest can save developer time by automatically generating information such as the minimal set of keypresses needed to reproduce the bug, and the revision which introduced the regression. Additionally, user generated bug reports tend to come only once a stable release has been made. Mon-Keytest can be run continuously on development snapshots, reporting bugs when the code is still fresh in the developers mind and giving them plenty of time to fix the bug before it affects real users. Hmm ... I guess I should also mention that Keytest is currently only of limited use to non-LyX projects, as it has lots of svn, trac and even LyX specific features. -- John C. McCabe-Dansted
