Ben Morgan wrote:
If you have 66 books of the Bible and many different ways of referencing them, testing every single way isn't necessarily all that easy. Then multiply that by every locale that is there.
A regression test doesn't (usually) try to give complete coverage of anything, just enough tests to convince the team that functionality that is now "correct" does not later become "incorrect" without someone quickly noticing the change.
In this case, testing (say) 3 common abbreviations for each book name followed by the first verse of the first chapter, for each of the 66 books, is 66 * 3 = 198 tests, and just using a single very boring locale (either C or en_US.UTF-8) would catch what happened recently -- and so would notify developers by failing if the change were to accidentally disappear (the wrong revision is tagged/checked out of svn by mistake, say) between now and the official 1.6 release, for example.
In practice, a really "good" regression test for abbreviations might include a couple of other locales too, perhaps one Western European one that needs a few accented Latin characters (French or German or Spanish, say), and one that needs its own non-Roman alphabet (Cyrillic, or Korean, or Arabic, maybe).
At this level of coverage, we'd be looking at a maximum of 66 * 3 * 3 = under 600 test items. That's entirely doable, surely?
Beyond that, subject matter experts for each locale might come up with "special" "known to be peculiar" abbreviations they feel it would be good to add as additional tests... but that's moving away from the realm of basic regression testing, really, I think.
If I have time later tonight, once I have a sword 1.6.0RC2 package out for front end developers to test with, I'll see about creating the 198 item single locale test and (if I succeed!) I'll contribute it to the SWORD codebase, just to demonstrate that creating such a test really is in fact doable :)
I think the vk1 = vk2 change is more important to test than the other, really.
You provided no basis for this opinion. I think that *any* significant change that happens after an rc release is worthy of a regression test, more or less by definition. My reasoning: the developers thought the codebase was good enough to release, and it turned out not to be. This most likely means that the bugs found were unexpected or subtle or hard to test for, and yet were deemed significant enough to warrant fixing before final release. Therefore, surely, each such bug deserves a test in the regression test suite.
Basically, bugs you find in an RC release are often the "creme de la creme" of bugs... treat them with respect :)
Jonathan _______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
