"Frank Gordon" <[EMAIL PROTECTED]> wrote: > I'm a newbie. I am also a test engineer interested in the amount and types > of testing that are performed on SQLite. Especially on releases of new > versions. > Can anyone shed any light on this topic? > > I have spent a considerable amount of days looking for SQLite regression > tests that are available, how often are they run, and where to find the > results. So far I feel I have exhausted my search and came up way to short > on information. I have only found regression tcl scripts at the following > url: http://www.sqlite.org/cvstrac/dir?d=sqlite/test However, no > documentation summarizing this testing and no posted results from > SQLite.org. My > understanding is that over 200,000 test cases are being run by SQLite.org, > according to check-in [3144] there is 98% test coverage, and that the > community of developers utilizing SQLite in there projects are also running > these regression tests on there individual platforms. >
Testing of SQLite is done with a program called "testfixture". Testfixture contains an instrumented version of SQLite plus a TCL interpreter for running the tests. You can build testfixture by typing "make testfixture" on unix. It is more complicated on windows. Once you have your testfixture, you run the tests by typing: ./testfixture ../sqlite/test/quick.test or ./testfixture ../sqlite/test/all.test The "all.test" script runs in excess of 95% of source lines, measured using gcov. The object code coverage is lower but is still relatively high. The code that is not run is usually unreachable. Special compile-time options are needed to get high code coverage. SQLite does a lot of very speciallized tests. For example, there is a customized backend to sqlite (similar to os_unix.c and os_win.c) that simulates a computer that can suffer a random power failure in the middle of a transaction. This is used to verify that the recovery and rollback logic is all working correctly. This customized backend is only available when you compile with the right flags. Over 60% of the non-comment source code in SQLite is devoted purely to testing. I generally run quick.test on Linux before each check-in and all.test on both Linux and windows and quick.test on MacOSX before each release. -- D. Richard Hipp <[EMAIL PROTECTED]>
