Robert Haas <robertmh...@gmail.com> writes: > ... That sounds a lot like the definition of a > regression test suite. Of course, we have that already, but it's > nowhere near comprehensive. Maybe we should be looking at an expanded > test suite that runs on a time scale of hours rather than seconds.
mysql's got one of those, and I haven't noticed that it's kept their defect rate down any. Hour-long regression suites are the sort of thing developers won't run. Worse, regression suites are necessarily designed to exercise only 100%-reproducible behavior. > I don't think that any test suite is going to eliminate the need for > beta-testing. Precisely... What I'd like to see is some sort of test mechanism for WAL recovery. What I've done sometimes in the past (and recently had to fix the tests to re-enable) is to kill -9 a backend immediately after running the regression tests, let the system replay the WAL for the tests, and then take a pg_dump and compare that to the dump gotten after a conventional run. However this is quite haphazard since (a) the regression tests aren't especially designed to exercise all of the WAL logic, and (b) pg_dump might not show the effects of some problems, particularly not corruption in non-system indexes. It would be worth the trouble to create a more specific test methodology. In short: merely making the tests bigger doesn't impress me in the least. Focused testing on areas we aren't covering at all could be worth the trouble. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers