Hi, Peter Eisentraut <pete...@gmx.net> writes: > On ons, 2009-08-26 at 14:26 -0400, Robert Haas wrote: >> Sure, but an aimless mandate to do testing for 4 (or 8, or 12) months >> doesn't necessarily buy you much, either. I'm good at focused >> activity - but there was nothing focused about 8.4 beta that I could >> see. Maybe we need some kind of TestFest process. > > Yeah, exactly. I can't imagine end users would know what to do during > beta. Even assuming that you have release notes at the beginning of > beta, you can't expect people to go through every item and do a formal > test for it. Surely it's been tested before, else it would not be in > the release, right?
Well we all know that developpers are really bad at testing, because they tend to test for cases they though about while developping the code rather than being creative and running a full application against the overall new product. Now, it could be that what we miss is some tool suite to enable application people to test their full code against our new releases and check results, performance, plans, etc. I know about a couple of tools to get there, Tsung and Playr. And the focus is performance testing and scalability, not that it still works. Is the offering good enough? We might need to run some kind of tutorials for users to be able to run large tests easily, and maybe think about some newer tools allowing to compare logs of two application runs in two database versions (capture all queries and result in a database, then have a way to diff). Then beta testing would mean having a spare machine where to run the magic regression test suite against some internal application. Regards, -- dim Tsung: http://tsung.erlang-projects.org/ Playr: https://area51.myyearbook.com/trac.cgi/wiki/Playr PS: sql level unit testing isn't an answer here as it means the application already have the tests when entering beta. Hard sell. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers