On 03/18/2012 06:12 PM, Peter Geoghegan wrote:
On 18 March 2012 16:13, Tom Lane<t...@sss.pgh.pa.us>  wrote:
Is there a really strong reason why adequate regression testing isn't
possible in a plain-vanilla pg_regress script?  A quick look at the
script says that it's just doing some SQL commands and then checking the
results of queries on the pg_stat_statements views.  Admittedly the
output would be bulkier in pg_regress, which would mean that we'd not
likely want several hundred test cases.  But IMO the objective of a
regression test is not to memorialize every single case the code author
thought about during development.  ISTM it would not take very many
cases to have reasonable code coverage.
Hmm. It's difficult to have much confidence that a greatly reduced
number of test cases ought to provide sufficient coverage. I don't
disagree with your contention, I just don't know how to judge this
matter. Given that there isn't really a maintenance burden with
regression tests, I imagine that that makes it compelling to be much
more inclusive.

The fact that we rely on there being no concurrent queries might have
to be worked around for parallel scheduled regression tests, such as
by doing everything using a separate database, with that database oid
always in the predicate of the query checking the pg_stat_statements

I probably would have written the tests in Perl in the first place,
but I don't know Perl. These tests existed in some form from day 1, as
I followed a test-driven development methodology, and needed to use a
language that I could be productive in immediately. There is probably
no reason why they cannot be re-written in Perl, but spit out
pg_regress tests, compacting the otherwise-verbose pg_regress input.
Should I cut my teeth on Perl by writing the tests to do so? How might
this be integrated with the standard regression tests, if that's
something that is important?

A pg_regress script doesn't require any perl. It's pure SQL.

It is perfectly possible to make a single test its own group in a parallel schedule, and this is done now for a number of cases. See src/test/regress/parallel_schedule. Regression tests run in their own database set up for the purpose. You should be able to restrict the regression queries to only the current database.

If you want to generate the tests using some tool, then use whatever works for you, be it Python or Perl or Valgol, but ideally what is committed (and this what should be in your patch) will be the SQL output of that, not the generator plus input. Tests built that way get automatically run by the buildfarm. Tests that don't use the standard testing framework don't. You need a *really* good reason, therefore, not to do it that way.



Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to