On Sun, Oct 26, 2014 at 12:29 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > The larger issue though is that even with both the above things fixed, > the TAP tests would still be an expensive no-op on the majority of > buildfarm members. AFAICT, I do not own a single machine on which the > current TAP tests will consent to run, and in most cases that's after > going out of my way to fetch CPAN modules that aren't in the vendor Perl > installs. Peter's mostly been fixing the portability issues by disabling > tests, which I guess is better than no fix at all, but it leaves darn > little useful functionality.
I agree, emphatically. Honestly, if we can't get these tests running everywhere with reasonable effort, we should just rip them out. We've gone to a lot of trouble in general to make sure that our source code can be ported even to systems that arguably nobody uses any more, and instrumental to that effort is keeping the system requirements to install and test PostgreSQL minimal. At this point, I wouldn't mind moving the goalposts from C89 to C89 + a bunch of C99 features that are available on all the platforms we have buildfarm coverage for, and I wouldn't mind require perl to compile and install, full stop. But this patch has gone much further than that: you need a new-enough version of perl, and a new-enough version of a bunch of modules that aren't installed by default, and maybe not even in the vendor install, and the documentation on how to make it work is an embarrassment: http://www.postgresql.org/docs/devel/static/regress-tap.html Beyond all that, I have serious doubts about whether, even if we eventually get these tests mostly working in most places, whether they will actually catch any bugs. For example, consider dropdb. The TAP tests cover the following points: - When run with --help or --version, it should exit with code 0, print something on stderr, and print nothing on stdout. - When run with --not-a-valid-option, it should exit with a non-zero exit code, print something on stderr, and print nothing on stdout. - "dropdb foobar1" should cause "statement: DROP DATABASE foobar1" to show up in the postgresql log file. - "dropdb nonexistent" should exit non-zero. These are certainly good things to test, but I'd argue that once you've verified that they are working, they're unlikely to get broken again in the future. I'm generally supportive of the idea that we need more automated tests, but these tests seem pretty low-value to me, even if they were running everywhere, and even moreso if they aren't. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
