[discussion from -committers]
On 3/26/20 4:31 PM, Tom Lane wrote: > Andrew Dunstan <andrew.duns...@2ndquadrant.com> writes: >> On 3/26/20 11:31 AM, Tom Lane wrote: >>> Andrew Dunstan <andrew.duns...@2ndquadrant.com> writes: >>>> I don't think this belongs in installcheck, we should add >>>> 'NO_INSTALLCHECK = 1' to the Makefile. >>> Why? The other src/test/modules/ modules with TAP tests do not >>> specify that, with the exception of commit_ts which has a solid >>> doesnt-work-in-the-default-configuration excuse. >> That seems wrong, installcheck should be testing against an installed >> instance, and the TAP tests don't. > So? We clearly document that for the TAP tests, "make installcheck" > means "use the installed executables, but run a new instance" [1]. I think we were probably a bit shortsighted about that. But what's done is done. I wonder if there is a simple way we could turn it off for the buildfarm? >> Moreover, from the buildfarm's POV >> it's completely wrong, as we call the installcheck targets multiple >> times, once for each configured locale. See one of the animals that >> tests multiple locales (e.g. crake or prion) > Yeah. That's productive if you think the tests might be > locale-sensitive. I doubt that any of the ones under src/test/modules/ > actually are at the moment, so maybe this is a waste of buildfarm effort. > But I don't think that it's the place of the Makefiles to dictate such > policy, and especially not for them to do so by breaking the ability to > use "make installcheck" at all. > > regards, tom lane > > [1] https://www.postgresql.org/docs/devel/regress-tap.html Right now the explicit TAP test code in the buildfarm knows how to collect all the relevant output. The installcheck code doesn't know about that for TAP tests. I get that developers want to be able to run tests in a small number of commands, but for the buildfarm I generally favor more disaggregated tests. That way if test X fails it's much easier to focus on the problem. (related note: I'm working on breaking up the log text blobs which will also help focussing on the right area). Maybe we need to take the discussion to -hackers. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
