Hi, On 2022-03-25 13:52:11 -0400, Robert Haas wrote: > On Fri, Mar 25, 2022 at 12:36 PM Andres Freund <and...@anarazel.de> wrote: > > Create a CF entry for it, or enable CI on a github repo? > > I created a CF entry for it. Then I had to try to Google around to > find the URL from the cfbot, because it's not even linked from > commitfest.postgresql.org for some reason. #blamemagnus
Yea, we really need to improve on that. I think Thomas has some hope of improving things after the release... > I don't think that the Windows CI is running the TAP tests for > contrib. At least, I can't find any indication of it in the output. So > it doesn't really help to assess how portable this test is, unless I'm > missing something. Yea. It's really unfortunate how vcregress.pl makes it hard to run all tests. And we're kind of stuck finding a way forward. It's easy enough to work around for individual tests by just adding them to the test file (like Thomas did nearby), but clearly that doesn't scale. Andrew wasn't happy with additional vcregress commands. The fact that vcregress doesn't run tests in parallel makes things take forever. And so it goes on. > I looked through the Linux output. It looks to me like that does run > the TAP tests for contrib. Unfortunately, the output is not in order > and is also not labelled, so it's hard to tell what output goes with > what contrib module. I named my test 001_basic.pl, but there are 12 of > those already. I see that 13 copies of 001_basic.pl seem to have > passed CI on Linux, so I guess the test ran and passed there. It seems > like it would be an awfully good idea to mention the subdirectory name > before each dump of output. Yea, the current output is *awful*. FWIW, the way it's hard to run tests the same way across platforms, the crappy output etc was one of the motivations behind the meson effort. If you just compare the output from both *nix and windows runs today with the meson output, it's imo night and day: https://cirrus-ci.com/task/5869668815601664?logs=check_world#L67 That's a recent run where I'd not properly mirrored 7c51b7f7cc0, leading to a failure on windows. Though it'd be more intersting to see a run with a failure. If one wants one can also see the test output of individual tests (it's always logged to a file). But I typically find that not useful for a 'general' test run, too much output. In that case there's a nice list of failed tests at the end: Summary of Failures: 144/219 postgresql:tap+vacuumlo / vacuumlo/t/001_basic.pl ERROR 0.48s (exit status 255 or signal 127 SIGinvalid) Ok: 218 Expected Fail: 0 Fail: 1 Unexpected Pass: 0 Skipped: 0 Timeout: 0 Greetings, Andres Freund
