Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-15 Thread Andrew Dunstan
On 2/14/22 17:37, Andres Freund wrote: > On 2022-02-14 17:32:11 -0500, Andrew Dunstan wrote: >> Working on that. There appear to be some issues with third party >> libraries. I might need to rebuild libxml2 and zlib for example. > Any reason not to use the ones from msys2? That seems to work.

Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-15 Thread Andrew Dunstan
On 2/14/22 18:02, Andres Freund wrote: > Hi, > > On 2022-02-14 17:32:11 -0500, Andrew Dunstan wrote: >> About the only thing missing in your recipe is this: > Re requiring out-of-tree builds: Thomas on IM noted that there's the > NoDefaultCurrentDirectoryInExePath environment variable. That

Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-14 Thread Andres Freund
Hi, On 2022-02-14 17:32:11 -0500, Andrew Dunstan wrote: > About the only thing missing in your recipe is this: Re requiring out-of-tree builds: Thomas on IM noted that there's the NoDefaultCurrentDirectoryInExePath environment variable. That should avoid the problem leading to requiring

Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-14 Thread Andres Freund
On 2022-02-14 17:32:11 -0500, Andrew Dunstan wrote: > Working on that. There appear to be some issues with third party > libraries. I might need to rebuild libxml2 and zlib for example. Any reason not to use the ones from msys2?

Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-14 Thread Andrew Dunstan
On 2/3/22 20:51, Andres Freund wrote: > Hi, > > On 2022-02-03 17:25:51 -0500, Andrew Dunstan wrote: >> OK, I have all the pieces working and I know what I need to do to adapt >> fairywren. The patch you provided is not necessary any more. > Cool. Are you going to post that? About the only

Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-03 Thread Andres Freund
Hi, On 2022-02-03 17:25:51 -0500, Andrew Dunstan wrote: > OK, I have all the pieces working and I know what I need to do to adapt > fairywren. The patch you provided is not necessary any more. Cool. Are you going to post that? > (I think your TMPDIR spec is missing a /build/) I think I went

Re: fairywren is generating bogus BASE_BACKUP commands

2022-02-03 Thread Andrew Dunstan
On 1/24/22 21:36, Andres Freund wrote: > Hi, > > On 2022-01-24 16:47:28 -0500, Andrew Dunstan wrote: >> Give me what you can and I'll see what I can do. I have a couple of >> moderately high priority items on my plate, but I will probably be able >> to fit in some testing when those make my eyes

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Andres Freund
Hi, On 2022-01-24 16:47:28 -0500, Andrew Dunstan wrote: > Give me what you can and I'll see what I can do. I have a couple of > moderately high priority items on my plate, but I will probably be able > to fit in some testing when those make my eyes completely glaze over. Steps: # install msys

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Andrew Dunstan
On 1/24/22 16:39, Andres Freund wrote: > Hi, > > On 2022-01-24 14:01:37 -0500, Andrew Dunstan wrote: >> Well if we can get Andres' suggestion to work all this might go away, >> which would keep everyone happy, especially me. > I successfully tried it for a few tests. But I see tests hanging a

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Andres Freund
Hi, On 2022-01-24 14:01:37 -0500, Andrew Dunstan wrote: > Well if we can get Andres' suggestion to work all this might go away, > which would keep everyone happy, especially me. I successfully tried it for a few tests. But I see tests hanging a lot independent of the way I run the tests,

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Andrew Dunstan
On 1/24/22 15:17, Robert Haas wrote: > On Mon, Jan 24, 2022 at 2:27 PM Robert Haas wrote: >> I really hate committing stuff that turns out to be broken. It's such >> a fire drill when the build farm turns red. > And there's a good chance it's about to break again, because I just > committed the

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 2:27 PM Robert Haas wrote: > I really hate committing stuff that turns out to be broken. It's such > a fire drill when the build farm turns red. And there's a good chance it's about to break again, because I just committed the next patch in the series which, shockingly,

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 2:01 PM Andrew Dunstan wrote: > Well if we can get Andres' suggestion to work all this might go away, > which would keep everyone happy, especially me. You're right that I was > a little careless upthread. Mea culpa. Meanwhile I am committing a > minimal one line fix. I

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Andrew Dunstan
On 1/23/22 22:52, Robert Haas wrote: > On Sun, Jan 23, 2022 at 4:09 PM Andrew Dunstan wrote: >> The most common issues we get are around this issue of virtualized paths >> in the TAP tests. If people followed the rule I suggested upthread, 99% >> of those problems would go away. > Well, that

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Robert Haas
On Sun, Jan 23, 2022 at 4:09 PM Andrew Dunstan wrote: > The most common issues we get are around this issue of virtualized paths > in the TAP tests. If people followed the rule I suggested upthread, 99% > of those problems would go away. Well, that makes it sound like it's the fault of people

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Andres Freund
Hi, On 2022-01-23 17:38:26 -0500, Andrew Dunstan wrote: > Nice idea. I have a suspicion that it's going to be harder than you > think, but I'll be very happy to be proved wrong. FWIW, a manual invocation of the pg_basebackup tests works via a ucrt perl (ucrt64/mingw-w64-ucrt-x86_64-perl

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Andrew Dunstan
On 1/23/22 16:31, Andres Freund wrote: >> The most common issues we get are around this issue of virtualized paths >> in the TAP tests. If people followed the rule I suggested upthread, 99% >> of those problems would go away. I realize it's annoying - I've been >> caught by it myself on more

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Andres Freund
Hi, On 2022-01-23 16:09:01 -0500, Andrew Dunstan wrote: > Msys is a unix-like environment that is useful to build Postgres. It's > not intended as a general runtime environment. We therefore don't build > msys-aware Postgres. We use msys to build standalone Postgres binaries > that don't need or

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Tom Lane
Andrew Dunstan writes: > The most common issues we get are around this issue of virtualized paths > in the TAP tests. If people followed the rule I suggested upthread, 99% > of those problems would go away. I realize it's annoying - I've been > caught by it myself on more than one occasion. Maybe

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Andrew Dunstan
On 1/23/22 15:07, Tom Lane wrote: > Robert Haas writes: >> Maybe we need to have a README in the tree somewhere that tries to >> explain this. Or maybe we should make our build artifacts msys-aware, >> if that's possible, so that this just works. Or maybe supporting msys >> is not worth the

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Tom Lane
Robert Haas writes: > Maybe we need to have a README in the tree somewhere that tries to > explain this. Or maybe we should make our build artifacts msys-aware, > if that's possible, so that this just works. Or maybe supporting msys > is not worth the trouble. I've been wondering that last

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Robert Haas
On Sun, Jan 23, 2022 at 12:20 PM Andrew Dunstan wrote: > It's not as simple as that :-( But you're on the right track. My > suggestion above doesn't work. > > The rule for paths is: when you're passing a path to an external program > that's not msys aware (typically, one of our build artefacts

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-23 Thread Andrew Dunstan
On 1/21/22 22:43, Thomas Munro wrote: > On Sat, Jan 22, 2022 at 3:55 PM Robert Haas wrote: >> On Fri, Jan 21, 2022 at 5:35 PM Andrew Dunstan wrote: >>> # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces >>> local $ENV{MSYS2_ARG_CONV_EXCL} = $source_ts_prefix; >>> Probably

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-22 Thread Andrew Dunstan
On 1/21/22 18:04, Andres Freund wrote: > Hi, > > On 2022-01-21 17:42:45 -0500, Tom Lane wrote: >> Andrew Dunstan writes: >>> c.f. src/bin/pg_verifybackup/t/003_corruption.pl which says: >>>     my $source_ts_prefix = $source_ts_path; >>>     $source_ts_prefix =~ s!(^[A-Z]:/[^/]*)/.*!$1!; >>>    

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Thomas Munro
On Sat, Jan 22, 2022 at 3:55 PM Robert Haas wrote: > On Fri, Jan 21, 2022 at 5:35 PM Andrew Dunstan wrote: > > # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces > > local $ENV{MSYS2_ARG_CONV_EXCL} = $source_ts_prefix; > > Probably in this case just setting it to 'server:'

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 5:35 PM Andrew Dunstan wrote: > # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces > local $ENV{MSYS2_ARG_CONV_EXCL} = $source_ts_prefix; > Probably in this case just setting it to 'server:' would do the trick. Oh, thanks for the tip. Do you want to

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Andres Freund
Hi, On 2022-01-21 17:26:32 -0500, Robert Haas wrote: > I think the syntax has been accepted since pg_basebackup was added in 2011, > and Andres added it to this test case earlier this week (with -cfast in the > subject line of the commit message). The reason I used -cfast instead of -c fast or

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 5:42 PM Tom Lane wrote: > The point I was trying to make is that if we have to jump through > that sort of hoop in the test scripts, then real users are going > to have to jump through it as well, and they won't like that > (and we will get bug reports about it). It'd be

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Andres Freund
Hi, On 2022-01-21 17:42:45 -0500, Tom Lane wrote: > Andrew Dunstan writes: > > c.f. src/bin/pg_verifybackup/t/003_corruption.pl which says: > >     my $source_ts_prefix = $source_ts_path; > >     $source_ts_prefix =~ s!(^[A-Z]:/[^/]*)/.*!$1!; > >     ... > > >     # See

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Tom Lane
Andrew Dunstan writes: > c.f. src/bin/pg_verifybackup/t/003_corruption.pl which says: >     my $source_ts_prefix = $source_ts_path; >     $source_ts_prefix =~ s!(^[A-Z]:/[^/]*)/.*!$1!; >     ... >     # See https://www.msys2.org/wiki/Porting/#filesystem-namespaces >     local

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Tom Lane
Robert Haas writes: > On Fri, Jan 21, 2022 at 5:09 PM Tom Lane wrote: >> While we're on the subject of ill-chosen option syntax: "-cfast" >> with non double dashes? Really? That's horribly ambiguous. > I'm not sure whether you're complaining that we accept that syntax or > using it, but AFAIK

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Andrew Dunstan
On 1/21/22 17:10, Thomas Munro wrote: > On Sat, Jan 22, 2022 at 10:42 AM Robert Haas wrote: >> # Running: pg_basebackup --no-sync -cfast --target >> server:/home/pgrunner/bf/root/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/tmp_test_Ag8r/backuponserver >> -X none >> pg_basebackup: error:

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
On Fri, Jan 21, 2022 at 5:09 PM Tom Lane wrote: > I think the backup_target string was already corrupted that way when > pg_basebackup absorbed it from optarg. It's pretty hard to believe that > the strchr/pnstrdup stanza got it wrong. However, comparing the > TARGET_DETAIL to what the TAP test

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Thomas Munro
On Sat, Jan 22, 2022 at 10:42 AM Robert Haas wrote: > # Running: pg_basebackup --no-sync -cfast --target > server:/home/pgrunner/bf/root/HEAD/pgsql.build/src/bin/pg_basebackup/tmp_check/tmp_test_Ag8r/backuponserver > -X none > pg_basebackup: error: could not initiate base backup: ERROR: >

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Tom Lane
Robert Haas writes: > "server" is a valid backup target, but "server;C" is not. And I think > this must be a bug on the client side, because the server logs the > generated query: > 2022-01-21 20:53:11.618 UTC [8404:10] 010_pg_basebackup.pl LOG: > received replication command: BASE_BACKUP (

fairywren is generating bogus BASE_BACKUP commands

2022-01-21 Thread Robert Haas
Thomas Munro pointed out this failure to me on fairywren: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren=2022-01-21%2020%3A10%3A22 He theorizes that I need some perl2host magic in there, which may well be true. But I also noticed this: # Running: pg_basebackup --no-sync