Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-Jul-20, Andrew Dunstan wrote: > On 2023-07-20 Th 05:52, Alvaro Herrera wrote: > > On 2023-Jul-19, Andrew Dunstan wrote: > > > > > The result you report suggest to me that somehow the old version is no > > > longer a PostgreSQL::Version object.  Here's the patch I suggest: > > Ahh, okay,

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-07-20 Th 05:52, Alvaro Herrera wrote: On 2023-Jul-19, Andrew Dunstan wrote: The result you report suggest to me that somehow the old version is no longer a PostgreSQL::Version object.  Here's the patch I suggest: Ahh, okay, that makes more sense; and yes, it does work. Your patch

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-Jul-19, Andrew Dunstan wrote: > The result you report suggest to me that somehow the old version is no > longer a PostgreSQL::Version object.  Here's the patch I suggest: Ahh, okay, that makes more sense; and yes, it does work. -- Álvaro Herrera PostgreSQL Developer —

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-07-19 We 16:44, Andrew Dunstan wrote: On 2023-07-19 We 15:20, Andrew Dunstan wrote: On 2023-07-19 We 12:05, Alvaro Herrera wrote: Maybe we need to make AdjustUpgrade just look at the major version, something like:    $old_version =

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-07-19 We 15:20, Andrew Dunstan wrote: On 2023-07-19 We 12:05, Alvaro Herrera wrote: Maybe we need to make AdjustUpgrade just look at the major version, something like:    $old_version = PostgreSQL::Version->new($old_version->major); It seems like that does work, but if we do

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-07-19 We 12:05, Alvaro Herrera wrote: On 2023-Jul-19, Andrew Dunstan wrote: On 2023-07-19 We 07:05, Alvaro Herrera wrote: I just hit a snag testing this. It turns out that the PostgreSQL::Version comparison stuff believes that 16beta2 < 16, which sounds reasonable. However, because

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-Jul-19, Andrew Dunstan wrote: > > On 2023-07-19 We 07:05, Alvaro Herrera wrote: > > I just hit a snag testing this. It turns out that the > > PostgreSQL::Version comparison stuff believes that 16beta2 < 16, which > > sounds reasonable. However, because of that, the AdjustUpgrade.pm > >

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-07-19 We 07:05, Alvaro Herrera wrote: I just hit a snag testing this. It turns out that the PostgreSQL::Version comparison stuff believes that 16beta2 < 16, which sounds reasonable. However, because of that, the AdjustUpgrade.pm stanza that tries to drop tables

Re: Extracting cross-version-upgrade knowledge from buildfarm client

I just hit a snag testing this. It turns out that the PostgreSQL::Version comparison stuff believes that 16beta2 < 16, which sounds reasonable. However, because of that, the AdjustUpgrade.pm stanza that tries to drop tables public.gtest_normal_child{2} in versions earlier than 16 fails, because

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-18 We 10:33, Tom Lane wrote: > Andrew Dunstan writes: >> fairwren and drongo are clean except for fairywren upgrading 9.6 to 11. >> This appears to be a longstanding issue that the fuzz processing was >> causing us to ignore. See for example >>

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > See > > I tested it and it seems to be doing the right thing. Yeah, seems to do what I want. Thanks! regards, tom lane

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-18 We 17:14, Tom Lane wrote: > Andrew Dunstan writes: >> I think we can do what you want but it's a bit harder than what you've >> done. If we're not going to save the current run's product then we need >> to run the upgrade test from a different directory (probably directly in >>

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > I think we can do what you want but it's a bit harder than what you've > done. If we're not going to save the current run's product then we need > to run the upgrade test from a different directory (probably directly in > "$buildroot/$this_branch/inst"). Otherwise we'll

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-18 We 16:14, Tom Lane wrote: > Andrew Dunstan writes: >> On 2023-01-18 We 14:32, Tom Lane wrote: >>> I suppose that the reason for not running under $from_source is to >>> avoid corrupting the saved installations with unofficial versions. >>> However, couldn't we skip the "save" step

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > On 2023-01-18 We 14:32, Tom Lane wrote: >> I suppose that the reason for not running under $from_source is to >> avoid corrupting the saved installations with unofficial versions. >> However, couldn't we skip the "save" step and still run the upgrade >> tests against

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-18 We 14:32, Tom Lane wrote: > One more thing before we move on from this topic. I'd been testing > modified versions of the AdjustUpgrade.pm logic by building from a > --from-source source tree, which seemed way easier than dealing > with a private git repo. As it stands,

Re: Extracting cross-version-upgrade knowledge from buildfarm client

One more thing before we move on from this topic. I'd been testing modified versions of the AdjustUpgrade.pm logic by building from a --from-source source tree, which seemed way easier than dealing with a private git repo. As it stands, TestUpgradeXversion.pm refuses to run under $from_source,

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > fairwren and drongo are clean except for fairywren upgrading 9.6 to 11. > This appears to be a longstanding issue that the fuzz processing was > causing us to ignore. See for example >

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-17 Tu 11:30, Tom Lane wrote: > Andrew Dunstan writes: >> FYI crake has just passed the test with flying colours. > Cool. I await the Windows machines' results with interest. fairwren and drongo are clean except for fairywren upgrading 9.6 to 11. This appears to be a longstanding

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > On 2023-01-16 Mo 21:58, Tom Lane wrote: >> I dunno if we want to stretch buildfarm owners' patience with yet >> another BF client release right now. On the other hand, I'm antsy >> to see if we can un-revert 1b4d280ea after doing a little more >> work in

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > FYI crake has just passed the test with flying colours. Cool. I await the Windows machines' results with interest. regards, tom lane

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-17 Tu 10:18, Tom Lane wrote: > Andrew Dunstan writes: >> On 2023-01-16 Mo 21:58, Tom Lane wrote: >>> I dunno if we want to stretch buildfarm owners' patience with yet >>> another BF client release right now. On the other hand, I'm antsy >>> to see if we can un-revert 1b4d280ea after

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > On 2023-01-16 Mo 21:58, Tom Lane wrote: >> I dunno if we want to stretch buildfarm owners' patience with yet >> another BF client release right now. On the other hand, I'm antsy >> to see if we can un-revert 1b4d280ea after doing a little more >> work in

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-16 Mo 21:58, Tom Lane wrote: > I've pushed the per-branch AdjustUpgrade.pm files and tested by performing > a fresh round of buildfarm runs with the patched TestUpgradeXversion.pm > file. I think we're in good shape with this project. > > I dunno if we want to stretch buildfarm

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On Mon, Jan 16, 2023 at 04:48:28PM -0500, Tom Lane wrote: > I'm slightly tempted to back-patch 002_pg_upgrade.pl so that there > is an in-tree way to verify back-branch AdjustUpgrade.pm files. > On the other hand, it's hard to believe that testing that in > HEAD won't be sufficient; I doubt the

Re: Extracting cross-version-upgrade knowledge from buildfarm client

I've pushed the per-branch AdjustUpgrade.pm files and tested by performing a fresh round of buildfarm runs with the patched TestUpgradeXversion.pm file. I think we're in good shape with this project. I dunno if we want to stretch buildfarm owners' patience with yet another BF client release

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-16 Mo 18:11, Tom Lane wrote: > I wrote: >> I think we're about ready to go, except for cutting down >> AdjustUpgrade.pm to make versions to put in the back branches. > Hmmm ... so upon trying to test in the back branches, I soon > discovered that PostgreSQL/Version.pm isn't there

Re: Extracting cross-version-upgrade knowledge from buildfarm client

I wrote: > I think we're about ready to go, except for cutting down > AdjustUpgrade.pm to make versions to put in the back branches. Hmmm ... so upon trying to test in the back branches, I soon discovered that PostgreSQL/Version.pm isn't there before v15. I don't see a good reason why we

Re: Extracting cross-version-upgrade knowledge from buildfarm client

OK, here's a v4: * It works with 002_pg_upgrade.pl now. The only substantive change I had to make for that was to define the $old_version arguments as being always PostgreSQL::Version objects not strings, because otherwise I got complaints like Argument "HEAD" isn't numeric in numeric

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > On 2023-01-16 Mo 14:34, Tom Lane wrote: >> I don't think that's any easier to read, and it risks masking >> diffs that we'd wish to know about. > OK, I'll make another pass and tighten things up. Don't sweat it, I'm just working the bugs out of a new version. I'll have

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-16 Mo 14:34, Tom Lane wrote: > Andrew Dunstan writes: >> OK, here's my version. It tests clean against all of crake's dump files >> back to 9.2. >> To some extent it's a matter of taste, but I hate very long regex lines >> - it makes it very hard to see what's actually changing, so I

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > OK, here's my version. It tests clean against all of crake's dump files > back to 9.2. > To some extent it's a matter of taste, but I hate very long regex lines > - it makes it very hard to see what's actually changing, so I broke up > most of those. I don't mind

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-15 Su 18:37, Tom Lane wrote: > Andrew Dunstan writes: >> Those replacement lines are very difficult to read. I think use of >> extended regexes and some multi-part replacements would help. I'll have >> a go at that tomorrow. > Yeah, after I wrote that code I remembered about \Q ...

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On Sun, Jan 15, 2023 at 06:02:07PM -0500, Tom Lane wrote: > I guess, but it seems like make-work as long as there's just the one > column. Well, the query is already written, so I would use that, FWIW. > I did find that 002_pg_upgrade.pl could load it. I got stuck at > the point of trying to

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > Those replacement lines are very difficult to read. I think use of > extended regexes and some multi-part replacements would help. I'll have > a go at that tomorrow. Yeah, after I wrote that code I remembered about \Q ... \E, which would eliminate the need for most of

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-15 Su 11:01, Tom Lane wrote: > Another thing I was just thinking about was not bothering to run > "diff" if the fixed dump strings are equal in-memory. You could > take that even further and not write out the fixed files at all, > but that seems like a bad idea for debuggability of

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Michael Paquier writes: > On Sat, Jan 14, 2023 at 03:06:06PM -0500, Tom Lane wrote: > + # Can't upgrade aclitem in user tables from pre 16 to 16+. > + _add_st($result, 'regression', > + 'alter table public.tab_core_types drop column aclitem'); > Could you just use a DO

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On Sat, Jan 14, 2023 at 03:06:06PM -0500, Tom Lane wrote: > I tried to use this to replace upgrade_adapt.sql, but failed so > far because I couldn't figure out exactly how you're supposed > to use 002_pg_upgrade.pl with an old source installation. > It's not terribly well documented. As in

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > On 2023-01-14 Sa 15:06, Tom Lane wrote: >> Here's version 2, incorporating your suggestions and with some >> further work to make it handle 9.2 fully. > This looks pretty good to me. Great! I'll work on making back-branch versions. > I'll probably change this line >   

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-14 Sa 15:06, Tom Lane wrote: > I wrote: >> OK, I'll take a look at that and make a new draft. > Here's version 2, incorporating your suggestions and with some > further work to make it handle 9.2 fully. I think this could > be committable so far as HEAD is concerned, though I still >

Re: Extracting cross-version-upgrade knowledge from buildfarm client

I wrote: > OK, I'll take a look at that and make a new draft. Here's version 2, incorporating your suggestions and with some further work to make it handle 9.2 fully. I think this could be committable so far as HEAD is concerned, though I still need to make versions of AdjustUpgrade.pm for the

Re: Extracting cross-version-upgrade knowledge from buildfarm client

Andrew Dunstan writes: > On 2023-01-13 Fr 19:48, Tom Lane wrote: >> Attached are two patches, one for PG git and one for the buildfarm >> client, that create a working POC for this approach. > OK, we've been on parallel tracks (sorry about that). Let's run with > yours, as it covers more ground.

Re: Extracting cross-version-upgrade knowledge from buildfarm client

On 2023-01-13 Fr 19:48, Tom Lane wrote: > This is a followup to the discussion at [1], in which we agreed that > it's time to fix the buildfarm client so that knowledge about > cross-version discrepancies in pg_dump output can be moved into > the community git tree, making it feasible for people

Extracting cross-version-upgrade knowledge from buildfarm client

This is a followup to the discussion at [1], in which we agreed that it's time to fix the buildfarm client so that knowledge about cross-version discrepancies in pg_dump output can be moved into the community git tree, making it feasible for people other than Andrew to fix problems when we change