Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
Andres Freund writes: > Ah, and that's finally also the explanation why I couldn't reproduce the > failure it in a different directory, with an otherwise identically configured > PG: The length of the path to the tablespace influences the size of the > XLOG_TBLSPC_CREATE record. Ohhh ...

Re: Failed transaction statistics to measure the logical replication progress

2022-03-24 Thread Masahiko Sawada
On Thu, Mar 24, 2022 at 12:30 PM Amit Kapila wrote: > > On Thu, Mar 3, 2022 at 8:58 AM osumi.takami...@fujitsu.com > wrote: > > > > This patch introduces two new subscription statistics columns > (apply_commit_count and apply_rollback_count) to the > pg_stat_subscription_stats view for counting

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-25 01:23:00 -0400, Tom Lane wrote: > Andres Freund writes: > > I do see that the LSN that ends up on the page is the same across a few runs > > of the test on serinus. Which presumably differs between different > > animals. Surprised that it's this predictable - but I guess the

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 21:54:38 -0700, Andres Freund wrote: > I do see that the LSN that ends up on the page is the same across a few runs > of the test on serinus. Which presumably differs between different > animals. Surprised that it's this predictable - but I guess the run is short > enough that

Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
Andres Freund writes: > I do see that the LSN that ends up on the page is the same across a few runs > of the test on serinus. Which presumably differs between different > animals. Surprised that it's this predictable - but I guess the run is short > enough that there's no variation due to

RE: Logical replication timeout problem

2022-03-24 Thread wangw.f...@fujitsu.com
On Thur, Mar 24, 2022 at 6:32 PM Amit Kapila wrote: > Thanks for your kindly update. > It seems by mistake you have removed the changes from pgoutput_message > and pgoutput_truncate functions. I have added those back. > Additionally, I made a few other changes: (a) moved the function >

Re: shared-memory based stats collector - v66

2022-03-24 Thread Kyotaro Horiguchi
At Thu, 24 Mar 2022 13:21:33 -0400, Melanie Plageman wrote in > On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > > > > The biggest todos are: > > - Address all the remaining AFIXMEs and XXXs > > Attached is a patch that addresses three of the existing AFIXMEs. Thanks! +

Re: Add LZ4 compression in pg_dump

2022-03-24 Thread Greg Stark
It seems development on this has stalled. If there's no further work happening I guess I'll mark the patch returned with feedback. Feel free to resubmit it to the next CF when there's progress.

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Michael Paquier
On Fri, Mar 25, 2022 at 11:11:46AM +0800, Julien Rouhaud wrote: > As an example, here's a POC for a new shmem_request_hook hook after > _PG_init(). > With it I could easily fix pg_wait_sampling shmem allocation (and checked that > it's indeed requesting the correct size). Are you sure that the

Re: CREATE INDEX CONCURRENTLY on partitioned index

2022-03-24 Thread Greg Stark
This patch is marked "waiting on author" in the CF. However the most recent emails have patches and it's not clear to me what's left from previous reviews that might not be addressed yet. Should this patch be marked "Needs Review"? Anastasia and Alexander are marked as reviewers. Are you still

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Michael Paquier
On Thu, Mar 24, 2022 at 08:54:14PM -0700, Peter Geoghegan wrote: > amcheck's palloc_btree_page() function validates that an 8KiB page is > in fact an nbtree page, in a maximally paranoid way. Might be an > example worth following here. Sure, we could do that. However, I don't think that going

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-25 00:08:20 -0400, Tom Lane wrote: > Andres Freund writes: > > The only thing I can really conclude here is that we apparently end up with > > the same checksum for exactly the modifications we are doing? Just on those > > two damn instances? Reliably? > > IIRC, the table's OID

Re: Is there any documentation on how to correctly create extensions in HA(primary-standby) setup?

2022-03-24 Thread Greg Stark
This doesn't seem to be getting any further attention. It sounds like Julien didn't agree with the scope of the text. Bharath do you think Julien's comments make sense? Will you have a chance to look at this?

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2022-03-24 Thread Greg Stark
This patch seems to have gotten some feedback but development has stalled. It's marked "waiting on author" but I'm not clear exactly what is expected from the authors here. It seems there isn't really consensus on the design at the moment. There's been no emails in over a month. Fwiw I find the

Re: Add psql command to list constraints

2022-03-24 Thread Greg Stark
Development of this seems to have stalled with the only review of this patch expressing some skepticism about whether it's needed at all. Unless anyone steps forward and says it's interesting I'm going to mark it rejected? I don't actually think it's a terrible idea myself but I think every use

Re: A test for replay of regression tests

2022-03-24 Thread Peter Geoghegan
On Thu, Mar 24, 2022 at 9:16 PM Andres Freund wrote: > > VACUUM FREEZE (without DISABLE_PAGE_SKIPPING) seems like it would do > > everything you want, without using a temp table. At least on the > > master branch. > > We tried that in a prior case: >

Re: A test for replay of regression tests

2022-03-24 Thread Thomas Munro
On Fri, Mar 25, 2022 at 5:16 PM Andres Freund wrote: > On 2022-03-24 21:06:21 -0700, Peter Geoghegan wrote: > > VACUUM FREEZE (without DISABLE_PAGE_SKIPPING) seems like it would do > > everything you want, without using a temp table. At least on the > > master branch. > > We tried that in a prior

Re: Add header support to text format and matching feature

2022-03-24 Thread Greg Stark
Great to see the first of the two patches committed. It looks like the second patch got some feedback from Peter in February and has been marked "Waiting on author" ever since. Remi, will you have a chance to look at this this month? Peter, are these comments blocking if Remi doesn't have a

Re: Schema variables - new implementation for Postgres 15

2022-03-24 Thread Julien Rouhaud
Hi, On Wed, Mar 23, 2022 at 09:58:59PM +0100, Pavel Stehule wrote: > > A bit more work seems to be needed for deparsing session variables: > > > > # create variable myvar text; > > CREATE VARIABLE > > > > # create view myview as select myvar; > > CREATE VIEW > > > > # \d+ myview > >

Re: A test for replay of regression tests

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 21:06:21 -0700, Peter Geoghegan wrote: > On Thu, Mar 24, 2022 at 8:56 PM Thomas Munro wrote: > > Interesting. IIUC your chaos gizmo shows that particular vacuum test > > still failing on master, but that wouldn't happen in real life because > > since 383f2221 it's a temp

Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
Andres Freund writes: > The only thing I can really conclude here is that we apparently end up with > the same checksum for exactly the modifications we are doing? Just on those > two damn instances? Reliably? IIRC, the table's OID or relfilenode enters into the checksum. Could it be that

Re: A test for replay of regression tests

2022-03-24 Thread Peter Geoghegan
On Thu, Mar 24, 2022 at 8:56 PM Thomas Munro wrote: > Interesting. IIUC your chaos gizmo shows that particular vacuum test > still failing on master, but that wouldn't happen in real life because > since 383f2221 it's a temp table. Your gizmo should probably detect > temp rels, as your comment

Re: logical decoding and replication of sequences

2022-03-24 Thread Amit Kapila
On Fri, Mar 25, 2022 at 3:29 AM Tomas Vondra wrote: > > Pushed. > Some of the comments given by me [1] don't seem to be addressed or responded to. Let me try to say again for the ease of discussion: * Don't we need some syncing mechanism between apply worker and sequence sync worker so that

Re: A test for replay of regression tests

2022-03-24 Thread Thomas Munro
On Fri, Mar 25, 2022 at 4:03 PM Peter Geoghegan wrote: > If you want to know whether or not the buildfarm will have problems > due to VACUUM failing to get a cleanup lock randomly, then I suggest > that you use an approach like the one from my patch here: > >

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Peter Geoghegan
On Wed, Mar 23, 2022 at 1:16 AM Michael Paquier wrote: > As far as I can see, this is > also possible in bt_page_items_bytea(), gist_page_opaque_info(), > gin_metapage_info() and gin_page_opaque_info(). All those code paths > should be protected with some checks on PageGetSpecialSize(), I >

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 19:43:02 -0700, Andres Freund wrote: > Just to be sure I'm going to clean out serinus' ccache dir and rerun. I'll > leave dragonet's alone for now. Turns out they had the same dir. But it didn't help. I haven't yet figured out why, but I now *am* able to reproduce the problem

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Michael Paquier
On Fri, Mar 25, 2022 at 11:03:47AM +0800, Julien Rouhaud wrote: > I'm happy with all the changes, except: > > + if (P_ISLEAF(opaque) && opaque->btpo_level != 0) > + ereport(ERROR, > + > (errcode(ERRCODE_INVALID_PARAMETER_VALUE),

Re: Skip partition tuple routing with constant partition key

2022-03-24 Thread Amit Langote
On Thu, Mar 24, 2022 at 1:55 AM Zhihong Yu wrote: > On Wed, Mar 23, 2022 at 5:52 AM Amit Langote wrote: >> I've attached an updated version of the patch, though I haven't >> changed the threshold constant. > + * Threshold of the number of tuples to need to have been processed before > + *

RE: Logical replication timeout problem

2022-03-24 Thread kuroda.hay...@fujitsu.com
Dear Amit, > It seems by mistake you have removed the changes from pgoutput_message > and pgoutput_truncate functions. I have added those back. > Additionally, I made a few other changes: (a) moved the function > UpdateProgress to pgoutput.c as it is not used outside it, (b) change > the new

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Julien Rouhaud
On Fri, Mar 25, 2022 at 10:39:51AM +0800, Julien Rouhaud wrote: > On Thu, Mar 24, 2022 at 04:27:36PM -0400, Robert Haas wrote: > > On Thu, Mar 24, 2022 at 4:20 PM Nathan Bossart > > wrote: > > > Another possibility could be to add a hook that is called _before_ > > > _PG_init() where libraries

Re: Column Filtering in Logical Replication

2022-03-24 Thread Amit Kapila
On Fri, Mar 25, 2022 at 5:44 AM Tomas Vondra wrote: > > Attached is a patch, rebased on top of the sequence decoding stuff I > pushed earlier today, also including the comments rewording, and > renaming the "transform" function. > > I'll go over it again and get it pushed soon, unless someone

Re: XID formatting and SLRU refactorings

2022-03-24 Thread Kyotaro Horiguchi
At Fri, 25 Mar 2022 00:02:55 +0400, Pavel Borisov wrote in > Hi! > It seems that CFbot was still unhappy with pg_upgrade test due to epoch > removal from NextXid in controldata. > I've reverted this change as support for "epochless" 64-bit control data > with xids that haven't yet switched to

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Julien Rouhaud
On Fri, Mar 25, 2022 at 11:44:26AM +0900, Michael Paquier wrote: > I have reviewed what you have sent, bumping on a couple of issues: Thanks! I'm happy with all the changes, except: + if (P_ISLEAF(opaque) && opaque->btpo_level != 0) + ereport(ERROR, +

Re: Corruption during WAL replay

2022-03-24 Thread Thomas Munro
On Fri, Mar 25, 2022 at 3:35 PM Andres Freund wrote: > So I'm not sure how much can be gleaned from raw "failure counts" without > taking the number of runs into account as well? Ah, right, it does indeed hold the record for most runs in 3 months, and taking runs into account its "stats" failure

Re: A test for replay of regression tests

2022-03-24 Thread Peter Geoghegan
On Mon, Mar 21, 2022 at 8:59 PM Thomas Munro wrote: > Thanks. Ahh, déjà vu... this probably needs the same treatment as > b700f96c and 3414099c provided for the reloptions test. Well, at > least the first one. Here's a patch like that. If you want to know whether or not the buildfarm will

Re: Fix typo in standby.c

2022-03-24 Thread Michael Paquier
On Fri, Mar 25, 2022 at 02:34:22AM +, houzj.f...@fujitsu.com wrote: > I noticed a possible typo in standby.c: > > --- > * The definitions of RunningTransactionsData and xl_xact_running_xacts are > * similar. We keep them separate because xl_xact_running_xacts is a > --- > > It seems

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Michael Paquier
On Thu, Mar 24, 2022 at 04:27:40PM +0800, Julien Rouhaud wrote: > I worked on a patch to fix the problem. The functions you mention were indeed > missing some check, but after digging a bit more I found that other problem > existed. For instance, feeding a btree page to a gist_page_items_bytea()

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 19:20:10 -0700, Andres Freund wrote: > I forced a run while writing the other email, with keep_error_whatnot, and I > just saw it failing... Looking whether there's anything interesting to glean. Unfortunately the test drops the table and it doesn't report the filepath of the

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Julien Rouhaud
On Thu, Mar 24, 2022 at 04:27:36PM -0400, Robert Haas wrote: > On Thu, Mar 24, 2022 at 4:20 PM Nathan Bossart > wrote: > > Another possibility could be to add a hook that is called _before_ > > _PG_init() where libraries are permitted to adjust GUCs. After the library > > is loaded, we first

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-25 15:23:24 +1300, Thomas Munro wrote: > One random thing I've noticed about serinus is that it seems to drop > UDP packets more than others, but dragonet apparently doesn't: Serinus is built with optimization. Which I guess could lead to other backends reporting stats more

Fix typo in standby.c

2022-03-24 Thread houzj.f...@fujitsu.com
Hi, I noticed a possible typo in standby.c: --- * The definitions of RunningTransactionsData and xl_xact_running_xacts are * similar. We keep them separate because xl_xact_running_xacts is a --- It seems "xl_xact_running_xacts" should be "xl_running_xacts". Best regards, Hou zhijie

Re: Corruption during WAL replay

2022-03-24 Thread Thomas Munro
On Fri, Mar 25, 2022 at 3:14 PM Andres Freund wrote: > On 2022-03-24 21:22:38 -0400, Tom Lane wrote: > > serinus is 0-for-3 since then, and dragonet 0-for-4, so we can be pretty > > confident that the failure is repeatable for them. > > That's weird. They run on the same host, but otherwise they

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 21:59:08 -0400, Tom Lane wrote: > Another thing that seems quite baffling, but is becoming clearer by > the hour, is that only serinus and dragonet are seeing this failure. > How is that? They're not very similarly configured --- one is gcc, > one clang, and one uses jit and

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 21:22:38 -0400, Tom Lane wrote: > serinus is 0-for-3 since then, and dragonet 0-for-4, so we can be pretty > confident that the failure is repeatable for them. That's weird. They run on the same host, but otherwise they have very little in common. There's plenty other animals

Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
I wrote: > ... So that leaves 7dac61402e, which did this to > the test script that's failing: > use strict; > use warnings; > -use Config; > use PostgreSQL::Test::Cluster; > use PostgreSQL::Test::Utils; > Discuss. Another thing that seems quite baffling, but is becoming clearer by the

Re: Remove an unnecessary errmsg_plural in dependency.c

2022-03-24 Thread Kyotaro Horiguchi
At Thu, 24 Mar 2022 16:00:58 +0100, Alvaro Herrera wrote in > On 2022-Mar-24, Tom Lane wrote: > > > Alvaro Herrera writes: > > > $ git grep 'Plural-Forms' src/backend/po/*.po > > > ru.po:"Plural-Forms: nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : > > > n%10>=2 && n" > > > > Oh,

Re: Remove an unnecessary errmsg_plural in dependency.c

2022-03-24 Thread Kyotaro Horiguchi
At Thu, 24 Mar 2022 10:19:18 -0400, Tom Lane wrote in > Bharath Rupireddy writes: > > Thanks. I think I get the point - is it dngettext doing things > > differently for different languages? > > Yeah. To be concrete, have a look in ru.po: I wondered why it takes two forms of format string but

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread Japin Li
On Fri, 25 Mar 2022 at 08:55, Thomas Munro wrote: > On Fri, Mar 25, 2022 at 1:43 PM Japin Li wrote: >> + printf(_(" -x, --xid=XID only show records with >> transaction ID XID\n")); >> >> I think the description of transaction ID is enough, IIUC, XID is use in >> core, >>

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Tatsuo Ishii
>> Oops. Thanks. New patch attached. Test has passed on my machine. > > This patch works for me. I think it is ok to use \N instead of \gN. Thanks. Patch pushed. Best reagards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Tatsuo Ishii
> I reproduced the failure on another machine with perl 5.8.8, > and I can confirm that this patch fixes it. Thank you for the test. I have pushed the patch. Best reagards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp

Re: Corruption during WAL replay

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 20:39:27 -0400, Robert Haas wrote: > But that leaves me even more confused. How can a change to only the server > code cause a client utility to fail to detect corruption that is being > created by Perl while the server is stopped? I guess it could somehow cause the first page

Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
Robert Haas writes: > I hate to say "no" because the evidence suggests that the answer might > be "yes" -- but it definitely isn't intending to change anything about > the shutdown sequence. It just introduces a mechanism to backends to > force the checkpointer to delay writing the checkpoint

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Yugo NAGATA
On Fri, 25 Mar 2022 09:14:00 +0900 (JST) Tatsuo Ishii wrote: > > Note that the \\g2 just above also needs to be changed. > > Oops. Thanks. New patch attached. Test has passed on my machine. This patch works for me. I think it is ok to use \N instead of \gN. Regards, Yugo Nagata -- Yugo

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread David G. Johnston
On Thu, Mar 24, 2022 at 5:40 PM Tom Lane wrote: > "David G. Johnston" writes: > > The extension object type does not seem to have gotten the > > --exclude-extension capability that it would need to conform to the > general > > design exemplified by --table and hopefully extended out to the

Re: Corruption during WAL replay

2022-03-24 Thread Robert Haas
On Thu, Mar 24, 2022 at 8:45 PM Tom Lane wrote: > Hmm, I'd supposed that the failing test cases were new as of 412ad7a55. > Now I see they're not, which indeed puts quite a different spin on > things. Your thought about maybe the server isn't shut down yet is > interesting --- did 412ad7a55

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Tom Lane
Tatsuo Ishii writes: > Oops. Thanks. New patch attached. Test has passed on my machine. I reproduced the failure on another machine with perl 5.8.8, and I can confirm that this patch fixes it. regards, tom lane

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread Thomas Munro
On Fri, Mar 25, 2022 at 1:43 PM Japin Li wrote: > + printf(_(" -x, --xid=XID only show records with transaction > ID XID\n")); > > I think the description of transaction ID is enough, IIUC, XID is use in core, > which means transaction ID. The mention of "XID" corresponds to XID

Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
Robert Haas writes: > And ... right after hitting send, I see that the recovery check > failures are under separate troubleshooting and thus probably > unrelated. Yeah, we've been chasing those for months. > But that leaves me even more confused. How can a change to > only the server code cause

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread Japin Li
On Fri, 25 Mar 2022 at 05:11, Thomas Munro wrote: > Cool. > > I had another thought while changing that (and also re-alphabetising): > Why don't we switch to -B for --block and -R for --relation? I > gather you used -k and -l because -b and -r were already taken, but > since we already

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread Tom Lane
"David G. Johnston" writes: > The extension object type does not seem to have gotten the > --exclude-extension capability that it would need to conform to the general > design exemplified by --table and hopefully extended out to the routine > object types. We're not going to instantly build out

Re: Corruption during WAL replay

2022-03-24 Thread Robert Haas
On Thu, Mar 24, 2022 at 8:37 PM Robert Haas wrote: > Any ideas? And ... right after hitting send, I see that the recovery check failures are under separate troubleshooting and thus probably unrelated. But that leaves me even more confused. How can a change to only the server code cause a client

Re: Corruption during WAL replay

2022-03-24 Thread Robert Haas
On Thu, Mar 24, 2022 at 6:04 PM Tom Lane wrote: > Robert Haas writes: > > Thanks, committed. > > Some of the buildfarm is seeing failures in the pg_checksums test. Hmm. So the tests seem to be failing because 002_actions.pl stops the database cluster, runs pg_checksums (which passes), writes

RE: logical replication empty transactions

2022-03-24 Thread houzj.f...@fujitsu.com
On Thursday, March 24, 2022 11:19 AM houzj.f...@fujitsu.com wrote: > On Tuesday, March 22, 2022 7:50 PM Amit Kapila > wrote: > > On Tue, Mar 22, 2022 at 7:25 AM houzj.f...@fujitsu.com > > wrote: > > > > > > > On Monday, March 21, 2022 6:01 PM Amit Kapila > > > > > > > > wrote: > > > > > > Oh,

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread David G. Johnston
On Mon, Jan 24, 2022 at 10:49 PM Michael Paquier wrote: > What about patterns? Switches like --table or > --extension are able to digest a psql-like pattern to decide which > objects to dump. > The extension object type does not seem to have gotten the --exclude-extension capability that it

Re: Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread David G. Johnston
On Thu, Mar 24, 2022 at 4:42 PM Chapman Flack wrote: > On 03/27/21 08:57, Andrew Dunstan wrote: > > We can bikeshed the name of the flag at some stage. --procedures-only > > might also make sense > > Any takers for --routines-only ? > > "Routine" is the genuine, ISO SQL umbrella term for a

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Tatsuo Ishii
> Note that the \\g2 just above also needs to be changed. Oops. Thanks. New patch attached. Test has passed on my machine. Best reagards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp diff --git

Re: SQL/JSON: functions

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 18:51:30 -0400, Andrew Dunstan wrote: > I wonder if we should add these compile flags to the cfbot's setup? Yes, I think we should. There's a bit of discussion of that in and below https://postgr.es/m/20220213051937.GO31460%40telsasoft.com - that veered a bit of course, so I

Re: Race conditions in 019_replslot_limit.pl

2022-03-24 Thread Tom Lane
Andres Freund writes: > How about the attached variation, which retries (for 15s, with 100ms sleeps) > if there are multiple walsenders, printing the debugging info each time? It'll > still fail the test if later iterations find just one walsender, which seems > the right behaviour for now. OK

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Tom Lane
Tatsuo Ishii writes: > I don't see a reason to use "\gN" either. Actually after applying > attached patch, my machine is still happy with pgbench test. Note that the \\g2 just above also needs to be changed. regards, tom lane

Re: Race conditions in 019_replslot_limit.pl

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 15:05:40 -0400, Tom Lane wrote: > I just noticed something very interesting: in a couple of recent > buildfarm runs with this failure, the pg_stat_activity printout > no longer shows the extra walsender: Oh. That is interesting. Thanks for catching that. > What this suggests

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-24 Thread Tatsuo Ishii
>> My machine (Ubuntu 20) did not complain either. Maybe perl version >> difference? Any way, the fix pushed. Let's see how prairiedog feels. > > Still not happy. After some digging in man pages, I believe the > problem is that its old version of Perl does not understand "\gN" > backreferences.

Re: Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread Chapman Flack
On 03/27/21 08:57, Andrew Dunstan wrote: > We can bikeshed the name of the flag at some stage. --procedures-only > might also make sense Any takers for --routines-only ? "Routine" is the genuine, ISO SQL umbrella term for a function or procedure, and we have used it that way in our docs and

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread Tom Lane
Daniel Gustafsson writes: >> On 24 Mar 2022, at 23:38, Greg Stark wrote: >> It looks like this discussion has reached a bit of an impasse with Tom >> being against this approach and Michael and Daniel being for it. It >> doesn't look like it's going to get committed this commitfest, shall >> we

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread Daniel Gustafsson
> On 24 Mar 2022, at 23:38, Greg Stark wrote: > It looks like this discussion has reached a bit of an impasse with Tom > being against this approach and Michael and Daniel being for it. It > doesn't look like it's going to get committed this commitfest, shall > we move it forward or mark it

Re: Add non-blocking version of PQcancel

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 17:41:53 -0400, Tom Lane wrote: > I kind of feel that this patch is going in the wrong direction. > I do see the need for a version of PQcancel that can encrypt the > transmitted cancel request (and yes, that should work on the backend > side; see recursion in

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread David Christensen
> On Mar 24, 2022, at 4:12 PM, Thomas Munro wrote: > > On Fri, Mar 25, 2022 at 1:43 AM David Christensen > wrote: On Mar 24, 2022, at 6:43 AM, Thomas Munro wrote: >>> On Fri, Mar 25, 2022 at 12:26 AM Thomas Munro >>> wrote: > On Fri, Mar 25, 2022 at 12:01 AM Peter Eisentraut

Re: [PATCH] Add native windows on arm64 support

2022-03-24 Thread Thomas Munro
On Thu, Mar 24, 2022 at 2:15 PM Andres Freund wrote: > I think we've generally seen the ASLR issue become less prominent on > windows. Whether that's because of the silent retries we added, or because > just about everyone moved to 64bit windows / PG, I don't know. I'd guess both, > with 64bit

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread Greg Stark
It looks like this discussion has reached a bit of an impasse with Tom being against this approach and Michael and Daniel being for it. It doesn't look like it's going to get committed this commitfest, shall we move it forward or mark it returned with feedback?

Re: Corruption during WAL replay

2022-03-24 Thread Tom Lane
Robert Haas writes: > Thanks, committed. Some of the buildfarm is seeing failures in the pg_checksums test. regards, tom lane

Re: logical decoding and replication of sequences

2022-03-24 Thread Tomas Vondra
On 3/24/22 22:52, Greg Stark wrote: > Awesome to see this get committed, thanks Tomas. > > Is there anything left or shall I update the CF entry to committed? Yeah, let's mark it as committed. I was waiting for some feedback from the buildfarm - there are some failures, but it seems unrelated.

Re: logical decoding and replication of sequences

2022-03-24 Thread Tomas Vondra
Hi, Pushed, after going through the patch once more, addressed the remaining FIXMEs, corrected a couple places in the docs and comments, etc. Minor tweaks, nothing important. I've been thinking about the grammar a bit more after pushing, and I realized that maybe it'd be better to handle the FOR

Re: New Object Access Type hooks

2022-03-24 Thread Andres Freund
Hi, On 2022-03-24 17:44:31 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 3/24/22 16:59, Andres Freund wrote: > >> I just rebased the meson tree (atop 75b1521dae1) and the test_oat_hooks > >> test > >> fail on windows with meson. > > > Even if you use NO_LOCALE=1/ENCODING=UTF8 as the

Re: automatically generating node support functions

2022-03-24 Thread David Rowley
On Fri, 18 Feb 2022 at 19:52, Peter Eisentraut wrote: > [ v5-0001-Automatically-generate-node-support-functions.patch ] I've been looking over the patch and wondering the best way to move this forward. But first a couple of things I noted down from reading the patch: 1. You're written: *

fixing a few backup compression goofs

2022-03-24 Thread Robert Haas
On Wed, Mar 23, 2022 at 5:52 PM Justin Pryzby wrote: > I think you should also test the return value when setting the compress level. > Not only because it's generally a good idea, but also because I suggested to > support negative compression levels. Which weren't allowed before v1.3.4, and >

Re: logical decoding and replication of sequences

2022-03-24 Thread Greg Stark
Awesome to see this get committed, thanks Tomas. Is there anything left or shall I update the CF entry to committed?

Re: [PATCH] improve the pg_upgrade error message

2022-03-24 Thread Daniel Gustafsson
> On 1 Dec 2021, at 11:15, Daniel Gustafsson wrote: > >> On 1 Dec 2021, at 10:59, Jeevan Ladhe wrote: > >> Was wondering if we had any barriers to getting this committed. > > No barrier other than available time to, I will try to get to it shortly. The "shortly" aspect wasn't really

Re: New Object Access Type hooks

2022-03-24 Thread Tom Lane
Andrew Dunstan writes: > On 3/24/22 16:59, Andres Freund wrote: >> I just rebased the meson tree (atop 75b1521dae1) and the test_oat_hooks test >> fail on windows with meson. > Even if you use NO_LOCALE=1/ENCODING=UTF8 as the Makefile now does? Note that that's basically a workaround for buggy

Re: Add non-blocking version of PQcancel

2022-03-24 Thread Tom Lane
Jacob Champion writes: > On Thu, 2022-01-13 at 14:51 +, Jelte Fennema wrote: >> 2. Cancel connections benefit automatically from any improvements made >> to the normal connection establishment codepath. Examples of things >> that it currently gets for free currently are TLS support and >>

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-24 Thread Peter Geoghegan
On Thu, Mar 24, 2022 at 1:21 PM Robert Haas wrote: > > How about: "Set relfrozenxid to oldest extant XID seen by VACUUM" > > Sure, that sounds nice. Cool. > > What you're saying here boils down to this: it doesn't matter what the > > visibility map would say right this microsecond (in the

Re: New Object Access Type hooks

2022-03-24 Thread Andrew Dunstan
On 3/24/22 16:59, Andres Freund wrote: > Hi, > > I just rebased the meson tree (atop 75b1521dae1) and the test_oat_hooks test > fail on windows with meson. They don't with our "homegrown" buildsystem, but > just because it won't run them. > > https://cirrus-ci.com/build/6101947223638016 >

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-24 Thread Thomas Munro
On Fri, Mar 25, 2022 at 1:43 AM David Christensen wrote: > > On Mar 24, 2022, at 6:43 AM, Thomas Munro wrote: > > On Fri, Mar 25, 2022 at 12:26 AM Thomas Munro > > wrote: > >>> On Fri, Mar 25, 2022 at 12:01 AM Peter Eisentraut > >>> wrote: > >>> Or even: Why are we exposing fork *numbers*

Re: New Object Access Type hooks

2022-03-24 Thread Andres Freund
Hi, I just rebased the meson tree (atop 75b1521dae1) and the test_oat_hooks test fail on windows with meson. They don't with our "homegrown" buildsystem, but just because it won't run them. https://cirrus-ci.com/build/6101947223638016

Re: Doc patch: replace 'salesmen' with 'salespeople'

2022-03-24 Thread Daniel Gustafsson
> On 24 Mar 2022, at 19:34, Dagfinn Ilmari Mannsåker wrote: > I just spotted an unnecessarily gendered example involving a 'salesmen' > table in the UPDATE docs. Here's a patch that changes that to > 'salespeople'. No objections to changing that, it's AFAICT the sole such usage in the docs. >

Re: identifying unrecognized node type errors

2022-03-24 Thread Andrew Dunstan
On 3/24/22 16:10, Tom Lane wrote: > Andrew Dunstan writes: >> As I was tracking down some of these errors in the sql/json patches I >> noticed that we have a whole lot of them in the code, so working out >> which one has triggered an error is not as easy as it might be. ISTM we >> could

Re: [PATCH] pg_statio_all_tables: several rows per table due to invalid TOAST index

2022-03-24 Thread Tom Lane
I wrote: > After a bit of further fooling, I found that we could make that > work with LEFT JOIN LATERAL. This formulation has a different > problem, which is that if you do want most or all of the output, > computing each sub-aggregation separately is probably less > efficient than it could be.

Re: Estimating HugePages Requirements?

2022-03-24 Thread Nathan Bossart
On Thu, Mar 24, 2022 at 02:07:26PM +0100, Magnus Hagander wrote: > On Wed, Mar 23, 2022 at 7:25 AM Michael Paquier wrote: >> On Tue, Mar 15, 2022 at 03:44:39PM -0700, Nathan Bossart wrote: >> > A simple approach could be to just set log_min_messages to PANIC before >> > exiting. I've attached a

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Robert Haas
On Thu, Mar 24, 2022 at 4:20 PM Nathan Bossart wrote: > Another possibility could be to add a hook that is called _before_ > _PG_init() where libraries are permitted to adjust GUCs. After the library > is loaded, we first call this _PG_change_GUCs() function, then we > initialize MaxBackends,

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-24 Thread Robert Haas
On Thu, Mar 24, 2022 at 3:28 PM Peter Geoghegan wrote: > But non-aggressive VACUUMs have always been able to do that. > > How about: "Set relfrozenxid to oldest extant XID seen by VACUUM" Sure, that sounds nice. > Believe it or not, I avoided functional changes in 0002 -- at least in > one

Re: make MaxBackends available in _PG_init

2022-03-24 Thread Nathan Bossart
On Wed, Mar 23, 2022 at 09:03:18PM +0800, Julien Rouhaud wrote: > On Wed, Mar 23, 2022 at 08:32:39AM -0400, Robert Haas wrote: >> Well, the conclusion upthread was that extensions might change the >> values of those GUCs from _PG_init(). If that's a real thing, then >> what you're asking for here

Re: identifying unrecognized node type errors

2022-03-24 Thread Tom Lane
Andrew Dunstan writes: > As I was tracking down some of these errors in the sql/json patches I > noticed that we have a whole lot of them in the code, so working out > which one has triggered an error is not as easy as it might be. ISTM we > could usefully prefix each such message with the name

  1   2   3   >