RE: Rework LogicalOutputPluginWriterUpdateProgress

2023-03-06 Thread Hayato Kuroda (Fujitsu)
Dear Wang, Thank you for updating the patch! Followings are my comments. --- 01. missing comments You might miss the comment from Peter[1]. Or could you pin the related one? --- 02. LogicalDecodingProcessRecord() Don't we have to call UpdateDecodingProgressAndKeepalive() when there is no

Re: pg_rewind: Skip log directory for file type check like pg_wal

2023-03-06 Thread Soumyadeep Chakraborty
On Mon, Mar 6, 2023 at 11:33 PM Alexander Kukushkin wrote: > > > Lets assume that on the source we have "pg_log" and on the target we have > "my_log" (they are configured using "log_directory" GUC). > When doing rewind in this case we want neither to remove the content of > "my_log" on the

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Peter Smith
On Mon, Mar 6, 2023 at 10:18 PM Önder Kalacı wrote: > >> 4. IdxIsRelationIdentityOrPK >> >> +/* >> + * Given a relation and OID of an index, returns true if the >> + * index is relation's replica identity index or relation's >> + * primary key's index. >> + * >> + * Returns false otherwise. >> +

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Thomas Munro
On Tue, Mar 7, 2023 at 7:47 PM Julien Rouhaud wrote: > I registered lapwing as a 32b Debian 7 so I thought it would be expected to > keep it as-is rather than upgrading to all newer major Debian versions, > especially since there were newer debian animal registered (no 32b though > AFAICS).

Re: pg_rewind: Skip log directory for file type check like pg_wal

2023-03-06 Thread Alexander Kukushkin
On Mon, 6 Mar 2023 at 19:37, Soumyadeep Chakraborty < soumyadeep2...@gmail.com> wrote: > > > 2. Unlike "pg_wal", the "log" directory is not necessarily located > inside PGDATA. The actual value is configured using "log_directory" GUC, > which just happened to be "log" by default. And in fact

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Michael Paquier
On Tue, Mar 07, 2023 at 03:49:02PM +0900, Kyotaro Horiguchi wrote: > Ah. Yes, that expansion sounds sensible. Okay, so, based on this idea, I have hacked on this stuff and finish with the attached that shows block data if it exists, as well as FPI stuff if any. bimg_info is showed as a text[]

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-06 Thread Bharath Rupireddy
On Tue, Mar 7, 2023 at 3:30 AM Nathan Bossart wrote: > > +void > +XLogReadFromBuffers(XLogRecPtr startptr, > > Since this function presently doesn't return anything, can we have it > return the number of bytes read instead of storing it in a pointer > variable? Done. > +ptr = startptr; > +

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Amit Kapila
On Tue, Mar 7, 2023 at 9:16 AM shiy.f...@fujitsu.com wrote: > > On Monay, Mar 6, 2023 7:19 PM Önder Kalacı wrote: > > > > Yeah, seems easier to follow to me as well. Reflected it in the comment as > > well. > > Few comments: = 1. +get_usable_indexoid(ApplyExecutionData *edata,

Re: [HACKERS] make async slave to wait for lsn to be replayed

2023-03-06 Thread Kartyshov Ivan
Fix build.meson troubles -- Ivan Kartyshov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companydiff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index dbe9394762..422bb1ed82 100644 ---

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Kyotaro Horiguchi
At Tue, 7 Mar 2023 14:44:49 +0900, Michael Paquier wrote in > On Tue, Mar 07, 2023 at 11:17:45AM +0900, Kyotaro Horiguchi wrote: > > Thus I'm inclined to agree with Michael's suggestion of creating a new > > normalized set-returning function that returns information of > > "blocks". > > Just

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Julien Rouhaud
On Tue, Mar 07, 2023 at 07:14:51PM +1300, Thomas Munro wrote: > On Tue, Mar 7, 2023 at 5:32 PM Michael Paquier wrote: > > On Tue, Mar 07, 2023 at 03:44:46PM +1300, Thomas Munro wrote: > > > Oh, you already pushed a fix. But now I'm wondering if it's useful to > > > have old buggy compilers set

Re: Make mesage at end-of-recovery less scary.

2023-03-06 Thread Kyotaro Horiguchi
At Mon, 6 Mar 2023 14:58:15 -0500, "Gregory Stark (as CFM)" wrote in > It looks like this needs a rebase and at a quick glance it looks like > more than a trivial conflict. I'll mark it Waiting on Author. Please > update it back when it's rebased Thanks for checking it! I think 4ac30ba4f2 is

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Thomas Munro
On Tue, Mar 7, 2023 at 5:32 PM Michael Paquier wrote: > On Tue, Mar 07, 2023 at 03:44:46PM +1300, Thomas Munro wrote: > > Oh, you already pushed a fix. But now I'm wondering if it's useful to > > have old buggy compilers set to run with -Werror. > > Yes, as far as I can see when investigating

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Julien Rouhaud
On Tue, Mar 07, 2023 at 01:56:24PM +0900, Michael Paquier wrote: > On Tue, Mar 07, 2023 at 12:42:20PM +0800, Julien Rouhaud wrote: > > ah right I should have checked. but the same ABI compatibility concern > > still exists for version 1.0 of the extension. > > Yes, we'd better make sure that the

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Michael Paquier
On Tue, Mar 07, 2023 at 11:17:45AM +0900, Kyotaro Horiguchi wrote: > Thus I'm inclined to agree with Michael's suggestion of creating a new > normalized set-returning function that returns information of > "blocks". Just to be clear here, I am not suggesting to add a new function for only the

Re: NumericShort vs NumericLong format

2023-03-06 Thread Tom Lane
"David G. Johnston" writes: > As an aside, for anyone more fluent than I who reads this, is the use of > the word "dynamic scale" in this code comment supposed to be "display > scale"? >

Re: Should vacuum process config file reload more often

2023-03-06 Thread Masahiko Sawada
On Mon, Mar 6, 2023 at 5:26 AM Melanie Plageman wrote: > > On Thu, Mar 2, 2023 at 6:37 PM Melanie Plageman > wrote: > > > > On Thu, Mar 2, 2023 at 2:36 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Mar 2, 2023 at 10:41 AM Melanie Plageman > > > wrote: > > > > On another topic, I've just

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 02:29:50PM -0800, Andres Freund wrote: > Thanks. Sure, no problem. If there is anything else needed for this thread, feel free to ping me here. -- Michael signature.asc Description: PGP signature

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Michael Paquier
On Tue, Mar 07, 2023 at 12:42:20PM +0800, Julien Rouhaud wrote: > ah right I should have checked. but the same ABI compatibility concern > still exists for version 1.0 of the extension. Yes, we'd better make sure that the past code is able to run, at least. Now I am not really convinced that we

Re: Raising the SCRAM iteration count

2023-03-06 Thread Michael Paquier
On Fri, Mar 03, 2023 at 11:13:36PM +0100, Daniel Gustafsson wrote: > That would indeed be nice, but is there a way to do this without a complicated > pump TAP expression? I was unable to think of a way but I might be missing > something? A SET command refreshes immediately the cache information

Re: NumericShort vs NumericLong format

2023-03-06 Thread David G. Johnston
I'll give this a go as a learning exercise for myself... On Mon, Mar 6, 2023 at 8:47 PM Amin wrote: > > - How can I determine which format will be used for a numeric type? > https://github.com/postgres/postgres/blob/cf96907aadca454c4094819c2ecddee07eafe203/src/backend/utils/adt/numeric.c#L491

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Julien Rouhaud
On Tue, 7 Mar 2023, 12:36 Michael Paquier, wrote: > On Tue, Mar 07, 2023 at 09:13:46AM +0800, Julien Rouhaud wrote: > > It's problematic to install the extension if we rely on upgrade scripts > only. > > We could also provide a pg_walinspect--1.2.sql file and it would just > work, and > > that

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Michael Paquier
On Tue, Mar 07, 2023 at 09:13:46AM +0800, Julien Rouhaud wrote: > It's problematic to install the extension if we rely on upgrade scripts only. > We could also provide a pg_walinspect--1.2.sql file and it would just work, > and > that may have been a good idea if there wasn't also the problem of

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Michael Paquier
On Tue, Mar 07, 2023 at 03:44:46PM +1300, Thomas Munro wrote: > On Tue, Mar 7, 2023 at 3:42 PM Thomas Munro wrote: >> Apparently ye olde GCC 4.7 on "lapwing" doesn't like the way you >> initialised that struct. I guess it wants {{0}} instead of {0}. >> Apparently old GCC was wrong about that

RE: [Proposal] Add foreign-server health checks infrastructure

2023-03-06 Thread Hayato Kuroda (Fujitsu)
Dear Katsuragi-san, Thank you for reviewing! PSA new version patches. > I think we can update the status to ready for committer after > this fix, if there is no objection. That's a very good news for me! How about other people? > >> 7. the document of postgres_fdw_verify_connection_states_all

Re: Making empty Bitmapsets always be NULL

2023-03-06 Thread David Rowley
On Sat, 4 Mar 2023 at 11:08, Tom Lane wrote: > > David Rowley writes: > > On Fri, 3 Mar 2023 at 15:17, Tom Lane wrote: > > It's true that the majority of Bitmapsets are going to be just 1 word, > > but it's important to acknowledge that we do suffer in some more > > extreme cases when

NumericShort vs NumericLong format

2023-03-06 Thread Amin
Hi, - How can I determine which format will be used for a numeric type? - What the precision and scale values should be for pgsql to use the long format? Is there a threshold?

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread shiy.f...@fujitsu.com
On Monay, Mar 6, 2023 7:19 PM Önder Kalacı wrote: > > Yeah, seems easier to follow to me as well. Reflected it in the comment as > well. > Thanks for updating the patch. Here are some comments on v33-0001 patch. 1. + if (RelationReplicaIdentityFullIndexScanEnabled(localrel) && +

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 04:59:49PM -0800, Nathan Bossart wrote: > That did cross my mind, but I was worried that trying to explain all that > here could cause confusion. > > If PROCESS_MAIN is set (the default), it's time to vacuum the main > relation. Otherwise, we can skip this

Re: Record queryid when auto_explain.log_verbose is on

2023-03-06 Thread Imseih (AWS), Sami
> > It's a bit annoying that the info is missing since pg 14, but we > > probably can't > > backpatch this as it might break log parser tools. > What do you think? That's a good point about log parsing tools, i.e. pgbadger. Backpatching does not sounds to appealing to me after giving this a

Re: Allow logical replication to copy tables in binary format

2023-03-06 Thread Amit Kapila
On Wed, Mar 1, 2023 at 7:58 PM Melih Mutlu wrote: > > Bharath Rupireddy , 1 Mar 2023 Çar, > 15:02 tarihinde şunu yazdı: >> > > That was my intention in the beginning with this patch. Then the new option > also made some sense at some point, and I added copy_binary option according > to

Re: Moving forward with TDE

2023-03-06 Thread Chris Travers
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested I have decided to write a review here in terms of whether we want this

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Kyotaro Horiguchi
At Mon, 6 Mar 2023 15:21:14 -0500, Melanie Plageman wrote in > On Mon, Mar 6, 2023 at 2:34 PM Andres Freund wrote: > > That, but also because it's simply more reliable. autovacuum=off doesn't > > protect against a anti-wraparound vacuum or such. Or a concurrent test > > somehow > > triggering

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Amit Kapila
On Tue, Mar 7, 2023 at 1:34 AM Andres Freund wrote: > > On 2023-03-01 14:10:07 +0530, Amit Kapila wrote: > > On Wed, Mar 1, 2023 at 12:09 AM Andres Freund wrote: > > > > > > > I see this as a way to provide this feature for users but I would > > > > prefer to proceed with this if we can get some

Re: proposal: possibility to read dumped table's name from file

2023-03-06 Thread Julien Rouhaud
Hi, On Mon, Mar 06, 2023 at 10:20:32PM +0100, Daniel Gustafsson wrote: > > On 6 Mar 2023, at 21:45, Gregory Stark (as CFM) wrote: > > > > So This patch has been through a lot of commitfests. And it really > > doesn't seem that hard to resolve -- Pavel has seemingly been willing > > to go

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Thomas Munro
On Tue, Mar 7, 2023 at 3:42 PM Thomas Munro wrote: > Apparently ye olde GCC 4.7 on "lapwing" doesn't like the way you > initialised that struct. I guess it wants {{0}} instead of {0}. > Apparently old GCC was wrong about that warning[1], but that system > doesn't have the back-patched fixes?

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Thomas Munro
On Mon, Mar 6, 2023 at 5:30 PM Michael Paquier wrote: > On Mon, Feb 20, 2023 at 01:54:00PM +0530, Bharath Rupireddy wrote: > > I ran some tests on my dev system [1] and I don't see much difference > > between v3 and v4. So, +1 for v3 patch (+ argument order swap) from > > Andres to keep the code

Re: Record queryid when auto_explain.log_verbose is on

2023-03-06 Thread torikoshia
On 2023-03-07 08:50, Imseih (AWS), Sami wrote: I am wondering if this patch should be backpatched? The reason being is in auto_explain documentation [1], there is a claim of equivalence of the auto_explain.log_verbose option and EXPLAIN(verbose) ". it's equivalent to the VERBOSE option of

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Kyotaro Horiguchi
At Tue, 7 Mar 2023 09:34:24 +0900, Michael Paquier wrote in > On Mon, Mar 06, 2023 at 04:08:28PM +0100, Matthias van de Meent wrote: > > On Mon, 6 Mar 2023 at 15:40, Bharath Rupireddy > >> IMO, pg_get_wal_records_extended_info as proposed doesn't look good to > >> me as it outputs most of the

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-06 Thread Alexander Korotkov
On Tue, Mar 7, 2023 at 4:50 AM Andres Freund wrote: > On 2023-03-02 14:28:56 +0400, Pavel Borisov wrote: > > 2. Heap updates with low tuple concurrency: > > Prepare with pkeys (pgbench -d postgres -i -I dtGvp -s 300 > > --unlogged-tables) > > Update 3*10^7 rows, 50 conns (pgbench postgres -f > >

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-03-06 Thread torikoshia
On 2023-03-06 23:03, Daniel Gustafsson wrote: On 28 Feb 2023, at 15:28, Damir Belyalov wrote: Tested patch on all cases: CIM_SINGLE, CIM_MULTI, CIM_MULTI_CONDITION. As expected it works. Also added a description to copy.sgml and made a review on patch. Thanks for your tests and

Re: [Proposal] Add foreign-server health checks infrastructure

2023-03-06 Thread Katsuragi Yuta
Hi Kuroda-san, Thank you for updating the patch! I think we can update the status to ready for committer after this fix, if there is no objection. 7. the document of postgres_fdw_verify_connection_states_all NULL + is returned if the local session does not have connection caches, or

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-06 Thread Andres Freund
Hi, On 2023-03-02 14:28:56 +0400, Pavel Borisov wrote: > 2. Heap updates with low tuple concurrency: > Prepare with pkeys (pgbench -d postgres -i -I dtGvp -s 300 --unlogged-tables) > Update 3*10^7 rows, 50 conns (pgbench postgres -f > ./update-only-account.sql -s 300 -P10 -M prepared -T 600 -j 5

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-03-06 Thread Alexander Korotkov
On Thu, Mar 2, 2023 at 11:16 PM Alexander Korotkov wrote: > On Thu, Mar 2, 2023 at 9:17 PM Pavel Borisov wrote: > > On Thu, 2 Mar 2023 at 18:53, Alexander Korotkov > > wrote: > > > On Thu, Mar 2, 2023 at 1:29 PM Pavel Borisov > > > wrote: > > > > > Let's see the performance results for the

Re: Inaccurate comment for pg_get_partkeydef

2023-03-06 Thread David Rowley
On Mon, 6 Mar 2023 at 18:54, Japin Li wrote: > PSA patch to fix a comment inaccurate. Thanks. Pushed. David

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-03-06 Thread Masahiko Sawada
On Tue, Mar 7, 2023 at 1:01 AM John Naylor wrote: > > On Mon, Mar 6, 2023 at 1:28 PM Masahiko Sawada wrote: > > > > Since the block-level measurement is likely overestimating quite a bit, I > > > propose to simply reverse the order of the actions here, effectively > > > reporting progress for

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-06 Thread Julien Rouhaud
On Mon, Mar 06, 2023 at 08:36:17PM +0530, Bharath Rupireddy wrote: > On Mon, Mar 6, 2023 at 2:22 PM Julien Rouhaud wrote: > > > Also: > > > > /* > > - * Get info and data of all WAL records from start LSN till end of WAL. > > + * NB: This function does nothing and stays here for backward > >

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Tue, Mar 07, 2023 at 09:20:12AM +0900, Michael Paquier wrote: > -* Do the actual work --- either FULL or "lazy" vacuum > +* If PROCESS_MAIN is set (the default), it's time to vacuum the main > +* relation. Otherwise, we can skip this part. If required, we'll > process > +*

Re: Evaluate arguments of correlated SubPlans in the referencing ExprState

2023-03-06 Thread Tom Lane
Andres Freund writes: > On 2023-03-03 15:09:18 -0500, Tom Lane wrote: >> It'd be good to have a header comment for ExecInitExprRec documenting >> the arguments, particularly that resv/resnull are where to put the >> subplan's eventual result. > Did you mean ExecInitSubPlanExpr()? Right,

Re: Add pg_walinspect function with block info columns

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 04:08:28PM +0100, Matthias van de Meent wrote: > On Mon, 6 Mar 2023 at 15:40, Bharath Rupireddy >> IMO, pg_get_wal_records_extended_info as proposed doesn't look good to >> me as it outputs most of the columns that are already given by >> pg_get_wal_records_info.What I

Re: Evaluate arguments of correlated SubPlans in the referencing ExprState

2023-03-06 Thread Andres Freund
Hi, On 2023-03-03 15:09:18 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-03-02 13:00:31 -0800, Andres Freund wrote: > >> I'm not opposed to EXPR_PARAM_SET, to be clear. I'll send an updated > >> version later. I was just thinking about the correctness in the current > >> world. > >

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 06:12:36PM -0500, Melanie Plageman wrote: > LGTM. -* Do the actual work --- either FULL or "lazy" vacuum +* If PROCESS_MAIN is set (the default), it's time to vacuum the main +* relation. Otherwise, we can skip this part. If required, we'll process +* the

Re: [PoC] Let libpq reject unexpected authentication requests

2023-03-06 Thread Jacob Champion
On Fri, Mar 3, 2023 at 6:35 PM Michael Paquier wrote: > I was refreshing my mind with 0001 yesterday, and except for the two > parts where we need to worry about AUTH_REQ_OK being sent too early > and the business with gssenc, this is a rather straight-forward. It > also looks like the the

Re: Normalization of utility queries in pg_stat_statements

2023-03-06 Thread Michael Paquier
On Mon, Mar 06, 2023 at 03:50:55PM +0300, Andrei Zubkov wrote: > Those statements is not related to any WAL tests. It seems a little bit > incorrect to me. The intention is to have each file run in isolation, so this is incorrect as it stands. Thanks for the report! -- Michael signature.asc

Re: a very minor bug and a couple of comment changes for basebackup.c

2023-03-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > Thanks for the review. I have committed the patches. No objections to what was committed. > On Thu, Mar 2, 2023 at 2:59 AM Michael Paquier wrote: > > There is more to it: the page LSN is checked before its checksum. > > Hence, if the

Re: Record queryid when auto_explain.log_verbose is on

2023-03-06 Thread Imseih (AWS), Sami
I am wondering if this patch should be backpatched? The reason being is in auto_explain documentation [1], there is a claim of equivalence of the auto_explain.log_verbose option and EXPLAIN(verbose) ". it's equivalent to the VERBOSE option of EXPLAIN." This can be quite confusing for users

RE: Ability to reference other extensions by schema in extension scripts

2023-03-06 Thread Regina Obe
> It looks like this patch needs a quick rebase, there's a conflict in the > meson.build. > > I'll leave the state since presumably this would be easy to resolve but it > would > be more likely to get attention if it's actually building cleanly. > > http://cfbot.cputube.org/patch_42_4023.log >

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 5:43 PM Nathan Bossart wrote: > > On Mon, Mar 06, 2023 at 05:09:58PM -0500, Melanie Plageman wrote: > > I would move this comment inside of the outer if statement since it is > > distinguishing between the two branches of the inner if statement. > > Oops, done. > > > Also,

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 13:29:50 +0900, Michael Paquier wrote: > On Mon, Feb 20, 2023 at 01:54:00PM +0530, Bharath Rupireddy wrote: > > I ran some tests on my dev system [1] and I don't see much difference > > between v3 and v4. So, +1 for v3 patch (+ argument order swap) from > > Andres to keep the

Re: On login trigger: take three

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 23:10, Andres Freund wrote: > > Hi, > > On 2023-03-06 15:55:01 -0500, Gregory Stark (as CFM) wrote: >> It looks like Daniel Gustafsson, Andres, and Tom have all weighed in >> on this patch with at least a neutral comment (+-0 from Andres :) >> >> It looks like the main

Re: On login trigger: take three

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 15:55:01 -0500, Gregory Stark (as CFM) wrote: > It looks like Daniel Gustafsson, Andres, and Tom have all weighed in > on this patch with at least a neutral comment (+-0 from Andres :) > > It looks like the main concern was breaking physical replicas and that > there was

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 06, 2023 at 01:13:37PM -0800, Nathan Bossart wrote: > On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote: > > On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart > > wrote: > >> On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: > >> > I noticed in vacuum_rel()

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-06 Thread Nathan Bossart
+void +XLogReadFromBuffers(XLogRecPtr startptr, +TimeLineID tli, +Size count, +char *buf, +Size *read_bytes) Since this function presently doesn't return anything, can we have it return the number of bytes read

Re: using memoize in in paralel query decreases performance

2023-03-06 Thread David Rowley
On Mon, 6 Mar 2023 at 21:55, Pavel Stehule wrote: > default https://explain.depesz.com/s/fnBe It looks like the slowness is coming from the Bitmap Index scan and Bitmap heap scan rather than Memoize. -> Bitmap Heap Scan on public.item i (cost=285.69..41952.12 rows=29021 width=16)

RE: Ability to reference other extensions by schema in extension scripts

2023-03-06 Thread Regina Obe
> It looks like this patch needs a quick rebase, there's a conflict in the > meson.build. > > I'll leave the state since presumably this would be easy to resolve but it > would > be more likely to get attention if it's actually building cleanly. > > http://cfbot.cputube.org/patch_42_4023.log >

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-06 Thread Regina Obe
> This patch too is conflicting on meson.build. Are these two patches > interdependent? > > This one looks a bit more controversial. I'm not sure if Tom has been > convinced and I haven't seen anyone else speak up. > > Perhaps this needs a bit more discussion of other options to solve this

Re: On login trigger: take three

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 21:55, Gregory Stark (as CFM) wrote: > > It looks like Daniel Gustafsson, Andres, and Tom have all weighed in > on this patch with at least a neutral comment (+-0 from Andres :) I think the concept of a login event trigger has merits, even though it's kind of a niche use

Re: proposal: possibility to read dumped table's name from file

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 21:45, Gregory Stark (as CFM) wrote: > > So This patch has been through a lot of commitfests. And it really > doesn't seem that hard to resolve -- Pavel has seemingly been willing > to go along whichever way the wind has been blowing but honestly it > kind of seems like

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote: > On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart > wrote: >> On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: >> > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is >> > called, 4211fbd84

Re: On login trigger: take three

2023-03-06 Thread Gregory Stark (as CFM)
It looks like Daniel Gustafsson, Andres, and Tom have all weighed in on this patch with at least a neutral comment (+-0 from Andres :) It looks like the main concern was breaking physical replicas and that there was consensus that as long as single-user mode worked that it was ok? So maybe it's

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart wrote: > > On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: > > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is > > called, 4211fbd84 changes the else into an else if [1]. I understand > > after reading the

Re: proposal: possibility to read dumped table's name from file

2023-03-06 Thread Gregory Stark (as CFM)
So This patch has been through a lot of commitfests. And it really doesn't seem that hard to resolve -- Pavel has seemingly been willing to go along whichever way the wind has been blowing but honestly it kind of seems like he's just gotten drive-by suggestions and he's put a lot of work into

Re: Support logical replication of DDLs

2023-03-06 Thread Zheng Li
On Mon, Mar 6, 2023 at 5:17 AM wangw.f...@fujitsu.com wrote: > > For v-75-0003* patch. > 2. In the function deparse_CreateSeqStmt. > It seems that we are not deparsing the "AS data_type" clause (CREATE SEQUENCE > ... AS data_type). I think this causes all data_type to be default (bigint) > after

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote: > I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is > called, 4211fbd84 changes the else into an else if [1]. I understand > after reading the commit and re-reading the code why that is now, but I > was

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Melanie Plageman
On Mon, Mar 6, 2023 at 2:34 PM Andres Freund wrote: > > Hi, > > On 2023-03-06 14:24:09 -0500, Melanie Plageman wrote: > > On Mon, Mar 06, 2023 at 11:09:19AM -0800, Andres Freund wrote: > > > On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > > > > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-03-06 Thread Andrei Zubkov
Hi Gregory, > patching file contrib/pg_stat_statements/sql/oldextversions.sql > can't find file to patch at input line 1833 > > > and > > > --- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql > > +++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql > -- >

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-06 Thread Gregory Stark (as CFM)
The CFBot says this patch is failing but I find it hard to believe this is related to this patch... 2023-03-05 20:56:58.705 UTC [33902][client backend] [pg_regress/btree_index][18/750:0] STATEMENT: ALTER INDEX btree_part_idx ALTER COLUMN id SET (n_distinct=100); 2023-03-05 20:56:58.709 UTC

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 17:02:38 +0530, Amit Kapila wrote: > Andres, do you have any thoughts on this? We seem to have figured out > the cause of regression in the case Shi-San has reported and others > also agree with it. We can't think of doing anything better than what > the patch currently is

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-03-06 Thread Gregory Stark (as CFM)
I'm sorry, It seems this is failing again? It's Makefile and meson.build again :( Though there are also patching file contrib/pg_stat_statements/sql/oldextversions.sql can't find file to patch at input line 1833 and |--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql |+++

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-06 Thread Andres Freund
Hi, On 2023-03-01 14:10:07 +0530, Amit Kapila wrote: > On Wed, Mar 1, 2023 at 12:09 AM Andres Freund wrote: > > > > > I see this as a way to provide this feature for users but I would > > > prefer to proceed with this if we can get some more buy-in from senior > > > community members (at least

Re: Make mesage at end-of-recovery less scary.

2023-03-06 Thread Gregory Stark (as CFM)
It looks like this needs a rebase and at a quick glance it looks like more than a trivial conflict. I'll mark it Waiting on Author. Please update it back when it's rebased -- Gregory Stark As Commitfest Manager

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-06 Thread Gregory Stark (as CFM)
This patch too is conflicting on meson.build. Are these two patches interdependent? This one looks a bit more controversial. I'm not sure if Tom has been convinced and I haven't seen anyone else speak up. Perhaps this needs a bit more discussion of other options to solve this issue. Maybe it can

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-03-06 Thread Heikki Linnakangas
On 01/03/2023 12:21, Aleksander Alekseev wrote: Hi, I'm surprised that these patches extend the page numbering to 64 bits, but never actually uses the high bits. The XID "epoch" is not used, and pg_xact still wraps around and the segment names are still reused. I thought we could stop doing

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Melanie Plageman
On Mon, Mar 06, 2023 at 09:37:23AM -0800, Nathan Bossart wrote: > On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote: > > That was mostly OK for me, so applied after tweaking a couple of > > places in the tests (extra explanations, for one), the comments and > > the code. I noticed

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 14:24:09 -0500, Melanie Plageman wrote: > On Mon, Mar 06, 2023 at 11:09:19AM -0800, Andres Freund wrote: > > On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > > > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi > > > wrote: > > > > > > > > At Mon, 06 Mar 2023 15:24:25

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Melanie Plageman
On Mon, Mar 06, 2023 at 11:09:19AM -0800, Andres Freund wrote: > Hi, > > On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi > > wrote: > > > > > > At Mon, 06 Mar 2023 15:24:25 +0900 (JST), Kyotaro Horiguchi > > > wrote in > > > > In any

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-06 Thread Andres Freund
Hi, On 2023-03-06 10:09:24 -0500, Melanie Plageman wrote: > On Mon, Mar 6, 2023 at 1:48 AM Kyotaro Horiguchi > wrote: > > > > At Mon, 06 Mar 2023 15:24:25 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > In any case, I think we need to avoid such concurrent autovacuum/analyze. > > > > If it

Re: Ability to reference other extensions by schema in extension scripts

2023-03-06 Thread Gregory Stark (as CFM)
It looks like this patch needs a quick rebase, there's a conflict in the meson.build. I'll leave the state since presumably this would be easy to resolve but it would be more likely to get attention if it's actually building cleanly. http://cfbot.cputube.org/patch_42_4023.log On Tue, 28 Feb

Re: About default inBufSize (connection read buffer size) in libpq

2023-03-06 Thread Andres Freund
Hi, On 2023-03-05 05:42:06 +0300, Трофимов Иван wrote: > I was running some benchmarks for PG driver built on top of libpq async > functionality, > and noticed that recv syscalls issued by the application are limited by 16Kb, > which seems to > be inBufSize coming from makeEmptyPGconn in

Re: Commitfest 2023-03 starting tomorrow!

2023-03-06 Thread Gregory Stark (as CFM)
Sorry, I wasn't feeling very well since Friday. I'm still not 100% but I'm going to try to do some triage this afternoon. There are a few patches that need a rebase. And a few patches failing Meson builds or autoconf stages -- I wonder if there's something unrelated broken there? But what I

Re: pg_rewind: Skip log directory for file type check like pg_wal

2023-03-06 Thread Soumyadeep Chakraborty
On Mon, Mar 6, 2023 at 12:28 AM Alexander Kukushkin wrote: > > Hello Soumyadeep, > > The problem indeed exists, but IMO the "log" directory case must be handled > differently: > 1. We don't need or I would even say we don't want to sync log files from the > new primary, because it destroys the

Re: add PROCESS_MAIN to VACUUM

2023-03-06 Thread Nathan Bossart
On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote: > That was mostly OK for me, so applied after tweaking a couple of > places in the tests (extra explanations, for one), the comments and > the code. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: HOT chain validation in verify_heapam()

2023-03-06 Thread Robert Haas
On Thu, Feb 9, 2023 at 12:09 PM Himanshu Upadhyaya wrote: > Initially while implementing logic to identify the root of the HOT chain > I was getting crash and regression failure's that time I thought of having > this check along with a few other changes that were required, > but you are right,

Re: running logical replication as the subscription owner

2023-03-06 Thread Jelte Fennema
> Yeah. As Andres pointed out somewhere or other, that also means you're > decoding the WAL once per user instead of just once. I'm surprised > that hasn't been cost-prohibitive. We'd definitely prefer to have one subscription and do the decoding only once. But we haven't run into big perf issues

Re: running logical replication as the subscription owner

2023-03-06 Thread Robert Haas
On Fri, Mar 3, 2023 at 6:57 PM Jelte Fennema wrote: > I'm definitely in favor of making it easier to use logical replication > in a safe manner. Cool. > In Citus we need to logically replicate and we're > currently using quite some nasty and undocumented hacks to do so: > We're creating a

Re: Timeline ID hexadecimal format

2023-03-06 Thread Peter Eisentraut
On 03.03.23 16:52, Sébastien Lardière wrote: On 02/03/2023 09:12, Peter Eisentraut wrote: On 24.02.23 17:27, Sébastien Lardière wrote: diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index be05a33205..7e26b51031 100644 --- a/doc/src/sgml/backup.sgml +++

Re: Track IO times in pg_stat_io

2023-03-06 Thread Melanie Plageman
Thanks for the review! On Tue, Feb 28, 2023 at 4:49 AM Drouvot, Bertrand wrote: > On 2/26/23 5:03 PM, Melanie Plageman wrote: > > As suggested in [1], the attached patch adds IO times to pg_stat_io; > > Thanks for the patch! > > I started to have a look at it and figured out that a tiny rebase

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-03-06 Thread Önder Kalacı
Hi all, Thanks for working on this. > I imagine that a typical use case would be to set min_send_delay to > several hours to days. I'm concerned that it could not be an > acceptable trade-off for many users that the system cannot collect any > garbage during that. > I'm not too worried about

Re: a very minor bug and a couple of comment changes for basebackup.c

2023-03-06 Thread Robert Haas
Thanks for the review. I have committed the patches. On Thu, Mar 2, 2023 at 2:59 AM Michael Paquier wrote: > Seems right, I think that you should backpatch that as > VERIFY_CHECKSUMS is the default. Done. > There is more to it: the page LSN is checked before its checksum. > Hence, if the

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-06 Thread Daniel Gustafsson
> On 6 Mar 2023, at 15:55, Tom Lane wrote: > Daniel Gustafsson writes: >> For readers without all context, wouldn't it be better to encode in the >> function name why we're not just calling a hash like md5? Something like >> fips_allowed_hash() or similar? > > I'd prefer shorter than that ---

  1   2   >