Re: BufferAlloc: don't take two simultaneous locks

2022-03-14 Thread Yura Sokolov
В Пн, 14/03/2022 в 14:57 +0300, Yura Sokolov пишет: > В Пн, 14/03/2022 в 17:12 +0900, Kyotaro Horiguchi пишет: > > At Mon, 14 Mar 2022 09:15:11 +0300, Yura Sokolov > > wrote in > > > В Пн, 14/03/2022 в 14:31 +0900, Kyotaro Horiguchi пишет: > > > > I'd like to ask you to remove nalloced from

Re: Column Filtering in Logical Replication

2022-03-14 Thread Amit Kapila
On Mon, Mar 14, 2022 at 7:02 PM Tomas Vondra wrote: > > On 3/14/22 13:47, Amit Kapila wrote: > > On Mon, Mar 14, 2022 at 5:42 PM Tomas Vondra > > wrote: > >> > >> On 3/14/22 12:12, houzj.f...@fujitsu.com wrote: > >>> On Monday, March 14, 2022 5:08 AM Tomas Vondra > >>> wrote: > >> > >> Anyway,

Re: Column Filtering in Logical Replication

2022-03-14 Thread Amit Kapila
On Mon, Mar 14, 2022 at 4:42 PM houzj.f...@fujitsu.com wrote: > > On Monday, March 14, 2022 5:08 AM Tomas Vondra > wrote: > > > > On 3/12/22 05:30, Amit Kapila wrote: > > >> ... > > > > > > Okay, please find attached. I have done basic testing of this, if we > > > agree with this approach then

Re: Issue with pg_stat_subscription_stats

2022-03-14 Thread Masahiko Sawada
On Tue, Mar 15, 2022 at 3:34 AM Melanie Plageman wrote: > > On Mon, Mar 14, 2022 at 4:02 AM Masahiko Sawada wrote: > > > > On Mon, Mar 14, 2022 at 2:05 AM Melanie Plageman > > wrote: > > > > > > On Sat, Mar 12, 2022 at 3:15 PM Andres Freund wrote: > > > > > > > > Hi, > > > > > > > > On

RE: Handle infinite recursion in logical replication setup

2022-03-14 Thread kuroda.hay...@fujitsu.com
Dear Vignesh, > Thanks for kind explanation. > I read above and your doc in 0002, and I put some comments. I forgot a comment about 0002 doc. 5. create_subscription.sgml - about your example Three possibilities were listed in the doc, but I was not sure about b) case. In the situation Node1

Re: Issue with pg_stat_subscription_stats

2022-03-14 Thread Amit Kapila
On Sun, Mar 13, 2022 at 1:45 AM Andres Freund wrote: > > On 2022-03-12 08:28:35 +0530, Amit Kapila wrote: > > On Sat, Mar 12, 2022 at 2:14 AM Melanie Plageman > > wrote: > > > > > > So, I noticed that pg_stat_reset_subscription_stats() wasn't working > > > properly, and, upon further

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Thomas Munro
On Tue, Mar 15, 2022 at 2:50 PM Michael Paquier wrote: > On Tue, Mar 15, 2022 at 02:33:17PM +1300, Thomas Munro wrote: > > As for the complaint about pg_tablespace_location() failing, would it > > be better to return an empty string? That's what was passed in as > > LOCATION. Something like the

Re: Skipping logical replication transactions on subscriber side

2022-03-14 Thread Masahiko Sawada
On Mon, Mar 14, 2022 at 6:50 PM shiy.f...@fujitsu.com wrote: > > On Fri, Mar 11, 2022 4:20 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. This patch can be applied on > > top of the latest disable_on_error patch[1]. > > > > Thanks for your patch. Here are some comments

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2022-03-14 Thread David Rowley
On Sat, 1 Jan 2022 at 15:40, Zhihong Yu wrote: > + locallock->nLocks -= locallockowner->nLocks; > + Assert(locallock->nLocks >= 0); > > I think the assertion is not needed since the above code is in if block : > > + if (locallockowner->nLocks < locallock->nLocks) > > the

Re: [PATCH] Fix various spelling errors

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 06:49:07PM -0500, Justin Pryzby wrote: > On Mon, Mar 14, 2022 at 11:03:50PM +, Kekalainen, Otto wrote: >> I propose the attached patch to be applied on the 'master' branch of >> PostgreSQL >> to fix various spelling errors. >> >> Most fixes are in comments and have no

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 09:37:25PM -0500, Justin Pryzby wrote: > One could argue that most of the pg_ls_* functions aren't needed (including > 1922d7c6e), since the same things are possible with pg_ls_dir() and > pg_stat_file(). > |1922d7c6e Add SQL functions to monitor the directory contents of

Re: Estimating HugePages Requirements?

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 10:34:17AM -0700, Nathan Bossart wrote: > On Mon, Mar 14, 2022 at 04:26:43PM +0100, Magnus Hagander wrote: >> And in fact, the command documented on >> https://www.postgresql.org/docs/devel/kernel-resources.html doesn't >> actually produce the output that the docs show, it

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 01:53:54PM +0900, Michael Paquier wrote: > On Wed, Mar 09, 2022 at 10:50:45AM -0600, Justin Pryzby wrote: > > I also changed pg_ls_dir_recurse() to handle concurrent removal of a dir, > > which > > I noticed caused an infrequent failure on CI. However I'm not including >

RE: Column Filtering in Logical Replication

2022-03-14 Thread shiy.f...@fujitsu.com
On Mon, Mar 14, 2022 5:08 AM Tomas Vondra wrote: > > On 3/12/22 05:30, Amit Kapila wrote: > >> ... > > > > Okay, please find attached. I have done basic testing of this, if we > > agree with this approach then this will require some more testing. > > > > Thanks, the proposed changes seem like a

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 03:54:19PM +0530, Bharath Rupireddy wrote: > At times, the snapshot or mapping files can be large in number and one > some platforms it takes time for checkpoint to process all of them. > Having the stats about them in server logs can help us better analyze > why checkpoint

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 19:43:40 +0400, Pavel Borisov wrote in > > I'd like to add a few quick notes on what's been done in v17. I have some commens by a quick look-through. Apologize in advance for wrong comments from the lack of the knowledge of the whole patch-set. > > Patches 0001 and 0002

RE: Optionally automatically disable logical replication subscriptions on error

2022-03-14 Thread osumi.takami...@fujitsu.com
On Tuesday, March 15, 2022 8:04 AM Nathan Bossart wrote: > My compiler is worried that syncslotname may be used uninitialized in > start_table_sync(). The attached patch seems to silence this warning. Thank you for your reporting ! Your fix looks good to me. Best Regards, Takamichi

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 01:53:54PM +0900, Michael Paquier wrote: > +select * from pg_ls_logicalmapdir() limit 0; > +select * from pg_ls_logicalsnapdir() limit 0; > +select * from pg_ls_replslotdir('') limit 0; > +select * from pg_ls_tmpdir() limit 0; > +select * from pg_ls_waldir() limit 0; >

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-14 Thread Bharath Rupireddy
On Mon, Mar 14, 2022 at 8:25 PM Stephen Frost wrote: > > > As this patch is currently written, pg_monitor has access these > > functions, though I don't think that's the right privilege level at > > least for pg_get_raw_wal_record(). > > Yeah, I agree that pg_monitor isn't the right thing for

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Michael Paquier
On Tue, Mar 15, 2022 at 02:33:17PM +1300, Thomas Munro wrote: > Ok, I pushed the fix for pg_basebackup. > > As for the complaint about pg_tablespace_location() failing, would it > be better to return an empty string? That's what was passed in as > LOCATION. Something like the attached. Hmm, I

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Thomas Munro
On Tue, Mar 15, 2022 at 2:33 PM Thomas Munro wrote: > As for the complaint about pg_tablespace_location() failing, would it > be better to return an empty string? That's what was passed in as > LOCATION. Something like the attached. (Hrrmm, the contract for pgwin32_is_junction() is a little

RE: Handle infinite recursion in logical replication setup

2022-03-14 Thread kuroda.hay...@fujitsu.com
Dear Vignesh, Thank you for updating your patch! > Let's consider an existing Multi master logical replication setup > between Node1 and Node2 that is created using the following steps: > a) Node1 - Publication publishing employee table - pub1 > b) Node2 - Subscription subscribing from

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Thomas Munro
On Tue, Mar 8, 2022 at 4:01 PM Kyotaro Horiguchi wrote: > At Tue, 8 Mar 2022 10:28:46 +0900, Michael Paquier > wrote in > > On Tue, Mar 08, 2022 at 10:06:50AM +0900, Kyotaro Horiguchi wrote: > > > At Tue, 8 Mar 2022 10:39:06 +1300, Thomas Munro > > > wrote in > > >> Thanks, you're right.

Re: [PATCH] Fix various spelling errors

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 11:03:50PM +, Kekalainen, Otto wrote: > Hello! > > I propose the attached patch to be applied on the 'master' branch of > PostgreSQL > to fix various spelling errors. > > Most fixes are in comments and have no effect on functionality. Some fixes are > also in

Re: Optionally automatically disable logical replication subscriptions on error

2022-03-14 Thread Nathan Bossart
My compiler is worried that syncslotname may be used uninitialized in start_table_sync(). The attached patch seems to silence this warning. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 05e4e03af5afa1658ede8d78b31c1c999b5c7deb Mon Sep 17 00:00:00 2001 From: Nathan Bossart

Re: Logical replication timeout problem

2022-03-14 Thread Björn Harrtell
Hi, I have been following this discussion for a while because I believe we are hit by this pretty hard. This sounds very reasonable to me: "Why don't we check both the count and the time? That is, I think we can send a keep-alive either if we skipped 1 changes or if we didn't sent anything

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-14 Thread Robert Haas
On Mon, Mar 7, 2022 at 3:39 AM Kyotaro Horiguchi wrote: > So the new framework has been dropped in this version. > The second test is removed as it is irrelevant to this bug. > > In this version the patch is a single file that contains the test. The status of this patch in the CommitFest was set

Re: document the need to analyze partitioned tables

2022-03-14 Thread Robert Haas
On Fri, Jan 21, 2022 at 1:31 PM Tomas Vondra wrote: > [ new patch ] This patch is originally by Justin. The latest version is by Tomas. I think the next step is for Justin to say whether he's OK with the latest version that Tomas posted. If he is, then I suggest that he also mark it Ready for

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2022-03-14 Thread Robert Haas
On Fri, Jan 28, 2022 at 9:51 PM Julien Rouhaud wrote: > On Fri, Jan 28, 2022 at 10:20:07AM -0800, Andres Freund wrote: > > On 2022-01-28 21:56:36 +0800, Julien Rouhaud wrote: > > > I think having a new option for vacuumdb is the right move. > > > > Can't we pass the option via the connection

Re: Query about time zone patterns in to_char

2022-03-14 Thread Robert Haas
On Fri, Jul 9, 2021 at 10:44 AM Tomas Vondra wrote: > Yeah, does not seem to be worth it, as there seem to be no actual > reports of issues in the field. > > FWIW there seem to be quite a bit of other to_char differences compared > to Oracle (judging by docs and playing with sqlfiddle). But the

Re: generic plans and "initial" pruning

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 3:38 PM Tom Lane wrote: > ... like EXPLAIN, for example? Exactly! I think that's the foremost example, but extension modules like auto_explain or even third-party extensions are also a risk. I think there was some discussion of this previously. > If "pruning" means

Re: generic plans and "initial" pruning

2022-03-14 Thread Tom Lane
Robert Haas writes: > In my opinion, the most important theoretical issue here is around > reuse of plans that are no longer entirely valid, but the parts that > are no longer valid are certain to be pruned. If, because we know that > some parameter has some particular value, we skip locking a

Re: Fix uninitialized variable access (src/backend/utils/mmgr/freepage.c)

2022-03-14 Thread Peter Geoghegan
On Mon, Mar 14, 2022 at 12:15 PM Robert Haas wrote: > If surgery does happen in the future, I would argue that this > change could easily *mask* bugs, because if somebody tries to apply > valgrind to this code the useless initializations will just cover up > what valgrind would otherwise detect

Re: Fix uninitialized variable access (src/backend/utils/mmgr/freepage.c)

2022-03-14 Thread Robert Haas
On Thu, Jan 27, 2022 at 7:33 AM Greg Nancarrow wrote: > Why not, at least, just add "Assert(result.page != NULL);" after the > "Assert(!result.found);" in FreePageManagerPutInternal()? > The following code block in FreePageBtreeSearch() - which lacks those > initializations - should never be

Re: ICU for global collation

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 8:51 AM Peter Eisentraut wrote: > [ new patches ] I'm not very knowledgeable about this topic, but to me, it seems confusing to think of having both a libc collation and an ICU collation associated with a database. I have two main questions: 1. What will happen if I set

Re: generic plans and "initial" pruning

2022-03-14 Thread Robert Haas
On Fri, Mar 11, 2022 at 9:35 AM Amit Langote wrote: > Attached is v5, now broken into 3 patches: > > 0001: Some refactoring of runtime pruning code > 0002: Add a plan_tree_walker > 0003: Teach AcquireExecutorLocks to skip locking pruned relations So is any other committer planning to look at

Re: Issue with pg_stat_subscription_stats

2022-03-14 Thread Melanie Plageman
On Mon, Mar 14, 2022 at 4:02 AM Masahiko Sawada wrote: > > On Mon, Mar 14, 2022 at 2:05 AM Melanie Plageman > wrote: > > > > On Sat, Mar 12, 2022 at 3:15 PM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2022-03-12 08:28:35 +0530, Amit Kapila wrote: > > > > On Sat, Mar 12, 2022 at 2:14 AM

Re: Lowering the ever-growing heap->pd_lower

2022-03-14 Thread Peter Geoghegan
On Thu, Mar 10, 2022 at 5:49 AM Matthias van de Meent wrote: > I double-checked the changes, and to me it seems like that was the > only place in the code where PageGetMaxOffsetNumber was not handled > correctly. This was fixed in the latest patch (v8). > > Peter, would you have time to further

Re: obsolete reference to a SubPlan field

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 1:45 PM Tom Lane wrote: > Robert Haas writes: > > On Wed, Feb 2, 2022 at 3:08 AM Amit Langote wrote: > >> Attached patch removes those. > > > Looks right to me. Tom, any comments? > > I'm pretty sure I left those comments alone on purpose back in 2007, > and I don't find

Re: obsolete reference to a SubPlan field

2022-03-14 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 2, 2022 at 3:08 AM Amit Langote wrote: >> Attached patch removes those. > Looks right to me. Tom, any comments? I'm pretty sure I left those comments alone on purpose back in 2007, and I don't find simply removing them to be an improvement. In principle,

Re: Estimating HugePages Requirements?

2022-03-14 Thread Nathan Bossart
On Mon, Mar 14, 2022 at 04:26:43PM +0100, Magnus Hagander wrote: > Nothing fixing this ended up actually getting committed, right? That > is, we still get the extra log output? Correct. > And in fact, the command documented on > https://www.postgresql.org/docs/devel/kernel-resources.html doesn't

Re: obsolete reference to a SubPlan field

2022-03-14 Thread Robert Haas
On Wed, Feb 2, 2022 at 3:08 AM Amit Langote wrote: > I noticed $subject while looking at something involving SubLinks and > SubPlans. It seems eab6b8b27eb removed the "plan" field from the > SubPlan node struct definition, but the following line from > expression_tree_mutator(): > >

Re: refactoring basebackup.c (zstd workers)

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 1:11 PM Justin Pryzby wrote: > Internally, I was thinking they'd all be handled as first-class options, with > separate struct fields and separate replication protocol options. If an > option > isn't known, it'd be rejected on the client side, rather than causing an

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-03-14 Thread Andres Freund
Hi A random thought I had while thinking about the size limits: We could use the low bits of the length and xl_prev to store XLR_SPECIAL_REL_UPDATE | XLR_CHECK_CONSISTENCY and give rmgrs the full 8 bit of xl_info. Which would allow us to e.g. get away from needing Heap2. Which would aestethically

Re: refactoring basebackup.c (zstd workers)

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 01:02:20PM -0400, Robert Haas wrote: > On Mon, Mar 14, 2022 at 12:35 PM Justin Pryzby wrote: > > I suggest to use a syntax that's more general than that, maybe something > > like > > > > :[level=]N,parallel=N,flag,flag,... > > > > For example, someone may want to use zstd

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-14 Thread Nathan Bossart
On Mon, Mar 14, 2022 at 05:03:59PM +0530, Bharath Rupireddy wrote: > On Mon, Mar 14, 2022 at 1:33 PM Michael Paquier wrote: >> On Mon, Mar 14, 2022 at 10:54:56AM +0530, Bharath Rupireddy wrote: >> > Yes, this is a concern. Also, when there were no logical replication >> > slots on a plain server

Re: refactoring basebackup.c (zstd workers)

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 12:35 PM Justin Pryzby wrote: > I suggest to use a syntax that's more general than that, maybe something like > > :[level=]N,parallel=N,flag,flag,... > > For example, someone may want to use zstd "long" mode or (when it's released) > rsyncable mode, or specify fine-grained

Re: Allow async standbys wait for sync replication

2022-03-14 Thread Andres Freund
Hi, On 2022-03-14 11:30:02 +0900, Kyotaro Horiguchi wrote: > That is, (as my understanding) async standbys are required to allow > overwriting existing unreplayed records after reconnection. But, > putting aside how to remember that LSN, if that happens at a segment > boundary, the async replica

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-03-14 Thread Matthias van de Meent
Thank you all for the feedback. Please find attached v2 of the patchset, which contains updated comments and applies the suggested changes. On Sat, 12 Mar 2022 at 02:03, Andres Freund wrote: > > Hi, > > On 2022-03-11 22:42:42 +0200, Heikki Linnakangas wrote: > > Have you been able to create a

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 12:44 PM Dilip Kumar wrote: > On Mon, Mar 14, 2022 at 10:04 PM Robert Haas wrote: > > Regarding 0004, I can't really see a reason for this function to take > > a LockRelId as a parameter rather than two separate OIDs. I also can't > > entirely see why it should be called

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-14 Thread Dilip Kumar
On Mon, Mar 14, 2022 at 10:04 PM Robert Haas wrote: > > Regarding 0004, I can't really see a reason for this function to take > a LockRelId as a parameter rather than two separate OIDs. I also can't > entirely see why it should be called LockRelationId. Maybe > LockRelationInDatabaseById(Oid

Re: refactoring basebackup.c (zstd workers)

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 09:41:35PM +0530, Dipesh Pandit wrote: > I tried to implement support for parallel ZSTD compression. The > library provides an option (ZSTD_c_nbWorkers) to specify the > number of compression workers. The number of parallel > workers can be set as part of compression

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 12:04 PM Robert Haas wrote: > On Mon, Mar 14, 2022 at 7:51 AM Dilip Kumar wrote: > > Other than that, I have fixed some mistakes in comments and supported > > tab completion for the new options. > > I was looking at 0001 and 0002 again and realized that I swapped the >

Re: Add index scan progress to pg_stat_progress_vacuum

2022-03-14 Thread Imseih (AWS), Sami
>I'm still unsure the current design of 0001 patch is better than other >approaches we’ve discussed. Even users who don't use parallel vacuum >are forced to allocate shared memory for index vacuum progress, with >GetMaxBackends() entries from the beginning. Also, it’s likely to >

Re: [PATCH] Add reloption for views to enable RLS

2022-03-14 Thread Laurenz Albe
On Mon, 2022-03-14 at 13:40 +0100, Christoph Heiss wrote: > On 3/9/22 16:06, Laurenz Albe wrote: > > This paragraph contains a couple of grammatical errors. > > Replaced the two paragraphs with your suggestion, it is indeed easier to > read. > > > Also, this: > > could be written like this

Re: refactoring basebackup.c

2022-03-14 Thread Dipesh Pandit
Hi, I tried to implement support for parallel ZSTD compression. The library provides an option (ZSTD_c_nbWorkers) to specify the number of compression workers. The number of parallel workers can be set as part of compression parameter and if this option is specified then the library performs

Re: On login trigger: take three

2022-03-14 Thread Andres Freund
Hi, On 2022-03-14 14:47:09 +0100, Daniel Gustafsson wrote: > > On 14 Mar 2022, at 01:08, Andres Freund wrote: > > > I was thinking that the way to use it would be to specify it as a client > > option. Like PGOPTIONS='-c ignore_event_trigger=login' psql. > > Attached is a quick PoC/sketch of

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 7:51 AM Dilip Kumar wrote: > Other than that, I have fixed some mistakes in comments and supported > tab completion for the new options. I was looking at 0001 and 0002 again and realized that I swapped the names load_relmap_file() and read_relmap_file() from what I should

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Dmitry Dolgov
> On Mon, Mar 14, 2022 at 11:38:23AM -0400, Tom Lane wrote: > Dmitry Dolgov <9erthali...@gmail.com> writes: > > On Mon, Mar 14, 2022 at 11:23:17AM -0400, Tom Lane wrote: > >> I do find it odd that the proposed patch doesn't cause the *entire* > >> list to be skipped over. That seems like extra

Re: role self-revocation

2022-03-14 Thread Mark Dilger
> On Mar 14, 2022, at 7:38 AM, Stephen Frost wrote: > > So ... do you feel like that's now the case? Or were you looking for > more? I don't have any more questions at the moment. Thanks! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Pavel Borisov
> > Hi, Hackers! > >> Hi! Here is updated version of the patch, based on Alexander's ver16. >> > I'd like to add a few quick notes on what's been done in v17. > > Patches 0001 and 0002 that are planned to be committed to PG15 are almost > unchanged with the exception of one unnecessary cast in

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Tom Lane
Dmitry Dolgov <9erthali...@gmail.com> writes: > On Mon, Mar 14, 2022 at 11:23:17AM -0400, Tom Lane wrote: >> I do find it odd that the proposed patch doesn't cause the *entire* >> list to be skipped over. That seems like extra complexity and confusion >> to no benefit. > That's a bit surprising

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Dmitry Dolgov
> On Mon, Mar 14, 2022 at 11:23:17AM -0400, Tom Lane wrote: > Robert Haas writes: > > I do find it odd that the proposed patch doesn't cause the *entire* > list to be skipped over. That seems like extra complexity and confusion > to no benefit. That's a bit surprising for me, I haven't even

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-14 Thread Robert Haas
On Sat, Mar 12, 2022 at 12:36 AM Dilip Kumar wrote: > So here my stand is that we need to drop database buffers and remove > pending sync requests because we are deleting underlying files and if > we do not do that in some extreme cases then there is no need to drop > the buffers or remove the

Re: Estimating HugePages Requirements?

2022-03-14 Thread Magnus Hagander
On Thu, Sep 9, 2021 at 11:53 PM Bossart, Nathan wrote: > > On 9/8/21, 9:19 PM, "Michael Paquier" wrote: > > FWIW, I don't have an environment at hand these days to test properly > > 0001, so this will have to wait a bit. I really like the approach > > taken by 0002, and it is independent of the

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 14, 2022 at 10:57 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: >> I'm not sure if I follow the last point. WHERE x in (1,3) and x = >> any(array[1,3]) are two different things for sure, but in which way are >> they going to be mixed together because of this

Re: support for MERGE

2022-03-14 Thread Japin Li
On Sat, 12 Mar 2022 at 11:53, Alvaro Herrera wrote: > On 2022-Mar-09, Zhihong Yu wrote: > >> Hi, >> For v14-0002-Split-ExecUpdate-and-ExecDelete-in-reusable-piec.patch : >> >> + TupleTableSlot *insertProjectedTuple; >> >> With `insert` at the beginning of the variable name, someone may think

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Dmitry Dolgov
> On Mon, Mar 14, 2022 at 11:02:16AM -0400, Robert Haas wrote: > On Mon, Mar 14, 2022 at 10:57 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Well, yeah, the commit message is somewhat clumsy in this regard. It > > works almost in the way you've described, except if the list is all > >

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Robert Haas
On Mon, Mar 14, 2022 at 10:57 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Well, yeah, the commit message is somewhat clumsy in this regard. It > works almost in the way you've described, except if the list is all > constants and long enough to satisfy the threshold then *first N > elements

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Dmitry Dolgov
> On Mon, Mar 14, 2022 at 10:17:57AM -0400, Robert Haas wrote: > On Sat, Mar 12, 2022 at 9:11 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Here is the limited version of list collapsing functionality, which > > doesn't utilize eval_const_expressions and ignores most of the stuff > > except

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-14 Thread Stephen Frost
Greetings, * Jeff Davis (pg...@j-davis.com) wrote: > On Thu, 2022-03-10 at 15:54 -0500, Stephen Frost wrote: > > The standard is basically that all of the functions it brings are > > written to enforce the PG privilege system and you aren't able to use > > the extension to bypass those

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Pavel Borisov
Hi, Hackers! > Hi! Here is updated version of the patch, based on Alexander's ver16. > I'd like to add a few quick notes on what's been done in v17. Patches 0001 and 0002 that are planned to be committed to PG15 are almost unchanged with the exception of one unnecessary cast in 0002 removed.

Re: role self-revocation

2022-03-14 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Mar 11, 2022, at 4:56 PM, Stephen Frost wrote: > > First … I outlined a fair bit of further description in the message you > > just responded to but neglected to include in your response, which strikes > > me as odd that

Re: support for MERGE

2022-03-14 Thread Amit Langote
On Sun, Mar 13, 2022 at 12:36 AM Alvaro Herrera wrote: > FYI I intend to get the ModifyTable split patch (0001+0003) pushed > hopefully on Tuesday or Wednesday next week, unless something really > ugly is found on it. I looked at 0001 and found a few things that could perhaps be improved. +

Re: pg_stat_statements and "IN" conditions

2022-03-14 Thread Robert Haas
On Sat, Mar 12, 2022 at 9:11 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Here is the limited version of list collapsing functionality, which > doesn't utilize eval_const_expressions and ignores most of the stuff > except ArrayExprs. Any thoughts/more suggestions? The proposed commit message

Re: On login trigger: take three

2022-03-14 Thread Daniel Gustafsson
> On 14 Mar 2022, at 01:08, Andres Freund wrote: > I was thinking that the way to use it would be to specify it as a client > option. Like PGOPTIONS='-c ignore_event_trigger=login' psql. Attached is a quick PoC/sketch of such a patch, where 0001 adds a guc, 0002 is the previously posted v25

Re: Column Filtering in Logical Replication

2022-03-14 Thread Tomas Vondra
On 3/14/22 13:47, Amit Kapila wrote: > On Mon, Mar 14, 2022 at 5:42 PM Tomas Vondra > wrote: >> >> On 3/14/22 12:12, houzj.f...@fujitsu.com wrote: >>> On Monday, March 14, 2022 5:08 AM Tomas Vondra >>> wrote: >> >> Anyway, the fix does not address tablesync, as explained in [1]. I'm not >>

Re: refactoring basebackup.c

2022-03-14 Thread Robert Haas
On Fri, Mar 11, 2022 at 8:52 PM Andres Freund wrote: > You could also just append a manifest as a compresed tar to the compressed tar > stream. Unfortunately GNU tar requires -i to read concated compressed > archives, so perhaps that's not quite an alternative. s/Unfortunately/Fortunately/ :-p

Re: On login trigger: take three

2022-03-14 Thread Robert Haas
On Sun, Mar 13, 2022 at 7:34 PM Andres Freund wrote: > IMO the other types of event triggers make it a heck of a lot harder to get > yourself into a situation that you can't get out of... In particular, unless something has changed since I committed this stuff originally, there's no existing

Re: ICU for global collation

2022-03-14 Thread Peter Eisentraut
On 05.03.22 09:38, Julien Rouhaud wrote: I say it works because I did manually check, as far as I can see there isn't any test that ensures it. I'm using this naive scenario: DROP DATABASE IF EXISTS dbicu; CREATE DATABASE dbicu LOCALE_PROVIDER icu LOCALE 'en_US' ICU_LOCALE 'en-u-kf-upper'

Re: Column Filtering in Logical Replication

2022-03-14 Thread Amit Kapila
On Mon, Mar 14, 2022 at 5:42 PM Tomas Vondra wrote: > > On 3/14/22 12:12, houzj.f...@fujitsu.com wrote: > > On Monday, March 14, 2022 5:08 AM Tomas Vondra > > wrote: > > Anyway, the fix does not address tablesync, as explained in [1]. I'm not > sure what to do about it - in principle, we could

Re: [PATCH] Add reloption for views to enable RLS

2022-03-14 Thread Christoph Heiss
On 3/9/22 16:06, Laurenz Albe wrote: This paragraph contains a couple of grammatical errors. How about Note that the user performing the insert, update or delete on the view must have the corresponding insert, update or delete privilege on the view. Unless security_invoker is

RE: Skipping logical replication transactions on subscriber side

2022-03-14 Thread osumi.takami...@fujitsu.com
On Friday, March 11, 2022 5:20 PM Masahiko Sawada wrote: > I've attached an updated version patch. This patch can be applied on top of > the > latest disable_on_error patch[1]. Hi, few extra comments on v13. (1) src/backend/replication/logical/worker.c With regard to

Re: Tab completion not listing schema list for create/alter publication for all tables in schema

2022-03-14 Thread vignesh C
On Mon, Mar 14, 2022 at 5:23 AM Tom Lane wrote: > > vignesh C writes: > > Here "pg_%" should be "pg_%%". > > Right you are. Patch pushed, thanks! Thanks for pushing the patch. Regards, Vignesh

Re: Column Filtering in Logical Replication

2022-03-14 Thread Tomas Vondra
On 3/14/22 12:12, houzj.f...@fujitsu.com wrote: > On Monday, March 14, 2022 5:08 AM Tomas Vondra > wrote: >> >> On 3/12/22 05:30, Amit Kapila wrote: ... >>> >>> Okay, please find attached. I have done basic testing of this, if we >>> agree with this approach then this will require some

Re: Column Filtering in Logical Replication

2022-03-14 Thread Tomas Vondra
On 3/14/22 10:53, Amit Kapila wrote: > On Mon, Mar 14, 2022 at 2:37 AM Tomas Vondra > wrote: >> >> On 3/12/22 05:30, Amit Kapila wrote: ... >>> >>> Okay, please find attached. I have done basic testing of this, if we >>> agree with this approach then this will require some more testing.

Re: BufferAlloc: don't take two simultaneous locks

2022-03-14 Thread Yura Sokolov
В Пн, 14/03/2022 в 17:12 +0900, Kyotaro Horiguchi пишет: > At Mon, 14 Mar 2022 09:15:11 +0300, Yura Sokolov > wrote in > > В Пн, 14/03/2022 в 14:31 +0900, Kyotaro Horiguchi пишет: > > > I'd like to ask you to remove nalloced from partitions then add a > > > global atomic for the same use? > >

RE: Optionally automatically disable logical replication subscriptions on error

2022-03-14 Thread osumi.takami...@fujitsu.com
On Monday, March 14, 2022 7:49 PM Amit Kapila wrote: > On Thu, Mar 10, 2022 at 12:04 PM Amit Kapila > wrote: > > > > On Wed, Mar 9, 2022 at 7:57 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > Hi, attached v32 removed my additional code for > maybe_reread_subscription. > > > > > > >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-14 Thread Dilip Kumar
On Sat, Mar 12, 2022 at 11:06 AM Dilip Kumar wrote: > > In fact while we are rewriting the relation during vacuum full that > time also we are calling log_newpage() under RelationCopyStorage() and > during standby if it gets promoted we will be having some buffers in > the buffer pool with the

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-14 Thread Julien Rouhaud
On Mon, Mar 14, 2022 at 03:16:50PM +0530, Nitin Jadhav wrote: > > > I am not suggesting > > > removing the existing 'flags' field of pg_stat_progress_checkpoint > > > view and adding a new field 'throttled'. The content of the 'flags' > > > field remains the same. I was suggesting replacing the

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-14 Thread Bharath Rupireddy
On Mon, Mar 14, 2022 at 1:33 PM Michael Paquier wrote: > > On Mon, Mar 14, 2022 at 10:54:56AM +0530, Bharath Rupireddy wrote: > > Yes, this is a concern. Also, when there were no logical replication > > slots on a plain server or the server removed or cleaned up all the > > snapshot/mappings

RE: Column Filtering in Logical Replication

2022-03-14 Thread houzj.f...@fujitsu.com
On Monday, March 14, 2022 5:08 AM Tomas Vondra wrote: > > On 3/12/22 05:30, Amit Kapila wrote: > >> ... > > > > Okay, please find attached. I have done basic testing of this, if we > > agree with this approach then this will require some more testing. > > > > Thanks, the proposed changes seem

Re: Optionally automatically disable logical replication subscriptions on error

2022-03-14 Thread Amit Kapila
On Thu, Mar 10, 2022 at 12:04 PM Amit Kapila wrote: > > On Wed, Mar 9, 2022 at 7:57 PM osumi.takami...@fujitsu.com > wrote: > > > > Hi, attached v32 removed my additional code for maybe_reread_subscription. > > > > Thanks, the patch looks good to me. I have made minor edits in the > attached. I

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-14 Thread Aleksander Alekseev
Hi hackers, > > Here is a new version of the patchset. SLRU refactoring was moved to a > > separate patch. Both v14-0003 (XID_FMT macro) and v14-0004 (SLRU > > refactoring) can be delivered in PG15. > > Here is a new version of the patchset. The changes compared to v14 are > minimal. Most

Re: Implementing Incremental View Maintenance

2022-03-14 Thread Yugo NAGATA
Hello Zhihong Yu, I already replied to your comments before, but I forgot to include the list to CC, so I resend the same again. Sorry for the duplicate emails. On Thu, 3 Feb 2022 09:51:52 -0800 Zhihong Yu wrote: > For CreateIndexOnIMMV(): > > + ereport(NOTICE, > +

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-03-14 Thread Bharath Rupireddy
On Mon, Mar 14, 2022 at 1:33 PM Michael Paquier wrote: > > On Mon, Mar 14, 2022 at 10:54:56AM +0530, Bharath Rupireddy wrote: > > Yes, this is a concern. Also, when there were no logical replication > > slots on a plain server or the server removed or cleaned up all the > > snapshot/mappings

Re: Column Filtering in Logical Replication

2022-03-14 Thread Amit Kapila
On Mon, Mar 14, 2022 at 2:37 AM Tomas Vondra wrote: > > On 3/12/22 05:30, Amit Kapila wrote: > >> ... > > > > Okay, please find attached. I have done basic testing of this, if we > > agree with this approach then this will require some more testing. > > > > Thanks, the proposed changes seem like

RE: Skipping logical replication transactions on subscriber side

2022-03-14 Thread shiy.f...@fujitsu.com
On Fri, Mar 11, 2022 4:20 PM Masahiko Sawada wrote: > > I've attached an updated version patch. This patch can be applied on > top of the latest disable_on_error patch[1]. > Thanks for your patch. Here are some comments for the v13 patch. 1. doc/src/sgml/ref/alter_subscription.sgml +

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-14 Thread Nitin Jadhav
> > I am not suggesting > > removing the existing 'flags' field of pg_stat_progress_checkpoint > > view and adding a new field 'throttled'. The content of the 'flags' > > field remains the same. I was suggesting replacing the 'next_flags' > > field with 'throttled' field since the new request with

Re: simplifying foreign key/RI checks

2022-03-14 Thread Zhihong Yu
On Mon, Mar 14, 2022 at 1:33 AM Amit Langote wrote: > On Tue, Jan 18, 2022 at 3:30 PM Amit Langote > wrote: > > v13 is attached. > > I noticed that the recent 641f3dffcdf's changes to > get_constraint_index() made it basically unusable for this patch's > purposes. > > Reading in the thread that

  1   2   >