Re: Added schema level support for publication.

2021-11-02 Thread vignesh C
On Wed, Nov 3, 2021 at 11:07 AM houzj.f...@fujitsu.com wrote: > > On Wed, Nov 3, 2021 12:25 PM vignesh C wrote: > > On Wed, Nov 3, 2021 at 8:30 AM Amit Kapila > > wrote: > > > > > > On Tue, Nov 2, 2021 at 8:13 PM Tomas Vondra > > > wrote: > > > > > > > > > > > > > > > > > > Yeah, that is also t

Re: Added schema level support for publication.

2021-11-02 Thread vignesh C
On Wed, Nov 3, 2021 at 11:37 AM Peter Smith wrote: > > While you are changing these, maybe also change: > > Before: PUBLICATIONOBJ..._CURRSCHEMA > After: PUBLICATIONOBJ..._CUR_SCHEMA > > Which seems more readable to me. Thanks for the comment, the attached patch has the changes for the same. Reg

Re: Added schema level support for publication.

2021-11-02 Thread Peter Smith
While you are changing these, maybe also change: Before: PUBLICATIONOBJ..._CURRSCHEMA After: PUBLICATIONOBJ..._CUR_SCHEMA Which seems more readable to me. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: emit recovery stats via a new file or a new hook

2021-11-02 Thread Amit Kapila
On Mon, Nov 1, 2021 at 12:30 AM Andres Freund wrote: > > I'm not sure that the new log messages aren't sufficient. But if they > aren't, it seems better to keep additional data in the stats system, and > make them visible via views, rather than adding yet another place to > keep stats. > +1. This

Re: Logical insert/update/delete WAL records for custom table AMs

2021-11-02 Thread Amit Kapila
On Sun, Oct 31, 2021 at 11:40 PM Jeff Davis wrote: > > Attached is a WIP patch to add new WAL records to represent a logical > insert, update, or delete. These records do not do anything at REDO > time, they are only processed during logical decoding/replication. > > These are intended to be used

RE: Added schema level support for publication.

2021-11-02 Thread houzj.f...@fujitsu.com
On Wed, Nov 3, 2021 12:25 PM vignesh C wrote: > On Wed, Nov 3, 2021 at 8:30 AM Amit Kapila > wrote: > > > > On Tue, Nov 2, 2021 at 8:13 PM Tomas Vondra > > wrote: > > > > > > > > > > > > > > Yeah, that is also true. So maybe at this, we can just rename the > > > > few types as suggested by you a

Re: [PATCH] Native spinlock support on RISC-V

2021-11-02 Thread Thomas Munro
On Wed, Nov 3, 2021 at 5:13 PM Andres Freund wrote: > Any chance you could enable jit_dump_bitcode and manually try a failing > query? That should dump. bc files in the data directory. That'd might allow > debugging this outside the emulated environment. > > I don't see where the undef is origin

Re: Added schema level support for publication.

2021-11-02 Thread vignesh C
On Wed, Nov 3, 2021 at 8:30 AM Amit Kapila wrote: > > On Tue, Nov 2, 2021 at 8:13 PM Tomas Vondra > wrote: > > > > > > > > > > Yeah, that is also true. So maybe at this, we can just rename the few > > > types as suggested by you and we can look at it later if we anytime > > > have more number of

Re: [PATCH] Native spinlock support on RISC-V

2021-11-02 Thread Andres Freund
Hi, On November 2, 2021 3:55:58 PM PDT, Thomas Munro wrote: >2. When configured with all options on FreeBSD 13, everything looks >good so far except LLVM JIT, which fails with various "Cannot select" >errors. Clang works fine for compiling PostgreSQL itself. Tested >with LLVM 12 (LLVM has sup

Re: parallel vacuum comments

2021-11-02 Thread Amit Kapila
On Tue, Nov 2, 2021 at 11:17 AM Masahiko Sawada wrote: > > On Tue, Nov 2, 2021 at 5:57 AM Peter Geoghegan wrote: > > > > > Rather than inventing PARALLEL_VACUUM_KEY_INDVAC_CHECK (just for > > assert-enabled builds), we should invent PARALLEL_VACUUM_STATS -- a > > dedicated shmem area for the arra

Re: Skipping logical replication transactions on subscriber side

2021-11-02 Thread Amit Kapila
On Tue, Nov 2, 2021 at 2:17 PM Masahiko Sawada wrote: > > On Tue, Nov 2, 2021 at 2:35 PM Amit Kapila wrote: > > > > On Mon, Nov 1, 2021 at 7:18 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Oct 29, 2021 at 8:20 PM Amit Kapila > > > wrote: > > > > > > > > > > > > Fair enough. So statistics

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Jan Wieck
On 11/2/21 20:02, Tom Lane wrote: My objection to log_checkpoints=on is that that's going to produce a constant stream of messages even when *nothing at all* is wrong. Worse yet, a novice DBA would likely have a hard time understanding from those messages whether there was anything to worry abou

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-11-02 Thread Amit Kapila
On Thu, Oct 28, 2021 at 8:15 AM Amit Kapila wrote: > > On Wed, Oct 27, 2021 at 4:39 PM Dilip Kumar wrote: > > > > On Tue, Oct 26, 2021 at 9:19 AM Amit Kapila wrote: > > > > > > > > > > > Thanks, both your patches look good to me except that we need to > > > > remove the sentence related to the r

Re: Added schema level support for publication.

2021-11-02 Thread Amit Kapila
On Tue, Nov 2, 2021 at 8:13 PM Tomas Vondra wrote: > > > > > > Yeah, that is also true. So maybe at this, we can just rename the few > > types as suggested by you and we can look at it later if we anytime > > have more number of objects to add. > > > > +1 > Apart from what you have pointed above,

Re: parallel vacuum comments

2021-11-02 Thread Masahiko Sawada
On Wed, Nov 3, 2021 at 11:53 AM Peter Geoghegan wrote: > > On Tue, Nov 2, 2021 at 7:35 PM Masahiko Sawada wrote: > > It returns true in the above condition but it should return false > > since the index doesn't support parallel index cleanup at all. It > > seems that this bug was introduced by co

Re: parallel vacuum comments

2021-11-02 Thread Peter Geoghegan
On Tue, Nov 2, 2021 at 7:35 PM Masahiko Sawada wrote: > It returns true in the above condition but it should return false > since the index doesn't support parallel index cleanup at all. It > seems that this bug was introduced by commit b4af70cb21 (therefore > exists only in PG14) which flipped th

Re: parallel vacuum comments

2021-11-02 Thread Masahiko Sawada
On Tue, Nov 2, 2021 at 2:46 PM Masahiko Sawada wrote: > > Anyway, I'll write a patch accordingly. While writing a patch for these comments, I found another bug in parallel_processing_is_safe(): /* * Returns false, if the given index can't participate in parallel index * vacuum or parallel inde

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Nikolay Samokhvalov
On Tue, Nov 2, 2021 at 5:02 PM Tom Lane wrote: > I'm still of the position that the default ought to be that a > normally-functioning server generates no ongoing log output. > Only people who have got Nagios watching their logs, or some > such setup, are going to want anything different. And tha

Re: AArch64 has single-copy 64 bit atomicity

2021-11-02 Thread Alexander Korotkov
On Wed, Nov 3, 2021 at 1:34 AM Thomas Munro wrote: > Andres mentioned in passing that he'd defined > PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY only on Itanium, PPC and x86 but > not ARM. > > I took a look at https://developer.arm.com/documentation/ddi0487/gb/ > under "B2.2.1 Requirements for single-copy

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Isaac Morland
On Tue, 2 Nov 2021 at 19:00, Vik Fearing wrote: > On 11/2/21 11:14 PM, Vik Fearing wrote: > > > This would be nice, but there is nothing to hang our hat on: > > > > GRANT CHECKPOINT TO username; > > Thinking about this more, why don't we just add CHECKPOINT and > NOCHECKPOINT attributes to ro

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Tom Lane
Bruce Momjian writes: > On Sun, Oct 31, 2021 at 10:24:38PM +0100, Michael Banck wrote: >> It could be a bit of reverse-survivorship-bias, i.e., you're only seeing >> the pathological cases, while 99% of the Postgres installations out >> there just hum along fine without anybody ever having to look

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Bruce Momjian
On Sun, Oct 31, 2021 at 10:24:38PM +0100, Michael Banck wrote: > On Sun, Oct 31, 2021 at 01:16:33PM -0700, Andres Freund wrote: > > Shrug. It's based on many years of doing or being around people doing > > postgres support escalation shifts. And it's not like log_checkpoints > > incurs meaningful o

Re: enabling FOO=bar arguments to vcregress.pl

2021-11-02 Thread Michael Paquier
On Tue, Nov 02, 2021 at 05:59:49PM -0400, Andrew Dunstan wrote: > On 11/2/21 11:03, Andrew Dunstan wrote: >> I think you misunderstood the purpose of my email. It wasn't meant to >> be complete patch. >> >> >> But here's an untested patch that should do almost all of what I want >> for USE_MODULE_D

Re: Teach pg_receivewal to use lz4 compression

2021-11-02 Thread Michael Paquier
On Tue, Nov 02, 2021 at 12:31:47PM -0400, Robert Haas wrote: > On Tue, Nov 2, 2021 at 8:17 AM Magnus Hagander wrote: >> I think for the end user, it is strictly better to name it "gzip", >> and given that the target of this option is the end user we should >> do so. (It'd be different it we were t

Re: [PATCH] postgres_fdw: suppress explicit casts in text:text comparisons (was: column option to override foreign types)

2021-11-02 Thread Tom Lane
I wrote: > Now that I've looked this over I'm starting to feel uncomfortable > again, because we can't actually be quite sure about how the remote > parser's heuristic will act. Actually ... we could make that a lot safer by insisting that the other input be a plain Var, which'd necessarily be a c

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Vik Fearing
On 11/2/21 11:14 PM, Vik Fearing wrote: > This would be nice, but there is nothing to hang our hat on: > > GRANT CHECKPOINT TO username; Thinking about this more, why don't we just add CHECKPOINT and NOCHECKPOINT attributes to roles? ALTER ROLE username WITH CHECKPOINT; -- Vik Fearing

Re: [PATCH] Native spinlock support on RISC-V

2021-11-02 Thread Thomas Munro
On Wed, Sep 1, 2021 at 9:22 PM Christoph Berg wrote: > Re: Tom Lane > > I did not like confusing the RISC-V case with the ARM case: duplicating > > the code block seems better, in case there's ever reason to add > > arch-specific stuff like SPIN_DELAY. So I split it off to its own > > code block

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Isaac Morland
On Tue, 2 Nov 2021 at 18:14, Vik Fearing wrote: > On 11/2/21 4:06 PM, Robert Haas wrote: > > There's bound to be somebody who wants to grant some of > > these permissions and not others, or who wants to grant the ability to > > run those commands on some tables but not others. > Is there anything

Re: [PATCH] postgres_fdw: suppress explicit casts in text:text comparisons (was: column option to override foreign types)

2021-11-02 Thread Tom Lane
"Dian M Fay" writes: > Thanks Tom, that makes way more sense! I've attached a new patch which > tests operands and makes sure one side is a Const before feeding it to > deparseConst with a new showtype code, -2. The one regression is gone, > but I've left a couple of test output discrepancies for

AArch64 has single-copy 64 bit atomicity

2021-11-02 Thread Thomas Munro
Hi, Andres mentioned in passing that he'd defined PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY only on Itanium, PPC and x86 but not ARM. I took a look at https://developer.arm.com/documentation/ddi0487/gb/ under "B2.2.1 Requirements for single-copy atomicity" and it seemed like we should turn this on for

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread David G. Johnston
On Tue, Nov 2, 2021 at 3:14 PM Vik Fearing wrote: > On 11/2/21 4:06 PM, Robert Haas wrote: > > There's bound to be somebody who wants to grant some of > > these permissions and not others, or who wants to grant the ability to > > run those commands on some tables but not others. > Is there anythi

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Vik Fearing
On 11/2/21 4:06 PM, Robert Haas wrote: > There's bound to be somebody who wants to grant some of > these permissions and not others, or who wants to grant the ability to > run those commands on some tables but not others. Is there anything stopping us from adding syntax like this? GRANT VACUUM

Re: enabling FOO=bar arguments to vcregress.pl

2021-11-02 Thread Andrew Dunstan
On 11/2/21 11:03, Andrew Dunstan wrote: > On 11/1/21 21:23, Michael Paquier wrote: >> On Mon, Nov 01, 2021 at 11:33:21AM -0400, Andrew Dunstan wrote: >>> As I mentioned recently at >>> ,  >>> I want to get USE_MODULE_DB workin

Re: XTS cipher mode for cluster file encryption

2021-11-02 Thread Bruce Momjian
On Mon, Nov 1, 2021 at 02:24:36PM -0400, Stephen Frost wrote: > I can understand the general idea that we should be sure to engineer > this in a way that multiple methods can be used, as surely one day folks > will say that AES128 isn't acceptable any more. In terms of what we'll > do from the st

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Peter Geoghegan
On Tue, Nov 2, 2021 at 1:47 PM Robert Haas wrote: > Yeah. I have only very rarely run into cases where people actually end > up needing multiple passes, but it's always something I need to rule > out as part of the troubleshooting process, and it's hard to do that > without the log_autovacuum_min_

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Vik Fearing
On 10/31/21 10:24 PM, Michael Banck wrote: > To put another option on the table: maybe a compromise could be to log > xlog checkpoints unconditionally, and the (checkpoint_timeout) time ones > only if log_checkpoints are set (maybe with some exponential backoff to > avoid log spam)? If we're going

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 11:27 AM, "Stephen Frost" wrote: > * Bossart, Nathan (bossa...@amazon.com) wrote: >> The approach in the patch looks alright to me, but another one could >> be to build a SelectStmt when parsing CHECKPOINT. I think that'd >> simplify the standard_ProcessUtility() changes. > > For my 2

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 3:05 PM Peter Geoghegan wrote: > I think that 10 minutes is fine. But if it has to be 30 minutes, then > that's also probably fine. +1. > I think that the case for enabling autovacuum logging is particularly > good. The really big problems with autovacuum often involve > a

Re: make tuplestore helper function

2021-11-02 Thread Justin Pryzby
Several places have a conditional value for the first argument (randomAccess), but your patch changes the behavior to a constant "true". I didn't review the patch beyond that. > @@ -740,18 +724,14 @@ pg_prepared_statement(PG_FUNCTION_ARGS) > - tupstore = > - tuplestore_begin_heap(

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Justin Pryzby
On Sun, Oct 31, 2021 at 10:59:19AM -0400, Tom Lane wrote: > Bharath Rupireddy writes: > > How about we enable it out of the box? > > No. > > The general policy at the moment is that a normally-functioning server > should emit *no* log traffic by default (other than a few messages > at startup an

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Michael Banck
Hi, On Tue, Nov 02, 2021 at 11:55:23AM -0400, Robert Haas wrote: > I think shipping with log_checkpoints=on and > log_autovacuum_min_duration=10m or so would be one of the best things > we could possibly do to allow ex-post-facto troubleshooting of > system-wide performance issues. I don't disagr

Re: prevent immature WAL streaming

2021-11-02 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Oct-13, Andres Freund wrote: >> Yea, let's go for your patch then. I've verified that at least locally it >> passes under valgrind. > Ah great, thanks. Pushed then. Seems like this hasn't fixed the problem: skink still fails on this test occasionally. https://b

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2021-11-02 Thread Melanie Plageman
v14 attached. On Tue, Oct 19, 2021 at 3:29 PM Andres Freund wrote: > > > > > Is pgstattuple the best place for this helper? It's not really pgstatfuncs > > > specific... > > > > > > It also looks vaguely familiar - I wonder if we have a helper roughly like > > > this somewhere else already... > >

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Peter Geoghegan
On Tue, Nov 2, 2021 at 11:50 AM Robert Haas wrote: > I almost proposed 1m rather than 10m, but then I thought the better of > it. I think it's unlikely that an autovacuum that takes 1 minute is > really the cause of some big problem you're having on your system. > Typical problem cases I see are h

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Nikolay Samokhvalov
On Tue, Nov 2, 2021 at 11:50 AM Robert Haas wrote: > I almost proposed 1m rather than 10m, but then I thought the better of > it. I think it's unlikely that an autovacuum that takes 1 minute is > really the cause of some big problem you're having on your system. > Typical problem cases I see are

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 2:39 PM Nikolay Samokhvalov wrote: > +1 for log_checkpoints = on > and +1 for log_autovacuum_min_duration = 1m or so. I almost proposed 1m rather than 10m, but then I thought the better of it. I think it's unlikely that an autovacuum that takes 1 minute is really the cause

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Alvaro Herrera
On 2021-Nov-02, Nikolay Samokhvalov wrote: > Back to checkpoint logging. With log_checkpoints = off, and high write > activity with low max_wal_size we're already "spamming" the logs with > lots of "checkpoints are occurring too frequently" – and this happens > very often, any DBA running a restor

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Alvaro Herrera
On 2021-Nov-02, Robert Haas wrote: > I think shipping with log_checkpoints=on and > log_autovacuum_min_duration=10m or so would be one of the best things > we could possibly do to allow ex-post-facto troubleshooting of > system-wide performance issues. The idea that users care more about > the inc

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Nikolay Samokhvalov
On Tue, Nov 2, 2021 at 8:55 AM Robert Haas wrote: > I think shipping with log_checkpoints=on and > log_autovacuum_min_duration=10m or so would be one of the best things > we could possibly do to allow ex-post-facto troubleshooting of > system-wide performance issues. > Fully agree, it would be r

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

2021-11-02 Thread Tom Lane
Pushed with some adjustment of the comments. I also simplified the datestyle setting to just "ISO", because that's sufficient: that DateStyle doesn't care about DateOrder. Since the settings are supposed to match what pg_dump uses, it's just confusing if they don't. Also, I didn't commit the tes

Re: Eval expression R/O once time (src/backend/executor/execExpr.c)

2021-11-02 Thread Andres Freund
On 2021-11-02 13:43:46 -0400, Tom Lane wrote: > Ranier Vilela writes: > > It seems that 1ec7679f1b67e84be688a311dce234eeaa1d5de8 caused the problem. > > Indeed. Fix pushed. Thanks to both of you!

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Stephen Frost
Greetings, * Jeff Davis (pg...@j-davis.com) wrote: > On Tue, 2021-11-02 at 11:06 -0400, Robert Haas wrote: > > Just as a sort of general comment on this endeavor, I suspect that > > any > > attempt to lump things together that seem closely related is doomed > > to > > backfire. > > Agreed, I thin

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Jan Wieck
On 11/2/21 12:09, Peter Geoghegan wrote: On Tue, Nov 2, 2021 at 8:55 AM Robert Haas wrote: I think shipping with log_checkpoints=on and log_autovacuum_min_duration=10m or so would be one of the best things we could possibly do to allow ex-post-facto troubleshooting of system-wide performance is

Re: make tuplestore helper function

2021-11-02 Thread Alvaro Herrera
On 2021-Nov-02, Melanie Plageman wrote: > Attached is a patch to create a helper function which creates a > tuplestore to be used by FUNCAPI-callable functions. Looks good, and given the amount of code being removed, it seems a no-brainer that some helper(s) is/are needed. I think the name is a

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 10:29 AM, "Jeff Davis" wrote: > Great idea! Patch attached. > > This feels like a good pattern that we might want to use elsewhere, if > the need arises. The approach in the patch looks alright to me, but another one could be to build a SelectStmt when parsing CHECKPOINT. I think tha

Re: Eval expression R/O once time (src/backend/executor/execExpr.c)

2021-11-02 Thread Tom Lane
Ranier Vilela writes: > It seems that 1ec7679f1b67e84be688a311dce234eeaa1d5de8 caused the problem. Indeed. Fix pushed. regards, tom lane

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Jeff Davis
On Tue, 2021-11-02 at 11:06 -0400, Robert Haas wrote: > Just as a sort of general comment on this endeavor, I suspect that > any > attempt to lump things together that seem closely related is doomed > to > backfire. Agreed, I think that is apparent from the different opinions in this thread. Robe

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread David Steele
On 11/2/21 12:35 PM, Julien Rouhaud wrote: Le mer. 3 nov. 2021 à 00:18, Andrew Dunstan > a écrit : On 11/2/21 12:09, Peter Geoghegan wrote: > On Tue, Nov 2, 2021 at 8:55 AM Robert Haas mailto:robertmh...@gmail.com>> wrote: >> I think shipping with log_c

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Jeff Davis
On Mon, 2021-11-01 at 12:50 -0400, Stephen Frost wrote: > All that said, I wonder if we can have our cake and eat it too. I > haven't looked into this at all yet and perhaps it's foolish on its > face, but, could we make CHECKPOINT; basically turn around and just > run > select pg_checkpoint(); wi

Re: archive modules

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 9:46 AM, "Robert Haas" wrote: > On Tue, Nov 2, 2021 at 12:39 PM Bossart, Nathan wrote:> >> On 11/2/21, 9:17 AM, "Robert Haas" wrote: >> You could still introduce GUCs in _PG_init(), but they couldn't be >> defined as PGC_POSTMASTER. > > It seems like PGC_POSTMASTER isn't very desirab

Re: Add additional information to src/test/ssl/README

2021-11-02 Thread Kevin Burke
The patch looks great. Thanks! Kevin On Sun, Oct 31, 2021 at 3:13 PM Tom Lane wrote: > Daniel Gustafsson writes: > > On 31 Oct 2021, at 19:28, Tom Lane wrote: > >> Here's a quick stab at rearranging src/test/perl/README so that the > >> initial section is all how-to-run-the-tests info, and ad

Re: archive modules

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 12:39 PM Bossart, Nathan wrote:> > On 11/2/21, 9:17 AM, "Robert Haas" wrote: > You could still introduce GUCs in _PG_init(), but they couldn't be > defined as PGC_POSTMASTER. It seems like PGC_POSTMASTER isn't very desirable anyway. Wouldn't you want PGC_SIGHUP? I mean I'm

Re: archive modules

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 9:17 AM, "Robert Haas" wrote: > On Tue, Nov 2, 2021 at 12:10 PM Bossart, Nathan wrote: >> Yes, that seems doable. My point is that I've intentionally chosen to >> preload the libraries at the moment so that it's possible to define >> PGC_POSTMASTER GUCs and to use RegisterBackgroundW

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Julien Rouhaud
Le mer. 3 nov. 2021 à 00:18, Andrew Dunstan a écrit : > > On 11/2/21 12:09, Peter Geoghegan wrote: > > On Tue, Nov 2, 2021 at 8:55 AM Robert Haas > wrote: > >> I think shipping with log_checkpoints=on and > >> log_autovacuum_min_duration=10m or so would be one of the best things > >> we could po

make tuplestore helper function

2021-11-02 Thread Melanie Plageman
Attached is a patch to create a helper function which creates a tuplestore to be used by FUNCAPI-callable functions. It was suggested in [1] that I start a separate thread for this patch. A few notes: There are a few places with very similar code to the new helper (MakeTuplestore()) but which, f

Re: refactoring basebackup.c

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 10:32 AM Robert Haas wrote: > Looks pretty good. I think you should work on stuff like documentation > and tests, and I need to do some work on that stuff, too. Also, I > think you should try to figure out how to support different > compression levels. On second thought, ma

Re: Teach pg_receivewal to use lz4 compression

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 8:17 AM Magnus Hagander wrote: > Um, why? > > That we are using zlib to provide the compression is an implementation > detail. Whereas AFAIK "gzip" refers to both the program and the format. And > we specifically use the gzxxx() functions in zlib, in order to produce gzip

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Andrew Dunstan
On 11/2/21 12:09, Peter Geoghegan wrote: > On Tue, Nov 2, 2021 at 8:55 AM Robert Haas wrote: >> I think shipping with log_checkpoints=on and >> log_autovacuum_min_duration=10m or so would be one of the best things >> we could possibly do to allow ex-post-facto troubleshooting of >> system-wide p

Re: archive modules

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 12:10 PM Bossart, Nathan wrote: > Yes, that seems doable. My point is that I've intentionally chosen to > preload the libraries at the moment so that it's possible to define > PGC_POSTMASTER GUCs and to use RegisterBackgroundWorker(). If we > think that switching archive m

Re: archive modules

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 8:57 AM, "Robert Haas" wrote: > On Tue, Nov 2, 2021 at 11:26 AM Bossart, Nathan wrote: >> Well, the current patch does require a reload since the modules are >> preloaded, but maybe there is some way to avoid that. > > I think we could set things up so that if the value changes, you c

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Peter Geoghegan
On Tue, Nov 2, 2021 at 8:55 AM Robert Haas wrote: > I think shipping with log_checkpoints=on and > log_autovacuum_min_duration=10m or so would be one of the best things > we could possibly do to allow ex-post-facto troubleshooting of > system-wide performance issues. The idea that users care more

Re: archive modules

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 11:26 AM Bossart, Nathan wrote: > Well, the current patch does require a reload since the modules are > preloaded, but maybe there is some way to avoid that. I think we could set things up so that if the value changes, you call a shutdown hook for the old library, load the

Re: [PATCH] fix references to like_regex

2021-11-02 Thread Gilles Darold
Le 02/11/2021 à 16:50, Tom Lane a écrit : Gilles Darold writes: Since we have the regexp_like operator I have found that there is two references in the documentation about PostgreSQL lacking of LIKE_REGEX implementation. Here is a patch to fix the documentation. I simply remove the reference to

Re: should we enable log_checkpoints out of the box?

2021-11-02 Thread Robert Haas
On Sun, Oct 31, 2021 at 10:59 AM Tom Lane wrote: > The general policy at the moment is that a normally-functioning server > should emit *no* log traffic by default (other than a few messages > at startup and shutdown). log_checkpoints is a particularly poor > candidate for an exception to that po

Re: [PATCH] fix references to like_regex

2021-11-02 Thread Tom Lane
Gilles Darold writes: > Since we have the regexp_like operator I have found that there is two > references in the documentation about PostgreSQL lacking of LIKE_REGEX > implementation. Here is a patch to fix the documentation. I simply > remove the reference to non exist of LIKE_REGEX in Postgr

Re: inefficient loop in StandbyReleaseLockList()

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 8:36 AM, "Tom Lane" wrote: > I've pushed the SyncPostCheckpoint change, and I think I'm going > to stop here. It's not clear that the remaining list_delete_first > callers have any real problem; and changing them would be complex. > We can revisit the question if we find out there is

Re: inefficient loop in StandbyReleaseLockList()

2021-11-02 Thread Tom Lane
I've pushed the SyncPostCheckpoint change, and I think I'm going to stop here. It's not clear that the remaining list_delete_first callers have any real problem; and changing them would be complex. We can revisit the question if we find out there is an issue. Or, if somebody else wants to pursue t

Re: archive modules

2021-11-02 Thread Bossart, Nathan
On 11/2/21, 8:11 AM, "Robert Haas" wrote: > Why in the world would you want a plain hook rather than something > closer to the way logical replication works? > > Plain hooks are annoying to use. If you load things at the wrong time, > it silently doesn't work. It's also impossible to unload anythi

Re: Non-superuser subscription owners

2021-11-02 Thread Robert Haas
On Mon, Nov 1, 2021 at 6:44 PM Mark Dilger wrote: > > ALTER SUBSCRIPTION..[ENABLE | DISABLE] do not synchronously start or stop > > subscription workers. The ALTER command updates the catalog's subenabled > > field, but workers only lazily respond to that. Disabling and enabling the > > subsc

Re: archive modules

2021-11-02 Thread Bossart, Nathan
I've just realized I forgot to CC the active participants on the last thread to this one, so I've attempted to do that now. I didn't intentionally leave anyone out, but I'm sorry if I missed someone. On 11/1/21, 10:24 PM, "Michael Paquier" wrote: > It seems to me that this patch is not moving in

Re: archive modules

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 1:24 AM Michael Paquier wrote: > It seems to me that this patch is not moving into the right direction > implementation-wise (I have read the arguments about > backward-compatibility that led to the introduction of archive_library > and its shell mode), for what looks like a

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-02 Thread Robert Haas
On Sat, Oct 23, 2021 at 5:45 PM Jeff Davis wrote: > Add new predefined role pg_maintenance, which can issue VACUUM, > ANALYZE, CHECKPOINT. Just as a sort of general comment on this endeavor, I suspect that any attempt to lump things together that seem closely related is doomed to backfire. There'

Re: enabling FOO=bar arguments to vcregress.pl

2021-11-02 Thread Andrew Dunstan
On 11/1/21 21:23, Michael Paquier wrote: > On Mon, Nov 01, 2021 at 11:33:21AM -0400, Andrew Dunstan wrote: >> As I mentioned recently at >> ,  >> I want to get USE_MODULE_DB working for vcregress.pl. I started out >> writing co

Re: archive modules

2021-11-02 Thread Bossart, Nathan
On 11/1/21, 9:44 PM, "Fujii Masao" wrote: > What is the main motivation of this patch? I was thinking that > it's for parallelizing WAL archiving. But as far as I read > the patch very briefly, WAL file name is still passed to > the archive callback function one by one. The main motivation is pro

[PATCH] fix references to like_regex

2021-11-02 Thread Gilles Darold
Hi, Since we have the regexp_like operator I have found that there is two references in the documentation about PostgreSQL lacking of LIKE_REGEX implementation. Here is a patch to fix the documentation. I simply remove the reference to non exist of LIKE_REGEX in PostgreSQL in chapter "9.7.3.

Re: Added schema level support for publication.

2021-11-02 Thread Tomas Vondra
On 11/2/21 11:37 AM, Amit Kapila wrote: > On Mon, Nov 1, 2021 at 5:52 PM Tomas Vondra > wrote: >> >> On 11/1/21 11:18, Amit Kapila wrote: >>> On Mon, Nov 1, 2021 at 2:48 AM Tomas Vondra >>> wrote: I wonder if it'd be better to just separate the schema and object type specification, i

Re: Improve logging when using Huge Pages

2021-11-02 Thread Fujii Masao
On 2021/11/02 18:31, Shinoda, Noriyoshi (PN Japan FSIP) wrote: Fujii-san, Sawada-san, Thank you for your comment. Also, with the patch, the log message is emitted also during initdb and starting up in single user mode: Certainly the log output when executing the initdb command was a nois

Re: refactoring basebackup.c

2021-11-02 Thread Robert Haas
On Tue, Nov 2, 2021 at 7:53 AM Jeevan Ladhe wrote: > I have implemented the cleanup callback bbsink_lz4_cleanup() in the attached > patch. > > Please have a look and let me know of any comments. Looks pretty good. I think you should work on stuff like documentation and tests, and I need to do so

Re: inefficient loop in StandbyReleaseLockList()

2021-11-02 Thread Tom Lane
Kyotaro Horiguchi writes: > At Mon, 01 Nov 2021 18:01:18 -0400, Tom Lane wrote in >> So what I did in the attached is add a "canceled" flag to >> PendingUnlinkEntry, which lets us deal with canceled or finished >> entries without having to delete them from the list right away. >> Then we only ne

Re: pgbench bug candidate: negative "initial connection time"

2021-11-02 Thread Fujii Masao
On 2021/11/01 23:01, Fujii Masao wrote: The remainings are the changes of handling of initial connection or logfile open failures. I agree to push them at least for the master. But I'm not sure if they should be back-patched. Without these changes, even when those failures happen, pgbench proc

Re: Feature request for adoptive indexes

2021-11-02 Thread Pavel Borisov
вт, 2 нояб. 2021 г. в 16:04, Hayk Manukyan : > Tomas Vondra > > Are you suggesting those are not the actual best/worst cases and we > > should use some other indexes? If yes, which ones? > > I would say yes. > In my case I am not querying only sequence column. > I have the following cases which I

Re: Feature request for adoptive indexes

2021-11-02 Thread Tomas Vondra
On 11/2/21 13:04, Hayk Manukyan wrote: Tomas Vondra > Are you suggesting those are not the actual best/worst cases and we > should use some other indexes? If yes, which ones? I would say yes. In my case I am not querying only sequence column. I have the following cases which I want to optim

Re: Multi-Column List Partitioning

2021-11-02 Thread Nitin Jadhav
> I noticed that there's no commitfest entry for this. Will you please > add this to the next one? I have added it to Nov commitfest. Thanks & Regards, Nitin Jadhav On Fri, Oct 29, 2021 at 1:40 PM Amit Langote wrote: > > Hi Nitin, > > On Fri, Oct 22, 2021 at 6:48 PM Nitin Jadhav > wrote: > >

Re: removing global variable ThisTimeLineID

2021-11-02 Thread Robert Haas
On Mon, Nov 1, 2021 at 11:33 PM Michael Paquier wrote: > + /* > +* If we're writing and flushing WAL, the time line can't be changing, > +* so no lock is required. > +*/ > + if (insertTLI) > + *insertTLI = XLogCtl->ThisTimeLineID; > In 0002, there is no downside in putting th

Re: Teach pg_receivewal to use lz4 compression

2021-11-02 Thread Magnus Hagander
On Tue, Nov 2, 2021 at 9:51 AM Michael Paquier wrote: > On Tue, Nov 02, 2021 at 07:27:50AM +0900, Michael Paquier wrote: > > On Mon, Nov 01, 2021 at 08:39:59AM +, gkokola...@pm.me wrote: > > > Agreed. > > > > > > I have already started on v8 of the patch with that technique. I should > > > be

RE: Failed transaction statistics to measure the logical replication progress

2021-11-02 Thread osumi.takami...@fujitsu.com
On Monday, November 1, 2021 10:18 PM I wrote: > On Thursday, October 28, 2021 11:19 PM I wrote: > > I've created a new patch that extends pg_stat_subscription_workers to > > include other transaction statistics. > > > > Note that this patch depends on v18 patch-set in [1]... > Rebased based on the

Re: Feature request for adoptive indexes

2021-11-02 Thread Hayk Manukyan
Tomas Vondra > Are you suggesting those are not the actual best/worst cases and we > should use some other indexes? If yes, which ones? I would say yes. In my case I am not querying only sequence column. I have the following cases which I want to optimize. 1. Select * from Some_table where job =

Re: refactoring basebackup.c

2021-11-02 Thread Jeevan Ladhe
I have implemented the cleanup callback bbsink_lz4_cleanup() in the attached patch. Please have a look and let me know of any comments. Regards, Jeevan Ladhe On Fri, Oct 29, 2021 at 6:54 PM Robert Haas wrote: > On Fri, Oct 29, 2021 at 8:59 AM Jeevan Ladhe > wrote:> > > bbsink_gzip_ops have

Re: Teach pg_receivewal to use lz4 compression

2021-11-02 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Tuesday, November 2nd, 2021 at 9:51 AM, Michael Paquier wrote: > On Tue, Nov 02, 2021 at 07:27:50AM +0900, Michael Paquier wrote: > > On Mon, Nov 01, 2021 at 08:39:59AM +, gkokola...@pm.me wrote: > > > Agreed. > > > > > > I have already started on v8 of

Re: RFC: Logging plan of the running query

2021-11-02 Thread Ekaterina Sokolova
Hi! I'm here to answer your questions about contrib/pg_query_state. I only took a quick look at pg_query_state, I have some questions. pg_query_state seems using shm_mq to expose the plan information, but there was a discussion that this kind of architecture would be tricky to do properly [1]

  1   2   >