Re: Skipping logical replication transactions on subscriber side

2021-11-14 Thread Greg Nancarrow
On Mon, Nov 15, 2021 at 1:49 PM Masahiko Sawada wrote: > > I've attached an updated patch that incorporates all comments I got so > far. Please review it. > Thanks for the updated patch. A few minor comments: doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml (1) tab in doc updates

Re: Printing backtrace of postgres processes

2021-11-14 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 12:08 PM Dilip Kumar wrote: > > > 2. > > > postgres[64154]=# select pg_print_backtrace(64136); > > > WARNING: 01000: PID 64136 is not a PostgreSQL server process > > > LOCATION: pg_print_backtrace, signalfuncs.c:335 > > > pg_print_backtrace > > > >

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-11-14 Thread Bharath Rupireddy
On Sun, Mar 7, 2021 at 3:46 PM Bharath Rupireddy wrote: > > On Fri, Feb 5, 2021 at 5:15 PM Bharath Rupireddy > wrote: > > > > pg_terminate_backend and pg_cancel_backend with postmaster PID produce > > "PID is not a PostgresSQL server process" warning [1], which > > basically implies that

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-11-14 Thread Masahiko Sawada
On Sat, Nov 13, 2021 at 2:10 PM Amit Kapila wrote: > > On Fri, Nov 12, 2021 at 6:44 PM Alvaro Herrera > wrote: > > > > On 2021-Nov-11, Masahiko Sawada wrote: > > > > > On Thu, Nov 11, 2021 at 12:53 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Nov 11, 2021 at 9:11 AM Andres Freund > >

Re: row filtering for logical replication

2021-11-14 Thread Amit Kapila
On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > Attaching version 39- > > V39 fixes the following review comments: > > On Fri, Nov 5, 2021 at 7:49 PM Amit Kapila wrote: > > > > CheckObjSchemaNotAlreadyInPublication(rels, schemaidlist, > >PUBLICATIONOBJ_TABLE); > > > >I think for the

Re: Printing backtrace of postgres processes

2021-11-14 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 12:13 PM vignesh C wrote: > > On Mon, Nov 15, 2021 at 11:00 AM Bharath Rupireddy > wrote: > > > > On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > > > 2) I think "which is enough because the target process for logging of > > > > backtrace is a backend" isn't valid

Re: Printing backtrace of postgres processes

2021-11-14 Thread vignesh C
On Mon, Nov 15, 2021 at 11:00 AM Bharath Rupireddy wrote: > > On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > > 2) I think "which is enough because the target process for logging of > > > backtrace is a backend" isn't valid anymore with 0002, righit? Please > > > remove it. > > > + * to

Re: Printing backtrace of postgres processes

2021-11-14 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 11:53 AM Bharath Rupireddy wrote: > > On Mon, Nov 15, 2021 at 11:37 AM Dilip Kumar wrote: > > > > On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > > > > > > > Thanks for the comments, the attached v12 patch has the changes for the > > > same. > > > > I have

Re: Printing backtrace of postgres processes

2021-11-14 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 11:37 AM Dilip Kumar wrote: > > On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > > > > Thanks for the comments, the attached v12 patch has the changes for the > > same. > > I have reviewed this patch and have some comments on v12-0001, > > 1. > +This feature

Re: Emit a warning if the extension's GUC is set incorrectly

2021-11-14 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 6:33 AM Shinya Kato wrote: > > On 2021-11-15 04:50, Daniel Gustafsson wrote: > > Seems reasonable on a quick skim, commit da2c1b8a2 did a similar > > roundup back > > in 2009 but at the time most of these didn't exist (pg_trgm did but > > didn't have > > custom option back

RE: row filtering for logical replication

2021-11-14 Thread tanghy.f...@fujitsu.com
On Friday, November 12, 2021 6:20 PM Ajin Cherian wrote: > > Attaching version 39- > Thanks for the new patch. I met a problem when using "ALTER PUBLICATION ... SET TABLE ... WHERE ...", the publisher was crashed after executing this statement. Here is some information about this problem.

Re: Printing backtrace of postgres processes

2021-11-14 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > Thanks for the comments, the attached v12 patch has the changes for the same. I have reviewed this patch and have some comments on v12-0001, 1. +This feature is not supported for the postmaster, logger, checkpointer, +

RE: row filtering for logical replication

2021-11-14 Thread tanghy.f...@fujitsu.com
On Wednesday, November 10, 2021 7:46 AM Peter Smith wrote: > > On Tue, Nov 9, 2021 at 2:03 PM tanghy.f...@fujitsu.com > wrote: > > > > On Friday, November 5, 2021 1:14 PM, Peter Smith > wrote: > > > > > > PSA new set of v37* patches. > > > > > > > Thanks for your patch. I have a problem when

Re: Reuse of State value in Aggregates

2021-11-14 Thread gg pw
That's sufficient information for me thanks! I don't really have a concrete example at the moment. The idea just popped up in my head when I read that CTEs are reused inside parent queries.

Re: Printing backtrace of postgres processes

2021-11-14 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > 2) I think "which is enough because the target process for logging of > > backtrace is a backend" isn't valid anymore with 0002, righit? Please > > remove it. > > + * to call this function if we see PrintBacktracePending set. It is called > >

Re: Printing backtrace of postgres processes

2021-11-14 Thread vignesh C
On Mon, Nov 15, 2021 at 7:37 AM Bharath Rupireddy wrote: > > On Sun, Nov 14, 2021 at 8:49 PM vignesh C wrote: > > > 7) Do we need TAP tests for this function? I think it is sufficient to > > > test the function in misc_functions.sql, please remove > > > 002_print_backtrace_validation.pl. Note

Re: Add psql command to list constraints

2021-11-14 Thread Justin Pryzby
Hi, On Mon, Nov 15, 2021 at 10:38:55AM +0900, Tatsuro Yamada wrote: > postgres=# \dco > List of constsraints > Schema | Name | Definition > | Table >

Re: Clean up build warnings of plperl with clang-12+

2021-11-14 Thread Michael Paquier
On Thu, Nov 11, 2021 at 03:51:35PM -0500, Tom Lane wrote: > That'd be considerably messier wouldn't it? Yes, that would be a bit messier. For example, we could do that with something saved in Makefile.global.in by ./configure that plperl feeds on to add this extra CFLAGS in its own local

Re: Skipping logical replication transactions on subscriber side

2021-11-14 Thread Masahiko Sawada
On Wed, Nov 10, 2021 at 12:49 PM vignesh C wrote: > > > Thanks for the updated patch, Few comments: Thank you for the comments! > 1) should we change "Tables and functions hashes are initialized to > empty" to "Tables, functions and subworker hashes are initialized to > empty" > +

Re: Time to drop plpython2?

2021-11-14 Thread Tom Lane
... btw, there's a fairly critical gating factor for any plan to drop python2 support: the buildfarm. I just counted, and there are exactly as many members running python 2.x as 3.x (49 apiece), not counting Windows machines that aren't running configure. We can't commit something that's going

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-11-14 Thread Bharath Rupireddy
On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy wrote: > PSA v2 patch and review it. I've modified the docs part a bit, please consider v3 for review. Regards, Bharath Rupireddy. v3-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patch Description: Binary data

Re: Printing backtrace of postgres processes

2021-11-14 Thread Bharath Rupireddy
On Sun, Nov 14, 2021 at 8:49 PM vignesh C wrote: > > 7) Do we need TAP tests for this function? I think it is sufficient to > > test the function in misc_functions.sql, please remove > > 002_print_backtrace_validation.pl. Note that we don't do similar TAP > > testing for

RE: [BUG]Invalidate relcache when setting REPLICA IDENTITY

2021-11-14 Thread houzj.f...@fujitsu.com
On Sat, Nov 13, 2021 6:50 PM Amit Kapila wrote: > > The patch looks mostly good to me. I have slightly tweaked the comments in > the code (as per my previous suggestion) and test. Also, I have slightly > modified the commit message. If the attached looks good to you then kindly > prepare patches

Add psql command to list constraints

2021-11-14 Thread Tatsuro Yamada
Hi, I have been wondering why there is no meta-command for listing constraints in psql. So, I created a POC patch by using my experience developing \dX command in PG14. This feature is helpful for DBAs when they want to check or modify the definition of constraints. The current status of the

Re: Skipping logical replication transactions on subscriber side

2021-11-14 Thread Masahiko Sawada
On Tue, Nov 9, 2021 at 3:07 PM Dilip Kumar wrote: > > On Sun, Nov 7, 2021 at 7:50 PM Masahiko Sawada wrote: > > I've attached an updated patch. In this version patch, subscription > > worker statistics are collected per-database and handled in a similar > > way to tables and functions. I think

Re: Skipping logical replication transactions on subscriber side

2021-11-14 Thread Masahiko Sawada
On Mon, Nov 8, 2021 at 4:10 PM Greg Nancarrow wrote: > > On Mon, Nov 8, 2021 at 1:20 AM Masahiko Sawada wrote: > > > > I've attached an updated patch. In this version patch, subscription > > worker statistics are collected per-database and handled in a similar > > way to tables and functions. I

Re: Emit a warning if the extension's GUC is set incorrectly

2021-11-14 Thread Shinya Kato
On 2021-11-15 04:50, Daniel Gustafsson wrote: Seems reasonable on a quick skim, commit da2c1b8a2 did a similar roundup back in 2009 but at the time most of these didn't exist (pg_trgm did but didn't have custom option back then). There is one additional callsite defining custom variables in

Re: row filtering for logical replication

2021-11-14 Thread Peter Smith
On Fri, Nov 12, 2021 at 9:19 PM Ajin Cherian wrote: > > Attaching version 39- Here are some review comments for v39-0006: 1) @@ -261,9 +261,9 @@ rowfilter_expr_replident_walker(Node *node, rf_context *context) * Rule 1. Walk the parse-tree and reject anything other than very simple *

Re: JIT doing duplicative optimization?

2021-11-14 Thread Tom Lane
I wrote: > You could probably generate some queries with lots and lots of expressions > to characterize this better. If it is O(N^2), it should not be hard to > drive the cost up to the point where the guilty bit of code would stand > out in a perf trace. I experimented with that, using a few

Re: Inconsistent error message for varchar(n)

2021-11-14 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > Tracking it a bit further, the actual typmod limit is set by this: > > /* > * MaxAttrSize is a somewhat arbitrary upper limit on the declared size of > * data fields of char(n) and similar types. It need not have anything > * directly to do with the *actual* upper limit of

Re: Atomic rename feature for Windows.

2021-11-14 Thread Victor Spirin
Hi Added the pgunlink_windows_posix_semantics function and modified the pgunlink function I used FILE_DISPOSITION_POSIX_SEMANTICS flag for unlink files on Windows 10 (1607) and above. Victor Spirin Postgres Professional:http://www.postgrespro.com The Russian Postgres Company 05.07.2021

Re: Emit a warning if the extension's GUC is set incorrectly

2021-11-14 Thread Daniel Gustafsson
> On 14 Nov 2021, at 11:03, Shinya Kato wrote: > If wrong GUCs of auth_delay, pg_trgm, postgres_fdw and sepgsql are described > in postgresql.conf, a warning is not emitted unlike pg_stat_statements, > auto_explain and pg_prewarm. > So, I improved it by adding EmitWarningsOnPlaceholders. > An

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-14 Thread Tom Lane
Stephen Frost writes: > Attackers aren't likely to have the kind of isolated control over the > data in the WAL stream (which is a combination of data from lots of > ongoing activity in the system and isn't likely to be exactly what the > attacker supplied at some higher level anyway) and the

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-14 Thread Stephen Frost
Greetings, * Andrey Borodin (x4...@yandex-team.ru) wrote: > > On 11/10/21 16:54, Andrey Borodin wrote: > >> Compression is crucial for highly available setups. Replication traffic is > >> often billed. Or route has bandwidth limits. > >> An entropy added by WAL headers makes CRIME attack against

Re: Inconsistent error message for varchar(n)

2021-11-14 Thread Tom Lane
I wrote: > For comparison, it doesn't bring up the point that string values are > constrained to 1GB; that's dealt with elsewhere. Since the limit on > typmod is substantially more than that, I'm not sure there's much point > in mentioning it specifically. Oh, wait, I was not counting the zeroes

Re: Inconsistent error message for varchar(n)

2021-11-14 Thread Tom Lane
[ Please trim quotes appropriately when replying. Nobody wants to read the whole history of the thread to get to your comment. ] Japin Li writes: > Oh! I didn't consider this situation. Since the max size of varchar cannot > exceed 10485760, however, I cannot find this in documentation [1].

Re: JIT doing duplicative optimization?

2021-11-14 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Nov-11, Alvaro Herrera wrote: >> But what really surprised me is that the the average time to optimize >> per function is now 2.06ms ... less than half of the previous >> measurement. It emits 10% less functions than before, but the time to >> both optimize and

Re: JIT doing duplicative optimization?

2021-11-14 Thread Zhihong Yu
On Sun, Nov 14, 2021 at 9:07 AM Alvaro Herrera wrote: > On 2021-Nov-11, Alvaro Herrera wrote: > > > But what really surprised me is that the the average time to optimize > > per function is now 2.06ms ... less than half of the previous > > measurement. It emits 10% less functions than before,

Re: JIT doing duplicative optimization?

2021-11-14 Thread Alvaro Herrera
On 2021-Nov-11, Alvaro Herrera wrote: > But what really surprised me is that the the average time to optimize > per function is now 2.06ms ... less than half of the previous > measurement. It emits 10% less functions than before, but the time to > both optimize and emit is reduced by 50%. How

Re: Printing backtrace of postgres processes

2021-11-14 Thread vignesh C
On Fri, Nov 12, 2021 at 6:11 PM Bharath Rupireddy wrote: > > On Fri, Nov 12, 2021 at 5:15 PM Bharath Rupireddy > wrote: > > > > On Thu, Nov 11, 2021 at 12:14 PM vignesh C wrote: > > > Thanks for the comments, the attached v10 patch has the fixes for the > > > same. > > > > Thanks for the

Re: Printing backtrace of postgres processes

2021-11-14 Thread vignesh C
On Fri, Nov 12, 2021 at 5:15 PM Bharath Rupireddy wrote: > > On Thu, Nov 11, 2021 at 12:14 PM vignesh C wrote: > > Thanks for the comments, the attached v10 patch has the fixes for the same. > > Thanks for the patches. Here are some comments: > > 1) In the docs, let's have the similar

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-14 Thread Bharath Rupireddy
On Sat, Nov 13, 2021 at 7:57 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > On Sat, Nov 13, 2021 at 7:16 PM Euler Taveira wrote: > > Thanks. It is a good idea to use errdetail_relkind_not_supported. I > > slightly modified the API to "int

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-11-14 Thread Bharath Rupireddy
On Sat, Nov 13, 2021 at 9:04 PM Alvaro Herrera wrote: > > A bunch of these now execute snprintf()s unnecessarily. I think these > should be made conditional on message_level_is_interesting(DEBUG1) to > avoid that overhead. Thanks. Attaching the v2 to avoid that by directly using the message in

Re: support for MERGE

2021-11-14 Thread Amit Langote
On Sun, Nov 14, 2021 at 12:23 AM Álvaro Herrera wrote: > On 2021-Nov-13, Daniel Westermann wrote: > > /usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2 > > -I../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -flto=thin > > -emit-llvm -c -o execMerge.bc

Emit a warning if the extension's GUC is set incorrectly

2021-11-14 Thread Shinya Kato
Hi hackers, If wrong GUCs of auth_delay, pg_trgm, postgres_fdw and sepgsql are described in postgresql.conf, a warning is not emitted unlike pg_stat_statements, auto_explain and pg_prewarm. So, I improved it by adding EmitWarningsOnPlaceholders. An example output is shown below. ---

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-14 Thread Andrey Borodin
> On 11/10/21 16:54, Andrey Borodin wrote: > >> Compression is crucial for highly available setups. Replication traffic is >> often billed. Or route has bandwidth limits. >> An entropy added by WAL headers makes CRIME attack against replication >> encryption impractical. > > I very much

Re: make update-po problem with USE_PGXS

2021-11-14 Thread Peter Eisentraut
On 01.11.21 04:56, wangsh.f...@fujitsu.com wrote: I'm developing an extension, after modifying some source file(add some message like gettext('x')) and execute $ make USE_PGXS=1 update-po make: Nothing to be done for 'update-po'. I feel strange because I think *.po.new files should be