Re: PG DOCS - logical replication filtering

2022-04-17 Thread Amit Kapila
On Thu, Apr 14, 2022 at 8:55 AM Amit Kapila wrote: > > On Thu, Apr 14, 2022 at 1:29 AM Euler Taveira wrote: > > > > On Wed, Apr 13, 2022, at 12:24 AM, Peter Smith wrote: > > > > PSA patch v10 which addresses the remaining review comments from Euler [1] > > > > Looks good to me. > > > > Thanks,

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-17 Thread Tatsuo Ishii
>> One thing that this doesn't fix is that the existing text appears >> to suggest that the "failures" column is something different from >> the sum of the serialization_failures and deadlock_failures >> columns, which it's obvious from the code is not so. If this isn't >> a code bug then I think

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Amit Kapila
On Sat, Apr 16, 2022 at 10:42 PM Tom Lane wrote: > > My pet dinosaur prairiedog just failed in the contrib/test_decoding > tests [1]: > > diff -U3 > /Users/buildfarm/bf-data/HEAD/pgsql.build/contrib/test_decoding/expected/stream.out > >

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-17 Thread Michael Paquier
On Tue, Apr 12, 2022 at 12:49:48PM +0900, Michael Paquier wrote: > This comes from df86e52, where we want to recovery a history file that > would be created as RECOVERYHISTORY and make sure that the file gets > removed at the end of recovery. So $standby2 should choose a new > timeline different

Re: Logical replication timeout problem

2022-04-17 Thread Amit Kapila
On Mon, Apr 18, 2022 at 9:29 AM Amit Kapila wrote: > > On Thu, Apr 14, 2022 at 5:52 PM Euler Taveira wrote: > > > > On Wed, Apr 13, 2022, at 7:45 AM, Amit Kapila wrote: > > > > Sawada-San, Euler, do you have any opinion on this approach? I > > personally still prefer the approach implemented in

Re: Logical replication timeout problem

2022-04-17 Thread Masahiko Sawada
On Mon, Apr 18, 2022 at 1:01 PM Amit Kapila wrote: > > On Thu, Apr 14, 2022 at 5:50 PM Masahiko Sawada wrote: > > > > On Wed, Apr 13, 2022 at 7:45 PM Amit Kapila wrote: > > > > > > On Mon, Apr 11, 2022 at 12:09 PM wangw.f...@fujitsu.com > > > wrote: > > > > > > > > So I skip tracking lag

Re: Skipping logical replication transactions on subscriber side

2022-04-17 Thread Masahiko Sawada
On Mon, Apr 18, 2022 at 12:22 PM Noah Misch wrote: > > On Mon, Apr 18, 2022 at 10:45:50AM +0900, Masahiko Sawada wrote: > > On Fri, Apr 15, 2022 at 4:26 PM Noah Misch wrote: > > > On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > > > > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila

pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

2022-04-17 Thread Bharath Rupireddy
On Wed, Dec 29, 2021 at 8:06 PM Bharath Rupireddy wrote: > > On Wed, Dec 29, 2021 at 7:27 PM Stephen Frost wrote: > > > On Thu, Dec 23, 2021, at 9:58 AM, Bharath Rupireddy wrote: > > > > pg_archivecleanup currently takes a WAL file name as input to delete > > > > the WAL files prior to it [1].

Re: Timeout control within tests

2022-04-17 Thread Noah Misch
(I pushed the main patch as f2698ea, on 2022-03-04.) On Fri, Feb 18, 2022 at 06:41:36PM -0800, Noah Misch wrote: > On Fri, Feb 18, 2022 at 10:26:52AM -0500, Tom Lane wrote: > > Noah Misch writes: > > > On Thu, Feb 17, 2022 at 09:48:25PM -0800, Andres Freund wrote: > > >> Meson's test runner has

GSoC: New & Improved Website for PgJDBC

2022-04-17 Thread Kunal Kashyap
Hello all, PFA my proposal for GSoC 2022 Project: Improve PgArchives Potential Mentor: Dave Cramer Thank you Kunal Kashyap GSOC '22 - Project Proposal - New & Improved Website for PgJDBC - Kunal Kashyap.pdf Description: Adobe PDF document

Re: Probable memory leak with ECPG and AIX

2022-04-17 Thread Noah Misch
On Sat, Jan 01, 2022 at 04:07:50PM -0800, Noah Misch wrote: > On Sat, Jan 01, 2022 at 11:35:02AM -0500, Tom Lane wrote: > > Noah Misch writes: > > > I get the same results. The leak arises because AIX freelocale() doesn't > > > free > > > all memory allocated in newlocale(). The following

Re: Logical replication timeout problem

2022-04-17 Thread Amit Kapila
On Thu, Apr 14, 2022 at 5:50 PM Masahiko Sawada wrote: > > On Wed, Apr 13, 2022 at 7:45 PM Amit Kapila wrote: > > > > On Mon, Apr 11, 2022 at 12:09 PM wangw.f...@fujitsu.com > > wrote: > > > > > > So I skip tracking lag during a transaction just like the current HEAD. > > > Attach the new

Re: Logical replication timeout problem

2022-04-17 Thread Amit Kapila
On Thu, Apr 14, 2022 at 5:52 PM Euler Taveira wrote: > > On Wed, Apr 13, 2022, at 7:45 AM, Amit Kapila wrote: > > Sawada-San, Euler, do you have any opinion on this approach? I > personally still prefer the approach implemented in v10 [1] especially > due to the latest finding by Wang-San that we

Re: Printing backtrace of postgres processes

2022-04-17 Thread vignesh C
On Fri, Apr 15, 2022 at 11:49 AM Kyotaro Horiguchi wrote: > > At Thu, 14 Apr 2022 10:33:50 +0530, vignesh C wrote in > > On Wed, Apr 6, 2022 at 12:29 PM vignesh C wrote: > > > > > > On Tue, Apr 5, 2022 at 9:18 PM Robert Haas wrote: > > > > This looks like a grotty hack. > > > > > > I have

Re: Skipping logical replication transactions on subscriber side

2022-04-17 Thread Noah Misch
On Mon, Apr 18, 2022 at 10:45:50AM +0900, Masahiko Sawada wrote: > On Fri, Apr 15, 2022 at 4:26 PM Noah Misch wrote: > > On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > > > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila > > > wrote: > > > > On Thu, Apr 7, 2022 at 8:25 AM Amit

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Amit Kapila
On Sun, Apr 17, 2022 at 8:02 PM Andres Freund wrote: > > Adding Amit, I think this is stuff he worked on... > I'll look into this and share my findings. -- With Regards, Amit Kapila.

Re: Fixes for compression options of pg_receivewal and refactoring of backup_compression.{c,h}

2022-04-17 Thread Michael Paquier
On Thu, Apr 14, 2022 at 06:18:29AM +0900, Michael Paquier wrote: > The issue back then was how to design the option set to handle all > that (right? My memories may be short on that), and pg_basebackup > takes care of that with its option design. I have looked at that again this morning, and the

GSoC: Improve PgArchives

2022-04-17 Thread Kunal Kashyap
Hello all, PFA my proposal for GSoC 2022 Project: Improve PgArchives Potential Mentors: Ilaria Battiston, Stephen Frost Thank you Kunal Kashyap GSOC '22 - Project Proposal - Improve pgArchives - Kunal Kashyap.pdf Description: Adobe PDF document

GSoC: GUI representation of monitoring System Activity with the system_stats Extension in pgAdmin 4

2022-04-17 Thread Kunal Kashyap
Hello, PFA my proposal for GSoC 2022 Project: GUI representation of monitoring System Activity with the system_stats Extension in pgAdmin 4 Potential Mentor: Khushboo Vashi Thank you Kunal Kashyap GSOC '22 - Project Proposal - GUI representation of monitoring System Activity with the

Re: Skipping logical replication transactions on subscriber side

2022-04-17 Thread Masahiko Sawada
On Fri, Apr 15, 2022 at 4:26 PM Noah Misch wrote: > > On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila wrote: > > > On Thu, Apr 7, 2022 at 8:25 AM Amit Kapila > > > wrote: > > > > I'll take care of this today. I think we can mark

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-17 Thread Michael Paquier
On Sun, Apr 17, 2022 at 10:56:08AM -0400, Andrew Dunstan wrote: > I don't really think it's Cluster.pm's business to deal with that. It > takes an install path as given either explicitly or implicitly. > > It shouldn't be too hard to get Makefile.global to install valgrind > wrappers into the

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Tom Lane
Andres Freund writes: > On 2022-04-17 12:01:53 -0400, Tom Lane wrote: >> It seems likely to me that trying to make a test like this one blind to >> autovacuum/autoanalyze activity will make it less useful, not more so. >> Why is such blindness desirable? > Maybe I misunderstood - I thought you

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Andres Freund
Hi On 2022-04-17 12:01:53 -0400, Tom Lane wrote: > Andres Freund writes: > >> On 2022-04-16 13:11:59 -0400, Tom Lane wrote: > >>> We've seen this sort of problem before [2], and attempted to fix it [3] > >>> by making these tests ignore empty transactions. But of course > >>> auto-analyze's

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Tom Lane
Andres Freund writes: >> On 2022-04-16 13:11:59 -0400, Tom Lane wrote: >>> We've seen this sort of problem before [2], and attempted to fix it [3] >>> by making these tests ignore empty transactions. But of course >>> auto-analyze's transaction wasn't empty, so that didn't help. >> I don't

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

2022-04-17 Thread Tom Lane
Andres Freund writes: > On 2022-04-16 14:42:39 -0400, Tom Lane wrote: >> I think (based on the above argument) that what it intends to enforce >> is not really the system design we need, but it certainly isn't >> helping anyone that it enforces that design incorrectly. > I think it's

Re: Intermittent buildfarm failures on wrasse

2022-04-17 Thread Peter Geoghegan
On Sun, Apr 17, 2022 at 7:36 AM Andres Freund wrote: > > Part of the problem here is that we determine VACUUM's FreezeLimit by > > calculating `OldestXmin - vacuum_freeze_min_age` (more or less [1]). > > What the message outputs is OldestXmin and not FreezeLimit though. My higher level point is

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-17 Thread Andrew Dunstan
On 2022-04-17 Su 00:17, Michael Paquier wrote: >> Since it's quite painful to run TAP tests under valgrind, I found a >> place to stick a plain old sleep to repro these problems: > Actually, I am wondering how you are patching Cluster.pm to do that. I don't really think it's Cluster.pm's

Re: Intermittent buildfarm failures on wrasse

2022-04-17 Thread Andres Freund
Hi, On 2022-04-15 11:12:34 -0700, Peter Geoghegan wrote: > On Fri, Apr 15, 2022 at 10:43 AM Andres Freund wrote: > > I think it'd be interesting - particularly for large relations or when > > looking to adjust autovac cost limits. > > > Something like: > > removable cutoff: %u, age at start:

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Andres Freund
Hi, Adding Amit, I think this is stuff he worked on... On 2022-04-17 07:31:04 -0700, Andres Freund wrote: > On 2022-04-16 13:11:59 -0400, Tom Lane wrote: > > My pet dinosaur prairiedog just failed in the contrib/test_decoding > > tests [1]: > > > > diff -U3 > >

Re: Stabilizing the test_decoding checks, take N

2022-04-17 Thread Andres Freund
Hi, On 2022-04-16 13:11:59 -0400, Tom Lane wrote: > My pet dinosaur prairiedog just failed in the contrib/test_decoding > tests [1]: > > diff -U3 > /Users/buildfarm/bf-data/HEAD/pgsql.build/contrib/test_decoding/expected/stream.out > >

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

2022-04-17 Thread Andres Freund
Hi, On 2022-04-16 14:42:39 -0400, Tom Lane wrote: > In the meantime, since it's clear that HaveRegisteredOrActiveSnapshot > is failing to meet its contract, I'm going to go fix it. +1 > I think (based on the above argument) that what it intends to enforce > is not really the system design we

Re: GsoC: pgexporter: Custom file

2022-04-17 Thread Jesper Pedersen
Hi, On 4/17/22 04:43, dl x wrote: My name is Donglin Xie, a MSc. student at Zhejiang University, in China. I am interested in the project *pgexporter: Custom_file.* The proposal is attached to this email. Looking forward to the suggestions! Thanks for your proposal to Google Summer of

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-17 Thread Zhihong Yu
On Sun, Apr 17, 2022 at 1:48 AM Etsuro Fujita wrote: > On Mon, Apr 11, 2022 at 11:44 AM Zhihong Yu wrote: > > On Sun, Apr 10, 2022 at 7:41 PM Justin Pryzby > wrote: > >> On Sun, Apr 10, 2022 at 06:46:25AM -0700, Zhihong Yu wrote: > >> > Looking at the second hunk of the patch: > >> >

postgres_fdw: batch inserts vs. before row triggers

2022-04-17 Thread Etsuro Fujita
Hi, (I added Tomas in CC:.) One thing I noticed while reviewing the patch for fast copying into foreign tables/partitions using batch insert [1] is that in postgres_fdw we allow batch-inserting into foreign tables/partitions with before row triggers, but such triggers might query the target

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-17 Thread Etsuro Fujita
On Mon, Apr 11, 2022 at 11:44 AM Zhihong Yu wrote: > On Sun, Apr 10, 2022 at 7:41 PM Justin Pryzby wrote: >> On Sun, Apr 10, 2022 at 06:46:25AM -0700, Zhihong Yu wrote: >> > Looking at the second hunk of the patch: >> > FdwRoutine *fdwroutine = path->parent->fdwroutine; >> > ...