Re: Adding a pg_servername() function

2023-08-08 Thread Laetitia Avrot
Dear all, First, thank you so much for your interest in this patch. I didn't think I would have so many answers. I agree that the feature I'm suggesting could be done with a few tricks. I meant to simplify the life of the user by providing a simple new feature. (Also, I might have trust issues wi

Re: Handle infinite recursion in logical replication setup

2023-08-08 Thread Amit Kapila
On Wed, Aug 9, 2023 at 10:59 AM Peter Smith wrote: > > > Example output using the patch look like this: > > SINGLULAR > test_sub=# create subscription sub_test connection 'dbname=test_pub' > publication pub_all_at_pub with(origin=NONE); > WARNING: subscription "sub_test" requested copy_data with

Re: Adding a LogicalRepWorker type field

2023-08-08 Thread Amit Kapila
On Tue, Aug 8, 2023 at 1:39 PM Peter Smith wrote: > > On Fri, Aug 4, 2023 at 5:45 PM Peter Smith wrote: > > v4-0001 uses only 3 simple inline functions. Callers always pass > parameters as Bharath had suggested. > * - Assert(am_leader_apply_worker()); + Assert(is_leader_apply_worker(MyLogicalRep

Re: Incorrect handling of OOM in WAL replay leading to data loss

2023-08-08 Thread Michael Paquier
On Tue, Aug 08, 2023 at 05:44:03PM +0900, Kyotaro Horiguchi wrote: > I like the overall direction. Though, I'm considering enclosing the > errormsg and errorcode in a struct. Yes, this suggestion makes sense as it simplifies all the WAL routines that need to report back a complete error state, and

Re: Handle infinite recursion in logical replication setup

2023-08-08 Thread Peter Smith
On Wed, Aug 9, 2023 at 2:44 PM Amit Kapila wrote: > > On Wed, Aug 9, 2023 at 8:20 AM Peter Smith wrote: > > > > On Tue, Aug 8, 2023 at 6:52 PM Amit Kapila wrote: > > > > > > On Tue, Aug 8, 2023 at 1:50 PM Peter Eisentraut > > > wrote: > > > > > > > > This patch added the following error messag

Re: pgbench: allow to exit immediately when any client is aborted

2023-08-08 Thread Yugo NAGATA
On Wed, 9 Aug 2023 13:59:21 +0900 Yugo NAGATA wrote: > On Wed, 9 Aug 2023 10:46:38 +0900 > Yugo NAGATA wrote: > > > On Wed, 9 Aug 2023 02:15:01 +0200 (CEST) > > Fabien COELHO wrote: > > > > > > > > Hello Yugo-san, > > > > > > > There are cases where "goto done" is used where some error like

Re: 2023-08-10 release announcement draft

2023-08-08 Thread Noah Misch
On Mon, Aug 07, 2023 at 10:03:44PM -0400, Jonathan S. Katz wrote: > Fixes in PostgreSQL 16 Beta 3 > - > > The following includes fixes included in PostgreSQL 16 Beta 3: With both "includes" and "included" there, this line reads awkwardly to me. I'd just delete the line

Re: pgbench: allow to exit immediately when any client is aborted

2023-08-08 Thread Yugo NAGATA
On Wed, 9 Aug 2023 10:46:38 +0900 Yugo NAGATA wrote: > On Wed, 9 Aug 2023 02:15:01 +0200 (CEST) > Fabien COELHO wrote: > > > > > Hello Yugo-san, > > > > > There are cases where "goto done" is used where some error like > > > "invalid socket: ..." happens. I would like to make pgbench exit in

Re: pg_upgrade fails with in-place tablespace

2023-08-08 Thread Michael Paquier
On Wed, Aug 09, 2023 at 11:47:58AM +0800, Rui Zhao wrote: > I believe that utilizing appendPQExpBufferStr(buf, "''"); would be better and > more meaningful than using appendStringLiteralConn(buf, "", conn); in this > scenario. > I apologize for this wrong usage. Please help me fix it. > I will tr

Re: Handle infinite recursion in logical replication setup

2023-08-08 Thread Amit Kapila
On Wed, Aug 9, 2023 at 8:20 AM Peter Smith wrote: > > On Tue, Aug 8, 2023 at 6:52 PM Amit Kapila wrote: > > > > On Tue, Aug 8, 2023 at 1:50 PM Peter Eisentraut > > wrote: > > > > > > This patch added the following error message: > > > > > > errdetail_plural("Subscribed publication %s is subscri

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-08-08 Thread David Rowley
On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat wrote: > 0001 - to measure memory consumption during planning. This is the same > one as attached to [1]. I see you're recording the difference in the CurrentMemoryContext of palloc'd memory before and after planning. That won't really alert us to pro

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-08-08 Thread vignesh C
Hi Melih, Here is a patch to help in getting the execution at various phases like: a) replication slot creation time, b) Wal reading c) Number of WAL records read d) subscription relation state change etc Couple of observation while we tested with this patch: 1) We noticed that the patch takes mor

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-08 Thread Amit Kapila
On Wed, Aug 9, 2023 at 8:01 AM Masahiko Sawada wrote: > > On Mon, Aug 7, 2023 at 6:02 PM Amit Kapila wrote: > > > > On Mon, Aug 7, 2023 at 2:02 PM Masahiko Sawada > > wrote: > > > > > > WAL records for hint bit updates could be generated even in upgrading > > > mode? > > > > > > > Do you mean

Re: pg15: reltuples stuck at -1 after pg_upgrade and VACUUM

2023-08-08 Thread Peter Geoghegan
On Tue, Aug 8, 2023 at 8:43 PM Justin Pryzby wrote: > The problem isn't that reltuples == -1 after the upgrade (which is > normal). The issue is that if VACUUM skips all the pages, it can leave > reltuples -1. My expectation is that after running "vacuum", no tables > are left in the "never have

[BackendXidGetPid] only access allProcs when xid matches

2023-08-08 Thread Junwang Zhao
In function `BackendXidGetPid`, when looping every proc's TransactionId, there is no need to access its PGPROC since there is shared memory access: `arrayP->pgprocnos[index]`. Though the compiler can optimize this kind of inefficiency, I believe we should ship with better code. -

Re: pg_upgrade fails with in-place tablespace

2023-08-08 Thread Rui Zhao
I have found the patch and upon review, I believe the following code can be improved. + /* + * In-place tablespaces use a relative path, and need to be dumped + * with an empty string as location. + */ + if (is_absolute_path(spclocation)) + appendStringLiteralConn(buf, spclocation, conn); + else +

pg15: reltuples stuck at -1 after pg_upgrade and VACUUM

2023-08-08 Thread Justin Pryzby
Since 3d351d916 (pg14), reltuples -1 means that the rel has never been vacuumed nor analyzed. But since 4496020e6d (backpatched to pg15), following pg_upgrade, vacuum can leave reltuples=-1. commit 4496020e6dfaffe8217e4d3f85567bb2b6927b45 Author: Peter Geoghegan Date: Fri Aug 19 09:26:06 2022

Re: 2023-08-10 release announcement draft

2023-08-08 Thread Robert Treat
On Mon, Aug 7, 2023 at 9:15 PM Jonathan S. Katz wrote: > > Hi, > > Attached is the release announcement draft for the 2023-08-10 update > release, which also includes the release of PostgreSQL 16 Beta 3. > > Please provide your feedback no later than August 10, 2023 0:00 AoE[1]. > > Thanks, > > Jo

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 22:29:50 -0400, Robert Treat wrote: > In case it's helpful, from an SPI oriented perspective, $7K/month is > probably an order of magnitude more than what we can sustain, so I > don't see a way to make that work without some kind of additional > magic that includes other non-prof

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-08-08 Thread Zhijie Hou (Fujitsu)
On Thursday, August 3, 2023 7:30 PM Melih Mutlu wrote: > Right. I attached the v26 as you asked.  Thanks for posting the patches.   While reviewing the patch, I noticed one rare case that it's possible that there are two table sync worker for the same table in the same time. The patch relies o

Re: Handle infinite recursion in logical replication setup

2023-08-08 Thread Peter Smith
On Tue, Aug 8, 2023 at 6:52 PM Amit Kapila wrote: > > On Tue, Aug 8, 2023 at 1:50 PM Peter Eisentraut wrote: > > > > This patch added the following error message: > > > > errdetail_plural("Subscribed publication %s is subscribing to other > > publications.", > > "Subscribed publications %s are su

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-08 Thread Richard Guo
On Tue, Aug 8, 2023 at 7:34 PM Ashutosh Bapat wrote: > On Tue, Aug 8, 2023 at 12:44 PM Richard Guo > wrote: > > This is not an existing bug. Our current code (without this patch) > > would always call create_nestloop_path() after the reparameterization > > work has been done for the inner path.

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-08 Thread Masahiko Sawada
On Mon, Aug 7, 2023 at 6:02 PM Amit Kapila wrote: > > On Mon, Aug 7, 2023 at 2:02 PM Masahiko Sawada wrote: > > > > On Mon, Aug 7, 2023 at 12:54 PM Amit Kapila wrote: > > > > > > On Sun, Aug 6, 2023 at 6:02 PM Masahiko Sawada > > > wrote: > > > > > > > > IIUC the above query checks if the WAL

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Robert Treat
On Tue, Aug 8, 2023 at 9:26 PM Andres Freund wrote: > > Hi, > > On 2023-08-08 11:58:25 +0300, Heikki Linnakangas wrote: > > On 08/08/2023 05:15, Andres Freund wrote: > > > With the improvements detailed below, cirrus' free CI would last > > > about ~65 runs / month. > > > > I think that's plenty.

Re: pgbench: allow to exit immediately when any client is aborted

2023-08-08 Thread Yugo NAGATA
On Wed, 9 Aug 2023 02:15:01 +0200 (CEST) Fabien COELHO wrote: > > Hello Yugo-san, > > > There are cases where "goto done" is used where some error like > > "invalid socket: ..." happens. I would like to make pgbench exit in > > such cases, too, so I chose to handle errors below the "done:" labe

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 11:58:25 +0300, Heikki Linnakangas wrote: > On 08/08/2023 05:15, Andres Freund wrote: > > With the improvements detailed below, cirrus' free CI would last > > about ~65 runs / month. > > I think that's plenty. Not so sure, I would regularly exceed it, I think. But it definitely

Re: pg_upgrade fails with in-place tablespace

2023-08-08 Thread Michael Paquier
On Tue, Aug 08, 2023 at 04:32:45PM +0900, Michael Paquier wrote: > The pg_dump part should be OK to allow the restore of in-place > tablespaces, so I'll get that fixed first, but I think that we should > add a regression test as well. I have added a test for that in 002_pg_dump.pl, and applied the

Re: Support to define custom wait events for extensions

2023-08-08 Thread Andres Freund
Hi, On 2023-08-09 08:03:29 +0900, Michael Paquier wrote: > On Tue, Aug 08, 2023 at 03:59:54PM -0700, Andres Freund wrote: > > On 2023-08-08 08:54:10 +0900, Michael Paquier wrote: > >> - WaitEventExtensionShmemInit() should gain a dshash_create(), to make > >> sure that the shared table is around,

Re: pgbench: allow to exit immediately when any client is aborted

2023-08-08 Thread Fabien COELHO
Hello Yugo-san, There are cases where "goto done" is used where some error like "invalid socket: ..." happens. I would like to make pgbench exit in such cases, too, so I chose to handle errors below the "done:" label. However, we can change it to call "exit" instead of "goo done" at each place

Re: Support to define custom wait events for extensions

2023-08-08 Thread Michael Paquier
On Tue, Aug 08, 2023 at 03:59:54PM -0700, Andres Freund wrote: > On 2023-08-08 08:54:10 +0900, Michael Paquier wrote: >> - WaitEventExtensionShmemInit() should gain a dshash_create(), to make >> sure that the shared table is around, and we are going to have a >> reference to it in WaitEventExtensio

Re: Support to define custom wait events for extensions

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 08:54:10 +0900, Michael Paquier wrote: > - WaitEventExtensionShmemInit() should gain a dshash_create(), to make > sure that the shared table is around, and we are going to have a > reference to it in WaitEventExtensionCounterData, saved from > dshash_get_hash_table_handle(). I'm

Re: Support to define custom wait events for extensions

2023-08-08 Thread Michael Paquier
On Tue, Aug 08, 2023 at 08:40:53PM +0900, Masahiro Ikeda wrote: > I accidentally attached a patch in one previous email. > But, you don't need to check it, sorry. > (v1-0001-Change-to-manage-custom-wait-events-in-shared-hash.patch) Sure, no worries. With that in place, the init function in worker

Re: Configurable FP_LOCK_SLOTS_PER_BACKEND

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 16:44:37 -0400, Robert Haas wrote: > On Mon, Aug 7, 2023 at 6:05 PM Andres Freund wrote: > > I think the biggest flaw of the locking scheme is that the LockHash locks > > protect two, somewhat independent, things: > > 1) the set of currently lockable objects, i.e. the entries in

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Peter Geoghegan
On Tue, Aug 8, 2023 at 1:49 PM Tomas Vondra wrote: > So we expect 1250 rows. If that was accurate, the index scan would have > to do 1250 heap fetches. It's just luck the index scan doesn't need to > do that. I don't this there's a chance to improve this costing - if the > inputs are this off, it

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Peter Geoghegan
On Tue, Aug 8, 2023 at 1:24 PM Tomas Vondra wrote: > > Assuming that that happens, then it immediately gives index scans a > > huge advantage over bitmap index scans. At that point it seems > > important to describe (in high level terms) where it is that the > > advantage is innate, and where it's

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Tomas Vondra
On 8/8/23 21:15, Peter Geoghegan wrote: > On Tue, Aug 8, 2023 at 11:36 AM Peter Geoghegan wrote: >> Assuming that that happens, then it immediately gives index scans a >> huge advantage over bitmap index scans. At that point it seems >> important to describe (in high level terms) where it is that

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 18:34:58 +0200, Alvaro Herrera wrote: > On 2023-Aug-08, Andres Freund wrote: > > > Given the cost of macos, it seems like it'd be by far the most of affordable > > to just buy 1-2 mac minis (2x ~660USD) and stick them in a shelf somewhere, > > as > > persistent runners. Cirrus

Re: Configurable FP_LOCK_SLOTS_PER_BACKEND

2023-08-08 Thread Robert Haas
On Mon, Aug 7, 2023 at 6:05 PM Andres Freund wrote: > I think the biggest flaw of the locking scheme is that the LockHash locks > protect two, somewhat independent, things: > 1) the set of currently lockable objects, i.e. the entries in the hash table > [partition] > 2) the state of all the locks

Re: Extract numeric [field] in JSONB more effectively

2023-08-08 Thread Chapman Flack
Hi, Looking at the most recent patch, so far I have a minor spelling point, and a question (which I have not personally explored). The minor spelling point, the word 'field' has been spelled 'filed' throughout this comment (just as in the email subject): + /* +* Si

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Tomas Vondra
On 8/8/23 20:36, Peter Geoghegan wrote: > On Tue, Aug 8, 2023 at 11:14 AM Tomas Vondra > wrote: >> I agree this patch shouldn't make it harder to improve these cases, but >> TBH I don't quite see which part of the patch would do that? Which bit >> are you objecting to? If we decide to change ho

Re: should frontend tools use syncfs() ?

2023-08-08 Thread Nathan Bossart
I ran a couple of tests for pg_upgrade with 100k tables (created using the script here [0]) in order to demonstrate the potential benefits of this patch. pg_upgrade --sync-method fsync real5m50.072s user0m10.606s sys 0m40.298s pg_upgrade --sync-method syncfs real3m

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Peter Geoghegan
On Tue, Aug 8, 2023 at 11:36 AM Peter Geoghegan wrote: > Assuming that that happens, then it immediately gives index scans a > huge advantage over bitmap index scans. At that point it seems > important to describe (in high level terms) where it is that the > advantage is innate, and where it's jus

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Peter Geoghegan
On Tue, Aug 8, 2023 at 11:14 AM Tomas Vondra wrote: > I agree this patch shouldn't make it harder to improve these cases, but > TBH I don't quite see which part of the patch would do that? Which bit > are you objecting to? If we decide to change how match_clause_to_index() > deals with these cases

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Tomas Vondra
On 8/8/23 19:43, Peter Geoghegan wrote: > On Mon, Aug 7, 2023 at 9:21 PM Peter Geoghegan wrote: >> I see that get_op_btree_interpretation() will treat != as a kind of >> honorary member of an opfamily whose = operator has our != operator as >> its negator. Perhaps we should be finding a way to

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Peter Geoghegan
On Mon, Aug 7, 2023 at 9:21 PM Peter Geoghegan wrote: > I see that get_op_btree_interpretation() will treat != as a kind of > honorary member of an opfamily whose = operator has our != operator as > its negator. Perhaps we should be finding a way to pass != quals into > the index AM so that they b

Re: pgsql: Some refactoring to export json(b) conversion functions

2023-08-08 Thread Alvaro Herrera
On 2023-Jul-26, Amit Langote wrote: > Some refactoring to export json(b) conversion functions > > This is to export datum_to_json(), datum_to_jsonb(), and > jsonb_from_cstring(), though the last one is exported as > jsonb_from_text(). After this commit, Coverity started complaining that datum_to

Re: A failure in 031_recovery_conflict.pl on Debian/s390x

2023-08-08 Thread Thomas Munro
On Wed, Aug 9, 2023 at 2:01 AM Christoph Berg wrote: > Putting that patch on top of v8 made it pass 294 times before exiting > like this: > > [08:52:34.134](0.032s) ok 1 - buffer pin conflict: cursor with conflicting > pin established > Waiting for replication conn standby's replay_lsn to pass 0/

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Alvaro Herrera
On 2023-Aug-08, Andres Freund wrote: > Given the cost of macos, it seems like it'd be by far the most of affordable > to just buy 1-2 mac minis (2x ~660USD) and stick them in a shelf somewhere, as > persistent runners. Cirrus has builtin macos virtualization support - but can > only host two VMs o

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Peter Geoghegan
On Tue, Aug 8, 2023 at 7:28 AM Tomas Vondra wrote: > Are you sure? Because if I try with the 20230716 patch, I get this plan > (after disabling bitmapscan): > So the condition is recognized as index filter. Or did you mean > something different? No, I was just wrong about this (and about inequal

Re: generic plans and "initial" pruning

2023-08-08 Thread Robert Haas
On Tue, Aug 8, 2023 at 10:32 AM Amit Langote wrote: > But should ExecInitNode() subroutines return the partially initialized > PlanState node or NULL on detecting invalidation? If I'm > understanding how you think this should be working correctly, I think > you mean the former, because if it were

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Tristan Partin
On Tue Aug 8, 2023 at 10:38 AM CDT, Andres Freund wrote: Hi, On 2023-08-08 10:25:52 -0500, Tristan Partin wrote: > On Mon Aug 7, 2023 at 9:15 PM CDT, Andres Freund wrote: > > FWIW: with the patches applied, the "credit costs" in cirrus CI are roughly > > like the following (depends on caching et

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 10:25:52 -0500, Tristan Partin wrote: > On Mon Aug 7, 2023 at 9:15 PM CDT, Andres Freund wrote: > > FWIW: with the patches applied, the "credit costs" in cirrus CI are roughly > > like the following (depends on caching etc): > > > > task costs in credits > > linux-sanity: 0.

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Tristan Partin
On Mon Aug 7, 2023 at 9:15 PM CDT, Andres Freund wrote: FWIW: with the patches applied, the "credit costs" in cirrus CI are roughly like the following (depends on caching etc): task costs in credits linux-sanity: 0.01 linux-compiler-warnings: 0.05 linux-meson: 0.07

Re: Fix last unitialized memory warning

2023-08-08 Thread Tristan Partin
On Tue Aug 8, 2023 at 5:20 AM CDT, Peter Eisentraut wrote: On 19.07.23 19:15, Tristan Partin wrote: > On Sun Jul 9, 2023 at 2:23 AM CDT, Peter Eisentraut wrote: >> On 06.07.23 15:41, Tristan Partin wrote: >> > On Thu Jul 6, 2023 at 3:21 AM CDT, Peter Eisentraut wrote: >> >> On 05.07.23 23:06, Tri

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Andres Freund
Hi, On 2023-08-08 16:28:49 +0200, Peter Eisentraut wrote: > On 08.08.23 04:15, Andres Freund wrote: > > Potential paths forward for cfbot, in addition to the above: > > > > - Pay for compute / ask the various cloud providers to grant us compute > >credits. At least some of the cloud providers

Re: generic plans and "initial" pruning

2023-08-08 Thread Amit Langote
On Tue, Aug 8, 2023 at 12:36 AM Robert Haas wrote: > On Thu, Aug 3, 2023 at 4:37 AM Amit Langote wrote: > > Here's a patch set where the refactoring to move the ExecutorStart() > > calls to be closer to GetCachedPlan() (for the call sites that use a > > CachedPlan) is extracted into a separate pa

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Tomas Vondra
On 8/8/23 06:21, Peter Geoghegan wrote: > On Mon, Aug 7, 2023 at 3:18 PM Peter Geoghegan wrote: >> Even my patch cannot always make SAOP clauses into index quals. There >> are specific remaining gaps that I hope that your patch will still >> cover. The simplest example is a similar NOT IN() inequa

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Peter Eisentraut
On 08.08.23 04:15, Andres Freund wrote: Potential paths forward for individual CI: - migrate wholesale to another CI provider - split CI tasks across different CI providers, rely on github et al displaying the CI status for different platforms - give up With the proposed optimizations, it

Re: Use of additional index columns in rows filtering

2023-08-08 Thread Tomas Vondra
On 8/8/23 00:18, Peter Geoghegan wrote: > On Mon, Aug 7, 2023 at 12:34 PM Tomas Vondra > wrote: >> But then we call get_index_paths/build_index_path a little bit later, >> and that decides to skip "lower SAOP" (which seems a bit strange, >> because the column is "after" the equality, but meh).

Re: initial pruning in parallel append

2023-08-08 Thread Robert Haas
On Tue, Aug 8, 2023 at 2:58 AM Amit Langote wrote: > > That doesn't seem like a big problem because there aren't many node > > types that do pruning, right? I think we're just talking about Append > > and MergeAppend, or something like that, right? > > MergeAppend can't be parallel-aware atm, so o

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-08-08 Thread Ashutosh Bapat
On Tue, Aug 8, 2023 at 4:08 PM Richard Guo wrote: > > > I haven't looked into the details but with 0002 patch I came across a > crash while planning the query below. > > regression=# set enable_partitionwise_join to on; > SET > regression=# EXPLAIN (COSTS OFF) > SELECT * FROM prt1 t1, prt2 t2 > WH

Re: A failure in 031_recovery_conflict.pl on Debian/s390x

2023-08-08 Thread Christoph Berg
Re: Andres Freund > Hm, that could just be a "harmless" race. Does it still happen if you apply > the attached patch in addition? Putting that patch on top of v8 made it pass 294 times before exiting like this: [08:52:34.134](0.032s) ok 1 - buffer pin conflict: cursor with conflicting pin establ

Re: 2023-08-10 release announcement draft

2023-08-08 Thread Jonathan S. Katz
On 8/8/23 1:30 AM, Erik Rijkers wrote: Op 8/8/23 om 03:15 schreef Jonathan S. Katz: Please provide your feedback no later than August 10, 2023 0:00 AoE[1]. 'You us'  should be 'You use'    (2x) It should actually be just "Use" -- but I've fixed both instances. Thanks! Jonathan OpenPGP

Re: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-08-08 Thread Amit Kapila
On Mon, Aug 7, 2023 at 1:46 PM José Neves wrote: > > Humm, that's... challenging. I faced some issues after "the fix" because I > had a couple of transactions with 25k updates, and I had to split it to be > able to push to our event messaging system, as our max message size is 10MB. > Relying o

Re: Support to define custom wait events for extensions

2023-08-08 Thread Masahiro Ikeda
I accidentally attached a patch in one previous email. But, you don't need to check it, sorry. (v1-0001-Change-to-manage-custom-wait-events-in-shared-hash.patch) Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: Support to define custom wait events for extensions

2023-08-08 Thread Masahiro Ikeda
On 2023-08-08 10:05, Michael Paquier wrote: On Tue, Aug 08, 2023 at 09:39:02AM +0900, Masahiro Ikeda wrote: I am thinking a bit that we also need another hash where the key is a custom string. For extensions that have no dependencies with shared_preload_libraries, I think we need to avoid that W

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-08 Thread Ashutosh Bapat
On Tue, Aug 8, 2023 at 12:44 PM Richard Guo wrote: > > > I agree that we should mention in the function's comment that > reparameterize_path_by_child can only be used after the best path has > been selected because the RTE might be modified by this function. I'm > not sure if it's a good idea to

Re: How to build a new grammer for pg?

2023-08-08 Thread Hannu Krosing
I would look at how Babelfish DB did it when adding SQL Server compatibility https://babelfishpg.org/ and https://github.com/babelfish-for-postgresql/ another source to inspect could be https://github.com/IvorySQL/IvorySQL for "oracle compatible PostgreSQL" On Tue, Aug 1, 2023 at 10:07 PM Jonah

Re: Avoid stack frame setup in performance critical routines using tail calls

2023-08-08 Thread John Naylor
On Wed, Jul 19, 2023 at 3:53 PM Andres Freund wrote: > > Hi, > > David and I were chatting about this patch, in the context of his bump > allocator patch. Attached is a rebased version that is also split up into two > steps, and a bit more polished. Here is a quick test -- something similar was

Suppress generating WAL records during the upgrade

2023-08-08 Thread Hayato Kuroda (Fujitsu)
Dear hackers, (CC: Julien, Sawada-san, Amit) This is a fork thread from "[PoC] pg_upgrade: allow to upgrade publisher node" [1]. # Background [1] is the patch which allows to replicate logical replication slots from old to new node. Followings describe the rough steps: 1. Boot old node as bin

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-08-08 Thread Richard Guo
On Thu, Jul 27, 2023 at 10:06 PM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > 0002 - WIP patch to avoid repeated translations of RestrictInfo. > The WIP patch avoids repeated translations by tracking the child for > which a RestrictInfo is translated and reusing the same translation > e

Re: [PATCH] psql: Add tab-complete for optional view parameters

2023-08-08 Thread Christoph Heiss
On Tue, Aug 08, 2023 at 09:17:51AM +0100, Dean Rasheed wrote: > > On Mon, 7 Aug 2023 at 19:49, Christoph Heiss wrote: > > > > On Fri, Jan 06, 2023 at 12:18:44PM +, Dean Rasheed wrote: > > > Hmm, I don't think we should be offering "check_option" as a tab > > > completion for CREATE VIEW at a

Re: Fix last unitialized memory warning

2023-08-08 Thread Peter Eisentraut
On 19.07.23 19:15, Tristan Partin wrote: On Sun Jul 9, 2023 at 2:23 AM CDT, Peter Eisentraut wrote: On 06.07.23 15:41, Tristan Partin wrote: > On Thu Jul 6, 2023 at 3:21 AM CDT, Peter Eisentraut wrote: >> On 05.07.23 23:06, Tristan Partin wrote: >>> Thanks for following up. My system is Fedora 3

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-08-08 Thread Richard Guo
On Tue, Aug 8, 2023 at 4:40 PM Etsuro Fujita wrote: > I modified the code a bit further to use an if-test to avoid a useless > function call, and added/tweaked comments and docs further. Attached > is a new version of the patch. I am planning to commit this, if there > are no objections. +1 t

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

2023-08-08 Thread Heikki Linnakangas
On 08/08/2023 05:15, Andres Freund wrote: IMO, for the individual user case it's important to use CI for "free", without a whole lot of complexity. Which imo rules approaches like providing $cloud_provider compute accounts, that's too much setup work. +1 With the improvements detailed below,

Re: Handle infinite recursion in logical replication setup

2023-08-08 Thread Amit Kapila
On Tue, Aug 8, 2023 at 1:50 PM Peter Eisentraut wrote: > > On 12.09.22 07:23, vignesh C wrote: > > On Fri, 9 Sept 2022 at 11:12, Amit Kapila wrote: > >> > >> On Thu, Sep 8, 2022 at 9:32 AM vignesh C wrote: > >>> > >>> > >>> The attached patch has the changes to handle the same. > >>> > >> > >> P

Re: Incorrect handling of OOM in WAL replay leading to data loss

2023-08-08 Thread Kyotaro Horiguchi
At Tue, 8 Aug 2023 16:29:49 +0900, Michael Paquier wrote in > On Wed, Aug 02, 2023 at 01:16:02PM +0900, Kyotaro Horiguchi wrote: > > I believe this approach is sufficient to determine whether the error > > is OOM or not. If total_len is currupted and has an excessively large > > value, it's high

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-08-08 Thread Etsuro Fujita
Hi Richard, On Mon, Jul 31, 2023 at 5:52 PM Richard Guo wrote: > On Fri, Jul 28, 2023 at 4:56 PM Etsuro Fujita wrote: >> here is a rebased version of the second patch, in which I modified the >> ForeignPath and CustomPath cases in reparameterize_path_by_child() to >> reflect the new members fdw_

Re: Handle infinite recursion in logical replication setup

2023-08-08 Thread Peter Eisentraut
On 12.09.22 07:23, vignesh C wrote: On Fri, 9 Sept 2022 at 11:12, Amit Kapila wrote: On Thu, Sep 8, 2022 at 9:32 AM vignesh C wrote: The attached patch has the changes to handle the same. Pushed. I am not completely sure whether we want the remaining documentation patch in this thread i

Re: [PATCH] psql: Add tab-complete for optional view parameters

2023-08-08 Thread Dean Rasheed
On Mon, 7 Aug 2023 at 19:49, Christoph Heiss wrote: > > On Fri, Jan 06, 2023 at 12:18:44PM +, Dean Rasheed wrote: > > Hmm, I don't think we should be offering "check_option" as a tab > > completion for CREATE VIEW at all, since that would encourage users to > > use non-SQL-standard syntax, rat

Re: WIP: new system catalog pg_wait_event

2023-08-08 Thread Drouvot, Bertrand
Hi, On 8/8/23 7:37 AM, Drouvot, Bertrand wrote: Hi, On 8/8/23 5:05 AM, Michael Paquier wrote: On Tue, Aug 08, 2023 at 11:53:32AM +0900, Kyotaro Horiguchi wrote: As I mentioned in another thread, I'm uncertain about our stance on the class id of the wait event. If a class acts as a namespace,

Re: Adding a LogicalRepWorker type field

2023-08-08 Thread Peter Smith
PSA v4 patches. On Fri, Aug 4, 2023 at 5:45 PM Peter Smith wrote: > > Thank you for the feedback received so far. Sorry, I have become busy > lately with other work. > > IIUC there is a general +1 for the idea, but I still have some > juggling necessary to make the functions/macros of patch 0001

Re: pg_upgrade fails with in-place tablespace

2023-08-08 Thread Michael Paquier
On Tue, Aug 08, 2023 at 12:35:38PM +0800, Rui Zhao wrote: > I have only enabled allow_in_place_tablespaces to true during a binary > upgrade. The pg_dump part should be OK to allow the restore of in-place tablespaces, so I'll get that fixed first, but I think that we should add a regression test

Re: Doc limitation update proposal: include out-of-line OID usage per TOAST-ed columns

2023-08-08 Thread John Naylor
On Wed, Jul 5, 2023 at 9:45 PM Jakub Wartak wrote: > [v3] --- a/doc/src/sgml/limits.sgml +++ b/doc/src/sgml/limits.sgml @@ -10,6 +10,7 @@ hard limits are reached. + @@ -92,11 +93,25 @@ can be increased by recompiling PostgreSQL - -partition keys -32 -can be

Re: Incorrect handling of OOM in WAL replay leading to data loss

2023-08-08 Thread Michael Paquier
On Wed, Aug 02, 2023 at 01:16:02PM +0900, Kyotaro Horiguchi wrote: > I believe this approach is sufficient to determine whether the error > is OOM or not. If total_len is currupted and has an excessively large > value, it's highly unlikely that all subsequent pages for that length > will be consist

Re: [PATCH] Add loongarch native checksum implementation.

2023-08-08 Thread YANG Xudong
On 2023/8/8 14:38, John Naylor wrote: It seems that platforms capable of running Postgres only support 64 bit. I think so. > So I guess using aligned memory access is necessary and I have updated > the comment in the code. Okay, so it's not "necessary" in the sense that it's illegal,

Re: cpluspluscheck vs ICU

2023-08-08 Thread Peter Eisentraut
On 08.08.23 01:35, Andres Freund wrote: Hi, On 2023-03-10 20:10:30 -0800, Andres Freund wrote: On 2023-03-10 19:37:27 -0800, Andres Freund wrote: I just hit this once more - and I figured out a fairly easy fix: We just need a #ifndef U_DEFAULT_SHOW_DRAFT #define U_DEFAULT_SHOW_DRAFT 0

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-08 Thread Richard Guo
On Mon, Aug 7, 2023 at 9:29 PM Ashutosh Bapat wrote: > Thanks. The patch looks good overall. I like it because of its potential > to reduce memory consumption in reparameterization. That's cake on top of > cream :) > Thanks for reviewing this patch! > Some comments here. > > + { > + FLAT_COPY_