Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Joe Conway
On 2/19/24 13:13, Andres Freund wrote: On 2024-02-19 09:19:16 -0500, Joe Conway wrote: Couldn't we scale the rounding, e.g. allow small allocations as we do now, but above some number always round? E.g. maybe >= 2GB round to the nearest 256MB, >= 4GB round to the nearest 512MB, >= 8GB round to

Re: Patch: Add parse_type Function

2024-02-19 Thread Tom Lane
"David E. Wheeler" writes: > The only way I can think of to avoid that is to: > 1. Add a to_regtypmod() for those who just want the typemod. Seems like there's a good case for doing that. > 2. Add a format_type_string() function that returns a string, the equivalent > of this function but in

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Tom Lane
Tomas Vondra writes: > On 2/19/24 16:45, Tom Lane wrote: >> Tomas Vondra writes: >>> For example, I don't think we expect selectivity functions to allocate >>> long-lived objects, right? So maybe we could run them in a dedicated >>> memory context, and reset it aggressively (after each call).

Re: speed up a logical replica setup

2024-02-19 Thread Euler Taveira
On Mon, Feb 19, 2024, at 6:47 AM, Peter Eisentraut wrote: > Some review of the v21 patch: Thanks for checking. > - commit message > > Mention pg_createsubscriber in the commit message title. That's the > most important thing that someone doing git log searches in the future > will be looking

Re: Injection points: some tools to wait and wake

2024-02-19 Thread Michael Paquier
On Mon, Feb 19, 2024 at 02:28:04PM +, Bertrand Drouvot wrote: > +CREATE FUNCTION injection_points_wake() > > what about injection_points_wakeup() instead? Sure. > +/* Shared state information for injection points. */ > +typedef struct InjectionPointSharedState > +{ > + /* protects

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-19 Thread Jelte Fennema-Nio
On Tue, 20 Feb 2024 at 00:34, Ian Lawrence Barwick wrote: > With the addition of "pg_sync_replication_slots()", there is now a use-case > for > including "dbname" in "primary_conninfo" and the docs have changed from > stating [1]: > > Do not specify a database name in the primary_conninfo

Re: Proposal: Adjacent B-Tree index

2024-02-19 Thread Matthias van de Meent
On Mon, 19 Feb 2024 at 18:48, Dilshod Urazov wrote: > > - Motivation > > A regular B-tree index provides efficient mapping of key values to tuples > within a table. However, if you have two tables connected in some way, a > regular B-tree index may not be efficient enough. In this case, you

Re: Injection points: some tools to wait and wake

2024-02-19 Thread Michael Paquier
On Mon, Feb 19, 2024 at 11:54:20AM +0300, Andrey M. Borodin wrote: > 1. injection_points_wake() will wake all of waiters. But it's not > suitable for complex tests. I think there must be a way to wake only > specific waiter by injection point name. I don't disagree with that, but I don't have a

Re: Patch: Add parse_type Function

2024-02-19 Thread Tom Lane
I wrote: > I'm less thrilled about that, mainly because I can't think of > a good name for it ("format_type_string" is certainly not that). After some time ruminating, a couple of possibilities occurred to me: reformat_type(text) canonical_type_name(text) > Is the use-case for

Re: 035_standby_logical_decoding unbounded hang

2024-02-19 Thread Noah Misch
On Fri, Feb 16, 2024 at 06:37:38AM +, Bertrand Drouvot wrote: > On Thu, Feb 15, 2024 at 12:48:16PM -0800, Noah Misch wrote: > > On Wed, Feb 14, 2024 at 03:31:16PM +, Bertrand Drouvot wrote: > > > What about creating a sub, say wait_for_restart_lsn_calculation() in > > > Cluster.pm > > >

Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-19 Thread Ian Lawrence Barwick
Hi Hi With the addition of "pg_sync_replication_slots()", there is now a use-case for including "dbname" in "primary_conninfo" and the docs have changed from stating [1]: Do not specify a database name in the primary_conninfo string. to [2]: For replication slot synchronization (see

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-02-19 Thread Michael Paquier
On Mon, Feb 19, 2024 at 09:49:24AM +, Bertrand Drouvot wrote: > On Mon, Feb 19, 2024 at 01:45:16PM +0530, Bharath Rupireddy wrote: >> Prefix 'initial_' makes the variable names a bit longer, I think we >> can just use effective_xmin, catalog_effective_xmin and restart_lsn, >> the code updating

Re: serial not accepted as datatype in ALTER TABLE ... ALTER COLUMN

2024-02-19 Thread Andy Fan
Ashutosh Bapat writes: > On Sun, Feb 18, 2024 at 1:59 PM Andy Fan wrote: >> >> >> I tried your idea with the attatchment, it is still in a drafted state >> but it can be used as a prove-of-concept and for better following >> communicating. Just one point needs to metion is serial implies >>

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 13:54:01 -0500, Joe Conway wrote: > On 2/19/24 13:13, Andres Freund wrote: > > On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > > > Couldn't we scale the rounding, e.g. allow small allocations as we do now, > > > but above some number always round? E.g. maybe >= 2GB round to

Re: Patch: Add parse_type Function

2024-02-19 Thread David E. Wheeler
On Feb 19, 2024, at 15:47, Tom Lane wrote: >> 1. Add a to_regtypmod() for those who just want the typemod. > > Seems like there's a good case for doing that. I’ll work on that. > I'm less thrilled about that, mainly because I can't think of > a good name for it ("format_type_string" is

Re: Why is pq_begintypsend so slow?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 10:02:52 +0900, Sutou Kouhei wrote: > In <20240218200906.zvihkrs46yl2j...@awork3.anarazel.de> > "Re: Why is pq_begintypsend so slow?" on Sun, 18 Feb 2024 12:09:06 -0800, > Andres Freund wrote: > > >> [1] > >>

Re: Avoid switching between system-user and system-username in the doc

2024-02-19 Thread Michael Paquier
On Mon, Feb 19, 2024 at 06:00:11PM +0100, Jelte Fennema-Nio wrote: > On Mon, 19 Feb 2024 at 09:52, Bertrand Drouvot > wrote: >> Please find attached a tiny patch to clean that up. > > LGTM Looks like a mistake from me in efb6f4a4f9b6, will fix and backpatch for consistency. -- Michael

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-19 Thread wenhui qiu
Hi Heikki Linnakangas I saw git log found this commit: https://github.com/postgres/postgres/commit/3acc10c997f916f6a741d0b4876126b7b08e3892 ,I don't seem to see an email discussing this commit. As the commit log tells us, we don't know exactly how large a value is optimal, and I believe it's

Re: Add last_commit_lsn to pg_stat_database

2024-02-19 Thread Michael Paquier
On Mon, Feb 19, 2024 at 10:26:43AM +0100, Tomas Vondra wrote: > On 2/19/24 07:57, Michael Paquier wrote: > > On Sun, Feb 18, 2024 at 02:28:06AM +0100, Tomas Vondra wrote: >>> 1) Do we really need to modify RecordTransactionCommitPrepared and >>> XactLogCommitRecord to return the LSN of the commit

Re: Streaming read-ready sequential scan code

2024-02-19 Thread Melanie Plageman
On Mon, Jan 29, 2024 at 4:17 PM Melanie Plageman wrote: > > There is an outstanding question about where to allocate the > PgStreamingRead object for sequential scans I've written three alternative implementations of the actual streaming read user for sequential scan which handle the question of

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-19 Thread wenhui qiu
Hi Japlin Li Thank you for such important information ! Got it Japin Li 于2024年2月19日周一 10:26写道: > > On Mon, 19 Feb 2024 at 00:56, Tomas Vondra > wrote: > > On 2/18/24 03:30, Li Japin wrote: > >> > >> I find it seems need to change MAX_SIMUL_LWLOCKS if we enlarge the > NUM_BUFFER_PARTITIONS,

Possible to trigger autovacuum?

2024-02-19 Thread Chris Cleveland
Is it possible to launch an autovacuum from within an extension? I'm developing an index access method. After the index gets built it needs some cleanup and optimization. I'd prefer to do this in the amvacuumcleanup() method so it can happen periodically and asynchronously. I could fire up a

Re: Possible to trigger autovacuum?

2024-02-19 Thread Michael Paquier
On Mon, Feb 19, 2024 at 03:15:29PM -0600, Chris Cleveland wrote: > Is it possible to launch an autovacuum from within an extension? > > I'm developing an index access method. After the index gets built it needs > some cleanup and optimization. I'd prefer to do this in the > amvacuumcleanup()

Re: speed up a logical replica setup

2024-02-19 Thread Euler Taveira
On Mon, Feb 19, 2024, at 7:22 AM, Shlok Kyal wrote: > I have reviewed the v21 patch. And found an issue. > > Initially I started the standby server with a new postgresql.conf file > (not the default postgresql.conf that is present in the instance). > pg_ctl -D ../standby start -o "-c

Re: speed up a logical replica setup

2024-02-19 Thread Shlok Kyal
Hi, On Tue, 20 Feb 2024 at 06:59, Euler Taveira wrote: > > On Mon, Feb 19, 2024, at 7:22 AM, Shlok Kyal wrote: > > I have reviewed the v21 patch. And found an issue. > > Initially I started the standby server with a new postgresql.conf file > (not the default postgresql.conf that is present in

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Andrei Lepikhov
On 20/2/2024 04:51, Tom Lane wrote: Tomas Vondra writes: On 2/19/24 16:45, Tom Lane wrote: Tomas Vondra writes: For example, I don't think we expect selectivity functions to allocate long-lived objects, right? So maybe we could run them in a dedicated memory context, and reset it

Re: WIP Incremental JSON Parser

2024-02-19 Thread Andrew Dunstan
On 2024-01-26 Fr 12:15, Andrew Dunstan wrote: On 2024-01-24 We 13:08, Robert Haas wrote: Maybe you should adjust your patch to dump the manifests into the log file with note(). Then when cfbot runs on it you can see exactly what the raw file looks like. Although I wonder if it's possible

Re: JIT compilation per plan node

2024-02-19 Thread David Rowley
On Tue, 20 Feb 2024 at 05:26, Tomas Vondra wrote: > I doubt CreatePlanContext is a great way to achieve this. For one, it > breaks the long-standing custom that PlannerInfo is the first parameter, > usually followed by RelOptInfo, etc. CreatePlanContext is added to some > functions (but not all),

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2024-02-19 Thread Bharath Rupireddy
On Sat, Feb 17, 2024 at 10:27 AM Bharath Rupireddy wrote: > > On Fri, Feb 16, 2024 at 11:01 PM Jeff Davis wrote: > > > > > Here, I'm with v23 patch set: > > > > Thank you, I'll look at these. > > Thanks. Here's the v24 patch set after rebasing. Ran pgperltidy on the new TAP test file added.

Support boolcol IS [NOT] UNKNOWN in partition pruning

2024-02-19 Thread David Rowley
While working on 4c2369ac5, I noticed there's close to as much code to disallow BooleanTests in the form of "IS UNKNOWN" and "IS NOT UNKNOWN" in partition pruning as it would take to allow pruning to work for these. The attached makes it work. David From b9f3ff909652c96f1f0dced9e1165ffa8c93c7f1

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:25, Ashutosh Bapat wrote: On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: Live example: right now, I am working on the code like MSSQL has - a combination of NestLoop and HashJoin paths and switching between them in real-time. It requires both paths in the path list at

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-02-19 Thread Bharath Rupireddy
On Fri, Feb 9, 2024 at 1:12 PM Bertrand Drouvot wrote: > > I think "conflict" is an important topic and does contain several reasons. The > slot "first" conflict and then leads to slot "invalidation". > > > They both are the same internally, so why > > confuse the users? > > I don't think that

Re: Switching XLog source from archive to streaming when primary available

2024-02-19 Thread Bharath Rupireddy
On Tue, Feb 20, 2024 at 11:54 AM Japin Li wrote: > > > On Tue, 20 Feb 2024 at 13:40, Bharath Rupireddy > wrote: > > On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote: > >> [2] > >> +# Ensure checkpoint doesn't come in our way > >> +$primary->append_conf('postgresql.conf', qq( > >> +

Re: Patch: Add parse_type Function

2024-02-19 Thread David E. Wheeler
On Feb 19, 2024, at 21:58, Erik Wienhold wrote: > See the patch I wrote for my benchmarks. But it's pretty easy anyway to > cut down parse_type() ;) LOL, I missed that, just wrote it myself in the last hour. :-) v6 attached. > But you don't actually need reformat_type() in pgTAP. You can

Re: Synchronizing slots from primary to standby

2024-02-19 Thread shveta malik
On Tue, Feb 20, 2024 at 8:25 AM Masahiko Sawada wrote: > > > I've reviewed the v91 patch. Here are random comments: Thanks for the comments. > --- > /* > * Checks the remote server info. > * > - * We ensure that the 'primary_slot_name' exists on the remote server and the > - * remote

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread jian he
On Mon, Feb 19, 2024 at 4:35 PM Andrei Lepikhov wrote: > > In attachment - v17 for both patches. As I see it, the only general > explanation of the idea is not addressed. I'm not sure how deeply we > should explain it. > On Tue, Nov 28, 2023 at 5:04 AM Robert Haas wrote: > > On Mon, Nov 27,

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 20:47, Tomas Vondra wrote: On 9/8/23 07:11, Lepikhov Andrei wrote: Just for comparison, without partitioning: elems 1 1E1 1E2 1E3 1E4 master: 12kB14kB37kB266kB 2.5MB patched:12kB11.5kB 13kB24kB

Re: JIT compilation per plan node

2024-02-19 Thread David Rowley
On Tue, 20 Feb 2024 at 18:31, Tom Lane wrote: > FWIW, I seriously doubt that an extra walk of the plan tree is even > measurable compared to the number of cycles JIT compilation will > expend if it's called. So I don't buy your argument here. > We would be better off to do this in a way that's

pg_restore option --clean

2024-02-19 Thread Fabrice Chapuis
Hi, The --clean option of pg_restore allows you to replace an object before being imported. However, dependencies such as foreign keys or views prevent the deletion of the object. Is there a way to add the cascade option to force the deletion? Thanks for helping Fabrice

Re: Patch: Add parse_type Function

2024-02-19 Thread Erik Wienhold
On 2024-02-19 23:59 +0100, David E. Wheeler wrote: > On Feb 19, 2024, at 15:47, Tom Lane wrote: > > >> 1. Add a to_regtypmod() for those who just want the typemod. > > > > Seems like there's a good case for doing that. > > I’ll work on that. See the patch I wrote for my benchmarks. But it's

Re: RFC: Logging plan of the running query

2024-02-19 Thread Robert Haas
On Fri, Feb 16, 2024 at 12:29 AM Andres Freund wrote: > If we went with something like tht approach, I think we'd have to do something > like redirecting node->ExecProcNode to a wrapper, presumably from within a > CFI. That wrapper could then implement the explain support, without slowing > down

Re: JIT compilation per plan node

2024-02-19 Thread Tom Lane
David Rowley writes: > On Tue, 20 Feb 2024 at 05:26, Tomas Vondra > wrote: >> Wouldn't it be simpler to just build the plan as we do now, and then >> have an expression_tree_walker that walks the complete plan top-down, >> inspects the nodes, enables JIT where appropriate and so on? That can >>

Re: confirmed flush lsn seems to be move backward in certain error cases

2024-02-19 Thread vignesh C
On Sat, 17 Feb 2024 at 12:03, Amit Kapila wrote: > > On Fri, Feb 16, 2024 at 5:53 PM vignesh C wrote: > > > > > > After the insert operation is replicated to the subscriber, the > > subscriber will set the lsn value sent by the publisher in the > > replication origin (in my case it was

Re: automating RangeTblEntry node support

2024-02-19 Thread Peter Eisentraut
On 18.02.24 00:06, Matthias van de Meent wrote: I'm not sure that the cleanup which is done when changing a RTE's rtekind is also complete enough for this purpose. Things like inline_cte_walker change the node->rtekind, which could leave residual junk data in fields that are currently dropped

Re: Synchronizing slots from primary to standby

2024-02-19 Thread Amit Kapila
On Tue, Feb 20, 2024 at 8:25 AM Masahiko Sawada wrote: > > Some comments not related to the patch but to the existing code: > > --- > It might have already been discussed but is the > src/backend/replication/logical the right place for the slocsync.c? If > it's independent of logical

Re: logical decoding and replication of sequences, take 2

2024-02-19 Thread Robert Haas
On Fri, Feb 16, 2024 at 1:57 AM Tomas Vondra wrote: > For me, the part that I feel most uneasy about is the decoding while the > snapshot is still being built (and can flip to consistent snapshot > between the relfilenode creation and sequence change, confusing the > logic that decides which

Re: Shared detoast Datum proposal

2024-02-19 Thread Andy Fan
Hi, I didn't another round of self-review. Comments, variable names, the order of function definition are improved so that it can be read as smooth as possible. so v6 attached. -- Best Regards Andy Fan >From f2e7772228e8a18027b9c29f10caba9c6570d934 Mon Sep 17 00:00:00 2001 From: "yizhi.fzh"

Re: Switching XLog source from archive to streaming when primary available

2024-02-19 Thread Japin Li
On Tue, 20 Feb 2024 at 13:40, Bharath Rupireddy wrote: > On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote: >> [2] >> +# Ensure checkpoint doesn't come in our way >> +$primary->append_conf('postgresql.conf', qq( >> +min_wal_size = 2MB >> +max_wal_size = 1GB >> +checkpoint_timeout = 1h

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-02-19 Thread Masahiko Sawada
On Mon, Feb 19, 2024 at 7:47 PM John Naylor wrote: > > On Mon, Feb 19, 2024 at 9:02 AM Masahiko Sawada wrote: > > > > I think that vacuum and tidbitmap (and future users) would end up > > having the same max block size calculation. And it seems slightly odd > > layering to me that

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:53, Ranier Vilela wrote: v17-0002 1) move the vars *arrayconst and *dest, to after if, to avoid makeNode (palloc). + Const   *arrayconst; + ScalarArrayOpExpr  *dest; + + pd = (PredicatesData *) lfirst(lc); + if (pd->elems == NIL) + /* The index doesn't participate in this

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-02-19 Thread Michael Paquier
On Tue, Feb 20, 2024 at 08:51:17AM +0900, Michael Paquier wrote: > Prefixing these with "initial_" is fine, IMO. That shows the > intention that these come from the slot's data before doing the > termination. So I'm OK with what's been proposed in v3. I was looking at that a second time, and

Re: Switching XLog source from archive to streaming when primary available

2024-02-19 Thread Bharath Rupireddy
On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote: > > > Strengthened tests a bit by using recovery_min_apply_delay to mimic > > standby spending some time fetching from archive. PSA v18 patch. > > Here are some minor comments: Thanks for taking a look at it. > [1] > +primary). However,

Re: Add test module for verifying backtrace functionality

2024-02-19 Thread Bharath Rupireddy
On Tue, Feb 13, 2024 at 2:11 AM Bharath Rupireddy wrote: > > Hi, > > Postgres has a good amount of code for dealing with backtraces - two > GUCs backtrace_functions and backtrace_on_internal_error, > errbacktrace; all of which use core function set_backtrace from > elog.c. I've not seen this code

Re: partitioning and identity column

2024-02-19 Thread Alexander Lakhin
20.02.2024 07:57, Ashutosh Bapat wrote: Could you please name functions, which you suspect, for me to recheck them? Perhaps we should consider fixing all of such functions, in light of b0f7dd915 and d57b7cc33... Looks like the second commit has fixed all other places I knew except Identity

Re: Synchronizing slots from primary to standby

2024-02-19 Thread Masahiko Sawada
On Mon, Feb 19, 2024 at 9:59 PM shveta malik wrote: > > On Mon, Feb 19, 2024 at 5:32 PM Amit Kapila wrote: > > > > Few comments on 0001 > > Thanks for the feedback. > > > > > 1. I think it is better to error out when the valid GUC or option is > > not set in

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
On 20/2/2024 11:03, jian he wrote: Neither the code comments nor the commit message really explain the design idea here. That's unfortunate, principally because it makes review difficult. I'm very skeptical about the idea of using JumbleExpr for any part of this. It seems fairly expensive, and

Re: partitioning and identity column

2024-02-19 Thread Ashutosh Bapat
On Mon, Feb 19, 2024 at 8:30 PM Alexander Lakhin wrote: > > Hello Ashutosh, > > 19.02.2024 15:17, Ashutosh Bapat wrote: > > > >> Functions ATExecAddIdentity() and ATExecDropIdentity() are recursive too, > >> so I think they can be exploited as well. > > not just Identity related functions, but

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-19 Thread Amit Kapila
On Tue, Feb 20, 2024 at 5:04 AM Ian Lawrence Barwick wrote: > > > With the addition of "pg_sync_replication_slots()", there is now a use-case > for > including "dbname" in "primary_conninfo" and the docs have changed from > stating [1]: > > Do not specify a database name in the

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Tue, Feb 20, 2024 at 8:19 AM Andrei Lepikhov wrote: > > On 19/2/2024 19:25, Ashutosh Bapat wrote: > > On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov > > wrote: > >> Live example: right now, I am working on the code like MSSQL has - a > >> combination of NestLoop and HashJoin paths and

Re: logical decoding and replication of sequences, take 2

2024-02-19 Thread Amit Kapila
On Thu, Dec 21, 2023 at 6:47 PM Tomas Vondra wrote: > > On 12/19/23 13:54, Christophe Pettus wrote: > > Hi, > > > > I wanted to hop in here on one particular issue: > > > >> On Dec 12, 2023, at 02:01, Tomas Vondra > >> wrote: > >> - desirability of the feature: Random IDs (UUIDs etc.) are

Re: Patch: Add parse_type Function

2024-02-19 Thread jian he
On Tue, Feb 20, 2024 at 11:06 AM David E. Wheeler wrote: > > LOL, I missed that, just wrote it myself in the last hour. :-) v6 attached. > +SELECT to_regtypemod('interval nonesuch'); -- grammar error expected +ERROR: syntax error at or near "nonesuch" +LINE 1: SELECT to_regtypemod('interval

Re: System username in pg_stat_activity

2024-02-19 Thread Bertrand Drouvot
Hi, On Fri, Feb 16, 2024 at 08:39:26PM +0100, Magnus Hagander wrote: > On Fri, Jan 19, 2024 at 1:43 PM Julien Rouhaud wrote: > > + value as the identity part in , or > > NULL > > I was looking at > > https://www.postgresql.org/docs/current/auth-username-maps.html and > > noticed that this

Re: speed up a logical replica setup

2024-02-19 Thread Peter Eisentraut
Some review of the v21 patch: - commit message Mention pg_createsubscriber in the commit message title. That's the most important thing that someone doing git log searches in the future will be looking for. - doc/src/sgml/ref/allfiles.sgml Move the new entry to alphabetical order. -

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-19 Thread Amit Kapila
On Thu, Feb 15, 2024 at 10:46 AM vignesh C wrote: > > On Thu, 15 Feb 2024 at 08:42, Amit Kapila wrote: > > > > > To resolve the BF > > failure, I still feel, we should just recreate the subscription. This > > is a pre-existing problem and we can track it via a separate patch > > with a test case

Re: Injection points: some tools to wait and wake

2024-02-19 Thread Andrey M. Borodin
> On 19 Feb 2024, at 09:01, Michael Paquier wrote: > > Thoughts and comments are welcome. Hi Michael, thanks for your work on injection points! I want to test a bunch of stuff using this facility. I have a wishlist of functionality that I'd like to see in injection points. I hope you

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-02-19 Thread Bertrand Drouvot
Hi, On Mon, Feb 19, 2024 at 01:45:16PM +0530, Bharath Rupireddy wrote: > On Mon, Feb 19, 2024 at 11:44 AM Michael Paquier wrote: > > > > > Yeah, comments added in v3. > > > > The contents look rather OK, I may do some word-smithing for both. > > Here are some comments on v3: Thanks for looing

Re: Transaction timeout

2024-02-19 Thread Japin Li
On Mon, 19 Feb 2024 at 17:14, Andrey M. Borodin wrote: >> On 18 Feb 2024, at 22:16, Andrey M. Borodin wrote: >> >> But it seems a little strange that session 3 did not fail at all > It was only coincidence. Any test that verifies FATALing out in 100ms can > fail, see new failure here [0]. >

Re: CI and test improvements

2024-02-19 Thread Peter Eisentraut
On 13.02.24 20:10, Justin Pryzby wrote: On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote: On 03.02.23 15:26, Justin Pryzby wrote: 9baf41674ad pg_upgrade: tap test: exercise --link and --clone This seems like a good idea. On Wed, Mar 15, 2023 at 10:58:41AM +0100, Peter

Avoid switching between system-user and system-username in the doc

2024-02-19 Thread Bertrand Drouvot
Hi hackers, While working on [1] it has been mentioned that this page https://www.postgresql.org/docs/current/auth-username-maps.html is switching between system-user and system-username. Please find attached a tiny patch to clean that up. [1]:

Re: Add last_commit_lsn to pg_stat_database

2024-02-19 Thread Tomas Vondra
On 2/19/24 07:57, Michael Paquier wrote: > On Sun, Feb 18, 2024 at 02:28:06AM +0100, Tomas Vondra wrote: >> Thanks for the updated patch. I don't have a clear opinion on the >> feature and whether this is the way to implement it, but I have two >> simple questions. > > Some users I know of would

Re: speed up a logical replica setup

2024-02-19 Thread Shlok Kyal
Hi, I have reviewed the v21 patch. And found an issue. Initially I started the standby server with a new postgresql.conf file (not the default postgresql.conf that is present in the instance). pg_ctl -D ../standby start -o "-c config_file=/new_path/postgresql.conf" And I have made

Re: Improvement discussion of custom and generic plans

2024-02-19 Thread Quan Zongliang
Add the GUC parameter. On 2024/1/30 21:25, Quan Zongliang wrote: On 2023/11/3 15:27, Quan Zongliang wrote: Hi We have one such problem. A table field has skewed data. Statistics: n_distinct | -0.4481973 most_common_vals   | {5f006ca25b52ed78e457b150ee95a30c}

Re: Transaction timeout

2024-02-19 Thread Andrey M. Borodin
> On 18 Feb 2024, at 22:16, Andrey M. Borodin wrote: > > But it seems a little strange that session 3 did not fail at all It was only coincidence. Any test that verifies FATALing out in 100ms can fail, see new failure here [0]. In a nearby thread Michael is proposing injections points that

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-02-19 Thread Bharath Rupireddy
On Mon, Feb 19, 2024 at 11:44 AM Michael Paquier wrote: > > > Yeah, comments added in v3. > > The contents look rather OK, I may do some word-smithing for both. Here are some comments on v3: 1. +XLogRecPtrinitial_effective_xmin = InvalidXLogRecPtr; +XLogRecPtr

Re: System username in pg_stat_activity

2024-02-19 Thread Bertrand Drouvot
Hi, On Fri, Feb 16, 2024 at 08:17:41PM +0100, Magnus Hagander wrote: > On Fri, Jan 19, 2024 at 7:20 AM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Thu, Jan 18, 2024 at 04:01:33PM +0100, Magnus Hagander wrote: > > > On Mon, Jan 15, 2024 at 11:17 AM Bertrand Drouvot > > > > Did you forget to

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
On 16/2/2024 19:54, jian he wrote: After setting these parameters, overall enable_or_transformation ON is performance better. sorry for the noise. Don't worry, at least we know a weak point of partial paths estimation. so now I didn't find any corner case where enable_or_transformation is ON

Re: What about Perl autodie?

2024-02-19 Thread Daniel Gustafsson
> On 19 Feb 2024, at 01:54, Andrew Dunstan wrote: > On 2024-02-14 We 11:52, Peter Eisentraut wrote: >> A gentler way might be to start using some perlcritic policies like >> InputOutput::RequireCheckedOpen or the more general >> InputOutput::RequireCheckedSyscalls and add explicit error

Re: System username in pg_stat_activity

2024-02-19 Thread Bertrand Drouvot
Hi, On Fri, Feb 16, 2024 at 09:41:41PM +0100, Magnus Hagander wrote: > On Fri, Feb 16, 2024 at 8:55 PM Andres Freund wrote: > > > > Hi, > > > > On 2024-01-12 17:16:53 +0100, Magnus Hagander wrote: > > > On Thu, Jan 11, 2024 at 5:55 PM Bertrand Drouvot > > > wrote: > > > > On Thu, Jan 11, 2024

numeric_big in make check?

2024-02-19 Thread Daniel Gustafsson
numeric_big has been left out of parallel_schedule, requiring EXTRA_TESTS to run it, since going in back in 1999 (AFAICT it was even the reason EXTRA_TESTS was invented). The original commit states that it's huge, and it probably was. Today it runs faster than many tests we have in

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-02-19 Thread John Naylor
On Mon, Feb 19, 2024 at 9:02 AM Masahiko Sawada wrote: > > I think that vacuum and tidbitmap (and future users) would end up > having the same max block size calculation. And it seems slightly odd > layering to me that max-block-size-specified context is created on > vacuum (or tidbitmap) layer,

Re: partitioning and identity column

2024-02-19 Thread Ashutosh Bapat
On Thu, Feb 15, 2024 at 11:30 PM Alexander Lakhin wrote: > > Hello Ashutosh, > > 24.01.2024 09:34, Ashutosh Bapat wrote: > > > >>> There's another thing I found. The file isn't using > >>> check_stack_depth() in the function which traverse inheritance > >>> hierarchies. This isn't just a problem

Re: table inheritance versus column compression and storage settings

2024-02-19 Thread Ashutosh Bapat
On Fri, Feb 16, 2024 at 11:54 PM Tom Lane wrote: > > I wrote: > > I find it surprising that the committed patch does not touch > > pg_dump. Is it really true that pg_dump dumps situations with > > differing compression/storage settings accurately already? > > It's worse than I thought. Run

Re: Synchronizing slots from primary to standby

2024-02-19 Thread Amit Kapila
On Mon, Feb 19, 2024 at 9:46 AM shveta malik wrote: > > Okay I see. Thanks for pointing it out. Here are the patches > addressing your comments. Changes are in patch001, rest are rebased. > Few comments on 0001 1. I think it is better to error out when the valid GUC or

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: > Live example: right now, I am working on the code like MSSQL has - a > combination of NestLoop and HashJoin paths and switching between them in > real-time. It requires both paths in the path list at the moment when > extensions are coming.

Re: Switching XLog source from archive to streaming when primary available

2024-02-19 Thread Bharath Rupireddy
On Wed, Jan 31, 2024 at 6:30 PM Bharath Rupireddy wrote: > > Needed a rebase due to commit 776621a (conflict in > src/test/recovery/meson.build for new TAP test file added). Please > find the attached v17 patch. Strengthened tests a bit by using recovery_min_apply_delay to mimic standby spending

Re: Add an option to skip loading missing publication to avoid logical replication failure

2024-02-19 Thread vignesh C
On Mon, 19 Feb 2024 at 12:48, vignesh C wrote: > > Hi, > > Currently ALTER SUBSCRIPTION ... SET PUBLICATION will break the > logical replication in certain cases. This can happen as the apply > worker will get restarted after SET PUBLICATION, the apply worker will > use the existing slot and

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Ranier Vilela
Em seg., 19 de fev. de 2024 às 05:35, Andrei Lepikhov < a.lepik...@postgrespro.ru> escreveu: > On 16/2/2024 19:54, jian he wrote: > > After setting these parameters, overall enable_or_transformation ON is > > performance better. > > sorry for the noise. > Don't worry, at least we know a weak

Re: serial not accepted as datatype in ALTER TABLE ... ALTER COLUMN

2024-02-19 Thread Ashutosh Bapat
On Sun, Feb 18, 2024 at 1:59 PM Andy Fan wrote: > > > Hi Ashutosh, > > > data_type is described on that page as "Data type of the new column, > > or new data type for an existing column." but CREATE TABLE > > documentation [2] redirects data_type to [3], which mentions serial. > > The impression

Re: Experiments with Postgres and SSL

2024-02-19 Thread Matthias van de Meent
I've been asked to take a look at this thread and review some patches, and the subject looks interesting enough, so here I am. On Thu, 19 Jan 2023 at 04:16, Greg Stark wrote: > I had a conversation a while back with Heikki where he expressed that > it was annoying that we negotiate SSL/TLS the

Re: numeric_big in make check?

2024-02-19 Thread Tom Lane
Daniel Gustafsson writes: > numeric_big has been left out of parallel_schedule, requiring EXTRA_TESTS to > run it, since going in back in 1999 (AFAICT it was even the reason EXTRA_TESTS > was invented). The original commit states that it's huge, and it probably > was. > Today it runs faster

Re: Synchronizing slots from primary to standby

2024-02-19 Thread shveta malik
On Mon, Feb 19, 2024 at 5:32 PM Amit Kapila wrote: > > Few comments on 0001 Thanks for the feedback. > > 1. I think it is better to error out when the valid GUC or option is > not set in ensure_valid_slotsync_params() and > ensure_valid_remote_info() instead of waiting. And

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Sun, Feb 18, 2024 at 10:55 PM Tomas Vondra wrote: > > Hi, > > I took a quick look at this patch today. I certainly agree with the > intent to reduce the amount of memory during planning, assuming it's not > overly disruptive. And I think this patch is fairly localized and looks > sensible.

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-02-19 Thread Ashutosh Bapat
On Mon, Feb 19, 2024 at 4:35 AM Tomas Vondra wrote: > > Hi, > > After taking a look at the patch optimizing SpecialJoinInfo allocations, > I decided to take a quick look at this one too. I don't have any > specific comments on the code yet, but it seems quite a bit more complex > than the other

Re: numeric_big in make check?

2024-02-19 Thread Dean Rasheed
> > On 19 Feb 2024, at 12:48, Tom Lane wrote: > > > > Or we could just flush it. It's never detected a bug, and I think > > you'd find that it adds zero code coverage (or if not, we could > > fix that in a far more surgical and less expensive manner). > Off the top of my head, I can't say to

Use streaming read API in ANALYZE

2024-02-19 Thread Nazir Bilal Yavuz
Hi, I worked on using the currently proposed streaming read API [1] in ANALYZE. The patch is attached. 0001 is the not yet merged streaming read API code changes that can be applied to the master, 0002 is the actual code. The blocks to analyze are obtained by using the streaming read API now. -

Re: Add trim_trailing_whitespace to editorconfig file

2024-02-19 Thread Jelte Fennema-Nio
On Fri, 16 Feb 2024 at 11:45, Peter Eisentraut wrote: > I have committed that one. Thanks :) > v3-0002-Require-final-newline-in-.po-files.patch > > The .po files are imported from elsewhere, so I'm not sure this is going > to have the desired effect. Perhaps it's worth cleaning up, but it >

Re: Speeding up COPY TO for uuids and arrays

2024-02-19 Thread Laurenz Albe
On Sat, 2024-02-17 at 12:24 -0800, Andres Freund wrote: > On 2024-02-17 17:48:23 +0100, Laurenz Albe wrote: > > - Patch 0001 speeds up pq_begintypsend with a custom macro. > > That brought the binary copy down to 3.7 seconds, which is a > > speed gain of 15%. > > Nice win, but I think we can

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Joe Conway
On 2/18/24 15:35, Andres Freund wrote: On 2024-02-18 17:06:09 +0530, Robert Haas wrote: How many people set shared_buffers to something that's not a whole number of GB these days? I'd say the vast majority of postgres instances in production run with less than 1GB of s_b. Just because numbers

Re: Injection points: some tools to wait and wake

2024-02-19 Thread Bertrand Drouvot
Hi, On Mon, Feb 19, 2024 at 04:51:45PM +0900, Michael Paquier wrote: > On Mon, Feb 19, 2024 at 03:01:40PM +0900, Michael Paquier wrote: > > 0002 is a polished version of the TAP test that makes use of this > > facility, providing coverage for the bug fixed by 7863ee4def65 > > (reverting this

Re: Switching XLog source from archive to streaming when primary available

2024-02-19 Thread Japin Li
On Mon, 19 Feb 2024 at 18:36, Bharath Rupireddy wrote: > On Wed, Jan 31, 2024 at 6:30 PM Bharath Rupireddy > wrote: >> >> Needed a rebase due to commit 776621a (conflict in >> src/test/recovery/meson.build for new TAP test file added). Please >> find the attached v17 patch. > > Strengthened

  1   2   >