Re: A performance issue with Memoize

2024-01-31 Thread Anthonin Bonnefoy
Hi, I've seen a similar issue with the following query (tested on the current head): EXPLAIN ANALYZE SELECT * FROM tenk1 t1 LEFT JOIN LATERAL (SELECT t1.two, tenk2.hundred, tenk2.two FROM tenk2) t2 ON t1.hundred = t2.hundred WHERE t1.hundred < 5;

Re: Add system identifier to backup manifest

2024-01-31 Thread Amul Sul
On Thu, Feb 1, 2024 at 3:06 AM Robert Haas wrote: > On Thu, Jan 25, 2024 at 2:52 AM Amul Sul wrote: > > Thank you for the review-comments, updated version attached. > > I generally agree with 0001. I spent a long time thinking about your > decision to make verifier_context contain a pointer to

Re: [PATCH] LockAcquireExtended improvement

2024-01-31 Thread Jingxian Li
Hello Robert, Thank you for your advice. It is very helpful to me. On 2024/1/16 3:07, Robert Haas wrote: > Hello Jingxian Li! > > I agree with you that this behavior seems surprising. I don't think > it's quite a bug, more of a limitation. However, I think it would be > nice to fix it if we can

Re: set_cheapest without checking pathlist

2024-01-31 Thread Richard Guo
On Thu, Feb 1, 2024 at 11:37 AM David Rowley wrote: > On Thu, 1 Feb 2024 at 16:29, Richard Guo wrote: > > On Thu, Feb 1, 2024 at 10:04 AM James Coleman wrote: > >> I don't see any inherent reason why we must always assume that > >> gather_grouping_paths will always result in having at least

Re: Small fix on COPY ON_ERROR document

2024-01-31 Thread Yugo NAGATA
On Mon, 29 Jan 2024 15:47:25 +0900 Yugo NAGATA wrote: > On Sun, 28 Jan 2024 19:14:58 -0700 > "David G. Johnston" wrote: > > > > Also, I think "invalid input syntax" is a bit ambiguous. For example, > > > COPY FROM raises an error when the number of input column does not match > > > to the

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
Sorry for a minor correction, but.. At Thu, 01 Feb 2024 14:53:57 +0900 (JST), Kyotaro Horiguchi wrote in > Ah.. Understood. "NaN or Infinity" cannot be used in those > cases. Additionally, for jpiBoolean and jpiBigint, we lack the text > representation of the value. This "Additionally" was

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
At Thu, 1 Feb 2024 09:22:22 +0530, Jeevan Chalke wrote in > On Thu, Feb 1, 2024 at 7:24 AM Kyotaro Horiguchi > wrote: > > > At Thu, 01 Feb 2024 10:49:57 +0900 (JST), Kyotaro Horiguchi < > > horikyota@gmail.com> wrote in > > > By the way, while playing with this feature, I noticed the

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
At Thu, 1 Feb 2024 09:19:40 +0530, Jeevan Chalke wrote in > > As Tom suggested, given that similar situations have already been > > disregarded elsewhere, worrying about excessively long input strings > > in this specific instance won't notably improve safety in total. > > > > > Also, for

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Peter Smith
Here are some review comments for v740001. == src/sgml/logicaldecoding.sgml 1. + +Replication Slot Synchronization + + A logical replication slot on the primary can be synchronized to the hot + standby by enabling the failover option during slot + creation and setting

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Ajin Cherian
On Tue, Jan 30, 2024 at 11:53 PM shveta malik wrote: > On Tue, Jan 30, 2024 at 4:06 PM shveta malik > wrote: > > > > PFA v73-0001 which addresses the above comments. Other patches will be > > rebased and posted after pushing this one. > > Since v73-0001 is pushed, PFA rest of the patches.

Re: POC, WIP: OR-clause support for indexes

2024-01-31 Thread jian he
On Wed, Jan 31, 2024 at 7:10 PM Alena Rybakina wrote: > > Hi, thank you for your review and interest in this subject. > > On 31.01.2024 13:15, jian he wrote: > > On Wed, Jan 31, 2024 at 10:55 AM jian he wrote: > > based on my understanding of >

Re: Race condition in FetchTableStates() breaks synchronization of subscription tables

2024-01-31 Thread vignesh C
On Tue, 30 Jan 2024 at 17:22, Zhijie Hou (Fujitsu) wrote: > > On Tuesday, January 30, 2024 11:21 AM vignesh C wrote: > > > > On Tue, 30 Jan 2024 at 07:24, Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Monday, January 29, 2024 9:22 PM vignesh C > > wrote: > > > > > > > > On Fri, 26 Jan 2024 at

Re: pg_column_toast_chunk_id: a function to get a chunk ID of a TOASTed value

2024-01-31 Thread Yugo NAGATA
On Tue, 30 Jan 2024 14:57:20 +0800 jian he wrote: > On Tue, Jan 30, 2024 at 1:56 PM Yugo NAGATA wrote: > > > > I attached the correct one, v4. > > > > +-- Test pg_column_toast_chunk_id: > +-- Check if the returned chunk_id exists in the TOAST table > +CREATE TABLE test_chunk_id (v1 text, v2

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Amit Kapila
On Thu, Feb 1, 2024 at 8:15 AM Euler Taveira wrote: > > On Mon, Jan 29, 2024, at 10:17 AM, Zhijie Hou (Fujitsu) wrote: > > Attach the V72-0001 which addressed above comments, other patches will be > rebased and posted after pushing first patch. Thanks Shveta for helping > address > the comments.

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Junwang Zhao
On Thu, Feb 1, 2024 at 11:56 AM Michael Paquier wrote: > > On Thu, Feb 01, 2024 at 11:43:07AM +0800, Junwang Zhao wrote: > > The first 6 rounds are like 10% better than the later 9 rounds, is this > > normal? > > Even with HEAD? Perhaps you have some OS cache eviction in play here? > FWIW, I'm

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Michael Paquier
On Thu, Feb 01, 2024 at 11:43:07AM +0800, Junwang Zhao wrote: > The first 6 rounds are like 10% better than the later 9 rounds, is this > normal? Even with HEAD? Perhaps you have some OS cache eviction in play here? FWIW, I'm not seeing any of that with longer runs after 7~ tries in a loop of

Re: More new SQL/JSON item methods

2024-01-31 Thread Jeevan Chalke
On Thu, Feb 1, 2024 at 7:24 AM Kyotaro Horiguchi wrote: > At Thu, 01 Feb 2024 10:49:57 +0900 (JST), Kyotaro Horiguchi < > horikyota@gmail.com> wrote in > > By the way, while playing with this feature, I noticed the following > > error message: > > > > > select jsonb_path_query('1.1' ,

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Amit Kapila
On Wed, Jan 31, 2024 at 9:20 PM Masahiko Sawada wrote: > > On Wed, Jan 31, 2024 at 7:42 PM Amit Kapila wrote: > > > > > > Considering my previous where we don't want to restart for a required > > parameter change, isn't it better to avoid repeated restart (say when > > the user gave an invalid

Re: More new SQL/JSON item methods

2024-01-31 Thread Jeevan Chalke
On Thu, Feb 1, 2024 at 7:20 AM Kyotaro Horiguchi wrote: > Thank you for the fix! > > At Tue, 30 Jan 2024 13:46:17 +0530, Jeevan Chalke < > jeevan.cha...@enterprisedb.com> wrote in > > On Mon, Jan 29, 2024 at 11:03 AM Tom Lane wrote: > > > > > Kyotaro Horiguchi writes: > > > > Having said that,

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Michael Paquier
On Thu, Feb 01, 2024 at 10:57:58AM +0900, Michael Paquier wrote: > And here are the results I get for text and binary (ms, average of 15 > queries after discarding the three highest and three lowest values): > test | master | v7 | v10 > -++--+-- >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Junwang Zhao
Hi Michael, On Thu, Feb 1, 2024 at 9:58 AM Michael Paquier wrote: > > On Wed, Jan 31, 2024 at 02:39:54PM +0900, Michael Paquier wrote: > > Thanks, I'm looking into that now. > > I have much to say about the patch, but for now I have begun running > some performance tests using the patches,

Re: set_cheapest without checking pathlist

2024-01-31 Thread David Rowley
On Thu, 1 Feb 2024 at 16:29, Richard Guo wrote: > > > On Thu, Feb 1, 2024 at 10:04 AM James Coleman wrote: >> >> I don't see any inherent reason why we must always assume that >> gather_grouping_paths will always result in having at least one entry >> in pathlist. If, for example, we've disabled

Re: set_cheapest without checking pathlist

2024-01-31 Thread Richard Guo
On Thu, Feb 1, 2024 at 10:04 AM James Coleman wrote: > I don't see any inherent reason why we must always assume that > gather_grouping_paths will always result in having at least one entry > in pathlist. If, for example, we've disabled incremental sort and the > cheapest partial path happens to

RE: speed up a logical replica setup

2024-01-31 Thread Hayato Kuroda (Fujitsu)
Dear Fabrízio, Euler, I think you set the primary_slot_name to the standby server, right? While reading codes, I found below line in v11-0001. ``` if (primary_slot_name != NULL) { conn = connect_database(dbinfo[0].pubconninfo); if (conn != NULL)

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Andres Freund
Hi, On 2024-01-31 14:57:35 -0500, Robert Haas wrote: > > You're right and I'm open to doing more legwork. I'd also appreciate any > > suggestion about how to test this properly and/or useful scenarios to > > test. That would be really helpful. > > I think experimenting to see whether the

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:09 PM, Hayato Kuroda (Fujitsu) wrote: > > > Why? Are you suggesting that the dry run mode covers just the verification > part? If so, it is not a dry run mode. I would expect it to run until the end > (or until it accomplish its goal) but *does not* modify data. For

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-31 Thread Richard Guo
On Wed, Jan 31, 2024 at 11:21 PM Tom Lane wrote: > Richard Guo writes: > > On Wed, Jan 31, 2024 at 5:12 AM Tom Lane wrote: > >> * Why is it okay to just use pull_varnos on a tablesample expression, > >> when contain_references_to() does something different? > > > Hmm, the main reason is that

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Euler Taveira
On Mon, Jan 29, 2024, at 10:17 AM, Zhijie Hou (Fujitsu) wrote: > Attach the V72-0001 which addressed above comments, other patches will be > rebased and posted after pushing first patch. Thanks Shveta for helping > address > the comments. While working on another patch I noticed a new NOTICE

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread James Coleman
On Wed, Jan 31, 2024 at 3:18 PM Robert Haas wrote: > > On Tue, Jan 30, 2024 at 9:56 PM James Coleman wrote: > > I don't follow the "Idle since July" since it just hasn't received > > review since then, so there's been nothing to reply to. > > It wasn't clear to me if you thought that the patch

RE: speed up a logical replica setup

2024-01-31 Thread Hayato Kuroda (Fujitsu)
Dear Euler, Thanks for giving comments! I want to reply some of them. > I didn't provide the whole explanation. I'm envisioning the use case that pg_ctl doesn't reach the consistent state and the timeout is reached (the consequence is that pg_createsubscriber aborts the execution). It might

set_cheapest without checking pathlist

2024-01-31 Thread James Coleman
Hello, Robert: I've taken the liberty of cc'ing you since you worked on most of this code. My apologies if that wasn't appropriate. While working on "Parallelize correlated subqueries that execute within each worker" [1] I noticed that while in the other call to set_cheapest (for

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Michael Paquier
On Wed, Jan 31, 2024 at 02:39:54PM +0900, Michael Paquier wrote: > Thanks, I'm looking into that now. I have much to say about the patch, but for now I have begun running some performance tests using the patches, because this thread won't get far until we are sure that the callbacks do not impact

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
At Thu, 01 Feb 2024 10:49:57 +0900 (JST), Kyotaro Horiguchi wrote in > By the way, while playing with this feature, I noticed the following > error message: > > > select jsonb_path_query('1.1' , '$.boolean()'); > > ERROR: numeric argument of jsonpath item method .boolean() is out of range >

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
Thank you for the fix! At Tue, 30 Jan 2024 13:46:17 +0530, Jeevan Chalke wrote in > On Mon, Jan 29, 2024 at 11:03 AM Tom Lane wrote: > > > Kyotaro Horiguchi writes: > > > Having said that, I'm a bit concerned about the case where an overly > > > long string is given there. However,

RE: speed up a logical replica setup

2024-01-31 Thread Hayato Kuroda (Fujitsu)
Dear Fabrízio, Thanks for reporting. I understood that the issue occurred on v11 and v12. I will try to reproduce and check the reason. Best Regards, Hayato Kuroda FUJITSU LIMITED https://www.fujitsu.com/global/

Re: Rename setup_cancel_handler in pg_dump

2024-01-31 Thread Yugo NAGATA
On Tue, 30 Jan 2024 13:44:28 +0100 Daniel Gustafsson wrote: > > On 26 Jan 2024, at 01:42, Yugo NAGATA wrote: > > > I am proposing it because there is a public function with > > the same name in fe_utils/cancel.c. I know pg_dump/parallel.c > > does not include fe_utils/cancel.h, so there is no

Re: src/bin/pg_upgrade/t/004_subscription.pl test comment fix

2024-01-31 Thread Peter Smith
On Wed, Jan 31, 2024 at 7:48 PM Alvaro Herrera wrote: > > How about rewording it more extensively? It doesn't read great IMO. > I would use something like > > # In the upgraded instance, the running status and failover option of the > # subscription with the failover option should have been

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Tue, Jan 30, 2024, at 6:26 AM, Hayato Kuroda (Fujitsu) wrote: > > One open item that is worrying me is how to handle the pg_ctl timeout. This > > patch does nothing and the user should use PGCTLTIMEOUT environment > > variable to > > avoid that the execution is canceled after 60 seconds

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-29, Dilip Kumar wrote: > Thank you for working on this. There is one thing that I feel is > problematic. We have kept the allowed values for these GUCs to be in > multiple of SLRU_BANK_SIZE i.e. 16 and that's the reason the min > values were changed to 16 but in this refactoring

Re: Documentation: warn about two_phase when altering a subscription

2024-01-31 Thread Peter Smith
On Wed, Jan 31, 2024 at 4:55 PM Bertrand Drouvot wrote: ... > As a non native English speaker somehow I have to rely on you for those > suggestions ;-) > > They make sense to me so applied both in v2 attached. > Patch v2 looks OK to me, but probably there is still room for improvement. Let's see

Re: libpq fails to build with TSAN

2024-01-31 Thread Daniel Gustafsson
> On 31 Jan 2024, at 16:39, Tom Lane wrote: > > Daniel Gustafsson writes: >> I think it should, the idea of that check is to catch calls to actual exits, >> while this is instrumentation which has nothing to do with exit(2). The >> attached diff should be enough to handle this. > > +1

Re: Improve the connection failure error messages

2024-01-31 Thread Peter Smith
On Wed, Jan 31, 2024 at 9:58 PM Nisha Moond wrote: > > >> AFAIK some recent commits patches (e,g [1] for the "slot sync" >> development) have created some more cases of "could not connect..." >> messages. So, you might need to enhance your patch to deal with any >> new ones in the latest HEAD.

Re: Add system identifier to backup manifest

2024-01-31 Thread Robert Haas
On Thu, Jan 25, 2024 at 2:52 AM Amul Sul wrote: > Thank you for the review-comments, updated version attached. I generally agree with 0001. I spent a long time thinking about your decision to make verifier_context contain a pointer to manifest_data instead of, as it does currently, a pointer to

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alexander Kuzmenkov
On Wed, Jan 31, 2024 at 9:49 PM David Rowley wrote: > Pushed to master. > > Thanks for the report and the fix, Alexander. Thank you!

Re: Incorrect cost for MergeAppend

2024-01-31 Thread David Rowley
On Thu, 1 Feb 2024 at 04:32, Tom Lane wrote: > > Alvaro Herrera writes: > > Since we have a minor coming up very soon, I think it's not a good idea > > to backpatch right now. Maybe you can push to master now, and consider > > whether to backpatch later. > > As a rule, we don't back-patch

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread Robert Haas
On Tue, Jan 30, 2024 at 9:56 PM James Coleman wrote: > I don't follow the "Idle since July" since it just hasn't received > review since then, so there's been nothing to reply to. It wasn't clear to me if you thought that the patch was ready for review since July, or if it was waiting on you

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 2:23 PM Melih Mutlu wrote: >> That seems like it might be a useful refinement of Melih Mutlu's >> original proposal, but consider a message stream that consists of >> messages exactly 8kB in size. If that message stream begins when the >> buffer is empty, all messages are

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Melih Mutlu
Robert Haas , 31 Oca 2024 Çar, 20:23 tarihinde şunu yazdı: > On Tue, Jan 30, 2024 at 6:39 PM Jelte Fennema-Nio > wrote: > > I agree that it's hard to prove that such heuristics will always be > > better in practice than the status quo. But I feel like we shouldn't > > let perfect be the enemy of

Re: psql not responding to SIGINT upon db reconnection

2024-01-31 Thread Jelte Fennema-Nio
On Wed, 31 Jan 2024 at 19:07, Tristan Partin wrote: > I was looking for documentation of PQsocket(), but didn't find any > standalone (unless I completely missed it). So I just copied how > PQsocket() is documented in PQconnectPoll(). I am happy to document it > separately if you think it would

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 12:49 PM Jelte Fennema-Nio wrote: > Testing a bunch of scenarios to find a good one sounds like a good > idea, which can probably give us a more optimal heuristic. But it also > sounds like a lot of work, and probably results in a lot of > discussion. That extra effort

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-31 Thread Tomas Vondra
On 1/29/24 16:42, Ronan Dunklau wrote: > Le lundi 29 janvier 2024, 15:59:04 CET Tomas Vondra a écrit : >> I'm not sure work_mem is a good parameter to drive this. It doesn't say >> how much memory we expect the backend to use - it's a per-operation >> limit, so it doesn't work particularly well

Re: Fix some ubsan/asan related issues

2024-01-31 Thread Tristan Partin
On Tue Jan 30, 2024 at 10:00 PM CST, Alexander Lakhin wrote: Hello, 30.01.2024 18:57, Tristan Partin wrote: > Patch 1: > > Passing NULL as a second argument to memcpy breaks ubsan, ... Maybe you would like to fix also one more similar place, reached with: create extension xml2; select

Re: make dist using git archive

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 10:50 AM Eli Schwartz wrote: > Ideally git commits should be signed, but that requires large numbers of > people to have security-minded git commit habits. From a quick check of > the postgres commit logs, only one person seems to be regularly signing > commits, which does

Re: psql not responding to SIGINT upon db reconnection

2024-01-31 Thread Tristan Partin
On Tue Jan 30, 2024 at 4:42 PM CST, Jelte Fennema-Nio wrote: On Tue, 30 Jan 2024 at 23:20, Tristan Partin wrote: > Not next week, but here is a respin. I've exposed pqSocketPoll as > PQsocketPoll and am just using that. You can see the diff is so much > smaller, which is great! The exports.txt

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Jelte Fennema-Nio
On Wed, 31 Jan 2024 at 18:23, Robert Haas wrote: > That's kind of an odd artifact, but maybe it's fine in > practice. I agree it's an odd artifact, but it's not a regression over the status quo. Achieving that was the intent of my suggestion: A change that improves some cases, but regresses

Re: Reducing output size of nodeToString

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 11:17 AM Matthias van de Meent wrote: > I was also thinking about smaller per-attribute expression storage, for index > attribute expressions, table default expressions, and functions. Other than > that, less memory overhead for the serialized form of these constructs

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Wed, Jan 31, 2024 at 12:38 PM Euler Taveira wrote: > > > Hmm. I didn't try it with the failover patch that was recently applied. Did you > have any special configuration on primary? > Nothing special, here the configurations I've changed after bootstrap: port = '5432' wal_level = 'logical'

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Robert Haas
On Tue, Jan 30, 2024 at 6:39 PM Jelte Fennema-Nio wrote: > I agree that it's hard to prove that such heuristics will always be > better in practice than the status quo. But I feel like we shouldn't > let perfect be the enemy of good here. Sure, I agree. > I one approach that is a clear >

Abort if dup fail (src/bin/pg_dump/compress_none.c)

2024-01-31 Thread Ranier Vilela
Hi. Per Coverity. CID 1506240: (#1 of 1): Argument cannot be negative (NEGATIVE_RETURNS) 3. negative_returns: dup(fd) is passed to a parameter that cannot be negative. pg_dump function open_none, tries to associate a stream to a file descriptor, using function dup, which may fail and return

Re: pgsql: Clean pg_walsummary's tmp_check directory.

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 11:51 AM Tom Lane wrote: > Clean pg_walsummary's tmp_check directory. Ugh. The reason I keep doing this is because I've switched to using meson builds, where of course you don't get complaints about this. And it seems like CI doesn't tell you either. Nor does the

Re: Reducing output size of nodeToString

2024-01-31 Thread Matthias van de Meent
On Wed, 31 Jan 2024, 09:16 Peter Eisentraut, wrote: > On 30.01.24 12:26, Matthias van de Meent wrote: > >> Most of the other defaults I'm doubtful about. First, we are colliding > >> here between the goals of minimizing the storage size and making the > >> debug output more readable. > > I've

Re: Possibility to disable `ALTER SYSTEM`

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 5:16 AM Gabriele Bartolini wrote: > I very much like the idea of a file in the data directory that also controls > the copy operations. > > Just wanted to highlight though that in our operator we have already applied > the read-only postgresql.auto.conf trick to disable

Re: make dist using git archive

2024-01-31 Thread Eli Schwartz
On 1/31/24 3:03 AM, Peter Eisentraut wrote: >> What do you use this for? IMO a more robust way to track the commit used >> is to use gitattributes export-subst to write a `.git_archival.txt` file >> containing the commit sha1 and other info -- this can be read even after >> the file is extracted,

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Masahiko Sawada
On Wed, Jan 31, 2024 at 7:42 PM Amit Kapila wrote: > > On Wed, Jan 31, 2024 at 2:02 PM Masahiko Sawada wrote: > > > > Thank you for updating the patches. As for the slotsync worker patch, > > is there any reason why 0001, 0002, and 0004 patches are still > > separated? > > > > No specific

Re: [PATCH] Add native windows on arm64 support

2024-01-31 Thread Andrew Dunstan
On 2024-01-31 We 10:34, Peter Eisentraut wrote: On 31.01.24 16:20, Andrew Dunstan wrote: - PostgreSQL will only build for the x64 architecture on 64-bit Windows. + PostgreSQL will only build for the x64 and ARM64 architecture on 64-bit Windows. Are there any other 64-bit architectures for

Re: libpq fails to build with TSAN

2024-01-31 Thread Tom Lane
Daniel Gustafsson writes: > I think it should, the idea of that check is to catch calls to actual exits, > while this is instrumentation which has nothing to do with exit(2). The > attached diff should be enough to handle this. +1 regards, tom lane

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:55 AM, Fabrízio de Royes Mello wrote: > > On Wed, Jan 31, 2024 at 11:35 AM Euler Taveira wrote: > > > > On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > > > > Jumping into this a bit late here... I'm trying a simple > > pg_createsubscriber but

Re: [PATCH] Add native windows on arm64 support

2024-01-31 Thread Peter Eisentraut
On 31.01.24 16:20, Andrew Dunstan wrote: - PostgreSQL will only build for the x64 architecture on 64-bit Windows. + PostgreSQL will only build for the x64 and ARM64 architecture on 64-bit Windows. Are there any other 64-bit architectures for Windows? Possibly, the original sentence was meant

Re: cleanup patches for incremental backup

2024-01-31 Thread Robert Haas
On Tue, Jan 30, 2024 at 11:52 AM Robert Haas wrote: > Here's a patch for that. I now think > a7097ca630a25dd2248229f21ebce4968d85d10a was actually misguided, and > served only to mask some of the failures caused by waiting for the WAL > summary file. Committed. -- Robert Haas EDB:

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Tom Lane
Alvaro Herrera writes: > Since we have a minor coming up very soon, I think it's not a good idea > to backpatch right now. Maybe you can push to master now, and consider > whether to backpatch later. As a rule, we don't back-patch changes like this ever. People don't appreciate plans changing

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-31 Thread Tom Lane
Richard Guo writes: > On Wed, Jan 31, 2024 at 5:12 AM Tom Lane wrote: >> * Why is it okay to just use pull_varnos on a tablesample expression, >> when contain_references_to() does something different? > Hmm, the main reason is that the expression_tree_walker() function does > not handle

Re: [PATCH] Add native windows on arm64 support

2024-01-31 Thread Andrew Dunstan
On 2024-01-30 Tu 17:54, Dave Cramer wrote: On Tue, Jan 30, 2024 at 4:56 PM Andrew Dunstan wrote: On 2024-01-30 Tu 09:50, Dave Cramer wrote: On Tue, 30 Jan 2024 at 08:38, Andrew Dunstan wrote: On 2024-01-29 Mo 11:20, Dave Cramer wrote: Dave Cramer

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Wed, Jan 31, 2024 at 11:35 AM Euler Taveira wrote: > > On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > > Jumping into this a bit late here... I'm trying a simple pg_createsubscriber but getting an error: > > > Try v11. It seems v12-0002 is not correct. Using v11 I'm

Re: remaining sql/json patches

2024-01-31 Thread jian he
Hi. minor issues. I am wondering do we need add `pg_node_attr(query_jumble_ignore)` to some of our created structs in src/include/nodes/parsenodes.h in v39-0001-Add-SQL-JSON-query-functions.patch diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c new file

Re: Possibility to disable `ALTER SYSTEM`

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 12:28 AM Tom Lane wrote: > You cannot enforce such restrictions within Postgres. > It has to be done by an outside mechanism. If you think > different, you are mistaken. It seems like the biggest reason why we can't enforce such restrictions with Postgres is that you

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > Jumping into this a bit late here... I'm trying a simple pg_createsubscriber > but getting an error: Try v11. It seems v12-0002 is not correct. > Seems we need to escape connection params similar we do in dblink [1] I think it

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Wed, Jan 31, 2024 at 9:52 AM Hayato Kuroda (Fujitsu) < kuroda.hay...@fujitsu.com> wrote: > > Dear Euler, > > I extracted some review comments which may require many efforts. I hope this makes them > easy to review. > > 0001: not changed from yours. > 0002: avoid to use replication connections.

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Daniel Westermann (DWE)
Hi, >Since we have a minor coming up very soon, I think it's not a good idea >to backpatch right now. Maybe you can push to master now, and consider >whether to backpatch later. >The problem is -- if somebody has an application that gets good plans >with the current cost model, and you change

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread James Coleman
On Tue, Jan 30, 2024 at 10:34 PM Tom Lane wrote: > > James Coleman writes: > > I've finally had a chance to look at this, and I don't believe there's > > any real failure here, merely drift of how the planner works on master > > resulting in this query now being eligible for a different plan

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-31, Alexander Kuzmenkov wrote: > To put it another way, this change enables our usual cost model for > MergeAppend to work correctly in the presence of filters. I think we > can consider this model to be reasonably correct, and we don't > currently have major problems with MergeAppend

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alexander Kuzmenkov
On Wed, Jan 31, 2024 at 1:33 PM Alexander Kuzmenkov wrote: > I'd be happy to see this backpatched. What kind of regressions are we > worried about? I'd say partition-wise sort + merge should be faster > than append + sort for reasonably sized tables. That's basically what > tuplesort does inside.

Re: Emitting JSON to file using COPY TO

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-23, jian he wrote: > > + | FORMAT_LA copy_generic_opt_arg > > + { > > + $$ = makeDefElem("format", $2, @1); > > + } > > ; > > > > I think it's not necessary. "format" option is already handled in > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-01-31 Thread Bharath Rupireddy
On Sat, Jan 27, 2024 at 1:18 AM Bharath Rupireddy wrote: > > On Thu, Jan 11, 2024 at 10:48 AM Bharath Rupireddy > wrote: > > > > Hi, > > > > Replication slots in postgres will prevent removal of required > > resources when there is no connection using them (inactive). This > > consumes storage

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Thu, Jan 18, 2024 at 6:19 AM Peter Eisentraut wrote: > > Very early in this thread, someone mentioned the name > pg_create_subscriber, and of course there is pglogical_create_subscriber > as the historical predecessor. Something along those lines seems better > to me. Maybe there are other

Re: Switching XLog source from archive to streaming when primary available

2024-01-31 Thread Bharath Rupireddy
On Wed, Jan 3, 2024 at 4:58 PM Bharath Rupireddy wrote: > > On Thu, Dec 28, 2023 at 5:26 PM Bharath Rupireddy > wrote: > > > > I took a closer look at v14 and came up with the following changes: > > > > 1. Used advance_wal introduced by commit c161ab74f7. > > 2. Simplified the core logic and new

Re: libpq fails to build with TSAN

2024-01-31 Thread Daniel Gustafsson
> On 31 Jan 2024, at 04:21, Roman Lozko wrote: > > Hi, so libpq has this line in its Makefile > https://github.com/postgres/postgres/blob/6ee26c6a4bafabbd22a85f575d2446fd5ec6ad0d/src/interfaces/libpq/Makefile#L116 > which checks that libpq does not use any "exit" functions. With >

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alexander Kuzmenkov
I'd be happy to see this backpatched. What kind of regressions are we worried about? I'd say partition-wise sort + merge should be faster than append + sort for reasonably sized tables. That's basically what tuplesort does inside. Moreso, this can enable index scans on partitions, which is an even

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2024-01-31 Thread Bharath Rupireddy
On Wed, Jan 31, 2024 at 3:01 PM Alvaro Herrera wrote: > > Looking at 0003, where an XXX comment is added about taking a spinlock > to read LogwrtResult, I suspect the answer is probably not, because it > is likely to slow down the other uses of LogwrtResult. We avoided keeping LogwrtResult

Re: POC, WIP: OR-clause support for indexes

2024-01-31 Thread Alena Rybakina
Hi, thank you for your review and interest in this subject. On 31.01.2024 13:15, jian he wrote: On Wed, Jan 31, 2024 at 10:55 AM jian he wrote: based on my understanding of https://www.postgresql.org/docs/current/xoper-optimization.html#XOPER-COMMUTATOR I think you need move commutator check

Re: CI and test improvements

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-31, vignesh C wrote: > Are we planning to do anything more on this? I was not sure if we > should move this to next commitfest or return it. Well, the patches don't apply anymore since .cirrus.tasks.yml has been created. However, I'm sure we still want [some of] the improvements to

Re: Improve the connection failure error messages

2024-01-31 Thread Nisha Moond
> AFAIK some recent commits patches (e,g [1] for the "slot sync" > development) have created some more cases of "could not connect..." > messages. So, you might need to enhance your patch to deal with any > new ones in the latest HEAD. > > == > [1] >

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Amit Kapila
On Wed, Jan 31, 2024 at 2:02 PM Masahiko Sawada wrote: > > Thank you for updating the patches. As for the slotsync worker patch, > is there any reason why 0001, 0002, and 0004 patches are still > separated? > No specific reason, it could be easier to review those parts. > > Beside, here are

Re: Extending SMgrRelation lifetimes

2024-01-31 Thread Heikki Linnakangas
On 31/01/2024 10:54, Thomas Munro wrote: On Wed, Nov 29, 2023 at 1:42 PM Heikki Linnakangas wrote: I spent some more time digging into this, experimenting with different approaches. Came up with pretty significant changes; see below: Hi Heikki, I think this approach is good. As I wrote in

Re: Possibility to disable `ALTER SYSTEM`

2024-01-31 Thread Gabriele Bartolini
Hi there, I very much like the idea of a file in the data directory that also controls the copy operations. Just wanted to highlight though that in our operator we have already applied the read-only postgresql.auto.conf trick to disable the system (see

Re: POC, WIP: OR-clause support for indexes

2024-01-31 Thread jian he
On Wed, Jan 31, 2024 at 10:55 AM jian he wrote: > > based on my understanding of > https://www.postgresql.org/docs/current/xoper-optimization.html#XOPER-COMMUTATOR > I think you need move commutator check right after the `if > (get_op_rettype(opno) != BOOLOID)` branch > I was wrong about this

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-01-31 Thread Alvaro Herrera
This is great to hear. On 2024-Jan-31, Antonin Houska wrote: > Is your plan to work on it soon or should I try to write a draft patch? (I > assume this is for PG >= 18.) I don't have plans for it, so if you have resources, please go for it. -- Álvaro Herrera PostgreSQL Developer —

libpq fails to build with TSAN

2024-01-31 Thread Roman Lozko
Hi, so libpq has this line in its Makefile https://github.com/postgres/postgres/blob/6ee26c6a4bafabbd22a85f575d2446fd5ec6ad0d/src/interfaces/libpq/Makefile#L116 which checks that libpq does not use any "exit" functions. With ThreadSanitizer it triggers on function `__tsan_func_exit` which is used

Re: Emitting JSON to file using COPY TO

2024-01-31 Thread Junwang Zhao
Hi Vignesh, On Wed, Jan 31, 2024 at 5:50 PM vignesh C wrote: > > On Sat, 27 Jan 2024 at 11:25, Junwang Zhao wrote: > > > > Hi hackers, > > > > Kou-san(CCed) has been working on *Make COPY format extendable[1]*, so > > I think making *copy to json* based on that work might be the right > >

Re: Transaction timeout

2024-01-31 Thread Andrey Borodin
> On 31 Jan 2024, at 14:27, Japin Li wrote: > > LGTM. > > If there is no other objections, I'll change it to ready for committer > next Monday. I think we have a quorum, so I decided to go ahead and flipped status to RfC. Thanks! Best regards, Andrey Borodin.

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-01-31 Thread Antonin Houska
Alvaro Herrera wrote: > On 2024-Jan-30, Pavel Stehule wrote: > > > One of my customer today is reducing one table from 140GB to 20GB. Now he > > is able to run archiving. He should play with pg_repack, and it is working > > well today, but I ask myself, what pg_repack does not be hard to do >

  1   2   >