Re: post-freeze damage control

2024-04-08 Thread Peter Eisentraut
On 09.04.24 00:58, Michael Paquier wrote: That's more linked to the fact that I was going silent without a laptop for a few weeks before the end of the release cycle, and a way to say to not count on me, while I was trying to keep my room clean to avoid noise for others who would rush patches.

Re: post-freeze damage control

2024-04-08 Thread Tom Lane
Andrei Lepikhov writes: > On 9/4/2024 09:12, Tom Lane wrote: >> I have another one that I'm not terribly happy about: >> Author: Alexander Korotkov >> Branch: master [72bd38cc9] 2024-04-08 01:27:52 +0300 >> Transform OR clauses to ANY expression >> * What the medical community would call

Re: Allow non-superuser to cancel superuser tasks.

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 05:42:05PM +, Leung, Anthony wrote: > Are you suggesting that we check if the backend is B_AUTOVAC in > pg_cancel/ terminate_backend? That seems a bit unclean to me since > pg_cancel_backend & pg_cancel_backend does not access to the > procNumber to check the type of

Issue with the PRNG used by Postgres

2024-04-08 Thread Parag Paul
hi all, We have an interesting problem, where PG went to PANIC due to stuck spinlock case. On careful analysis and hours of trying to reproduce this(something that showed up in production after almost 2 weeks of stress run), I did some statistical analysis on the RNG generator that PG uses to

Re: post-freeze damage control

2024-04-08 Thread Andrei Lepikhov
On 9/4/2024 09:12, Tom Lane wrote: I have another one that I'm not terribly happy about: Author: Alexander Korotkov Branch: master [72bd38cc9] 2024-04-08 01:27:52 +0300 Transform OR clauses to ANY expression Because I'm primary author of the idea, let me answer. I don't

Re: WIP Incremental JSON Parser

2024-04-08 Thread Michael Paquier
On Tue, Apr 09, 2024 at 09:48:18AM +0900, Michael Paquier wrote: > There is no direct check on test_json_parser_perf.c, either, only a > custom rule in the Makefile without specifying something for meson. > So it looks like you could do short execution check in a TAP test, at > least. While

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Thomas Munro
On Tue, Apr 9, 2024 at 5:01 PM Michael Paquier wrote: > On Mon, Apr 08, 2024 at 12:23:56PM +0300, Nazir Bilal Yavuz wrote: > > On Mon, 8 Apr 2024 at 11:00, Alexander Lakhin wrote: > >> As I wrote in [1], I didn't observe the issue with clang-18, so maybe it > >> is fixed already. > >> Perhaps

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Amit Kapila
On Mon, Apr 8, 2024 at 7:01 PM Zhijie Hou (Fujitsu) wrote: > > Thanks for pushing. > > I checked the BF status, and noticed one BF failure, which I think is related > to > a miss in the test code. > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27 > > From

Re: libpq.sgml: "server ejectes GSS" -> server rejects GSS

2024-04-08 Thread Heikki Linnakangas
On 09/04/2024 07:40, Erik Rijkers wrote: Typo. fix: -attempted first. If the server ejectes GSS encryption, SSL is +attempted first. If the server rejects GSS encryption, SSL is Fixed, thanks! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 12:23:56PM +0300, Nazir Bilal Yavuz wrote: > On Mon, 8 Apr 2024 at 11:00, Alexander Lakhin wrote: >> As I wrote in [1], I didn't observe the issue with clang-18, so maybe it >> is fixed already. >> Perhaps it's worth rechecking... >> >> [1] >>

Re: Vectored I/O in bulk_write.c

2024-04-08 Thread Thomas Munro
Here's a rebase. I decided against committing this for v17 in the end. There's not much wrong with it AFAIK, except perhaps an unprincipled chopping up of writes with large io_combine_limit due to simplistic flow control, and I liked the idea of having a decent user of smgrwritev() in the tree,

libpq.sgml: "server ejectes GSS" -> server rejects GSS

2024-04-08 Thread Erik Rijkers
Typo. fix: -attempted first. If the server ejectes GSS encryption, SSL is +attempted first. If the server rejects GSS encryption, SSL is Erik--- doc/src/sgml/libpq.sgml.orig 2024-04-09 06:28:36.254541932 +0200 +++ doc/src/sgml/libpq.sgml 2024-04-09 06:30:55.818541454 +0200 @@

Streaming relation data out of order

2024-04-08 Thread Thomas Munro
Hi This idea is due to Robert Haas, who complained that he feared that the streaming I/O API already worked like this. It doesn't, but it could! Here is a concept patch to try it out. Normally, read_stream_next_buffer() spits out buffers in the order that the user's callback generated block

Re: "an SQL" vs. "a SQL"

2024-04-08 Thread David Rowley
On Tue, 11 Apr 2023 at 17:43, David Rowley wrote: > > On Fri, 11 Jun 2021 at 13:44, David Rowley wrote: > > Anyway, I'll set an alarm for this time next year so I can check on > > how many inconsistencies have crept back in over the development > > cycle. > > That alarm went off today. > > There

Re: CASE control block broken by a single line comment

2024-04-08 Thread Pavel Stehule
út 9. 4. 2024 v 0:55 odesílatel Tom Lane napsal: > Erik Wienhold writes: > > On 2024-04-07 06:33 +0200, Tom Lane wrote: > >> I suspect it'd be much more robust if we could remove the comment from > >> the expr->query string. No idea how hard that is. > > > I slept on it and I think this can be

Re: GenBKI emits useless open;close for catalogs without rows

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 11:11:07AM +0300, Andrey M. Borodin wrote: > This is kind reminder that this thread is waiting for your response. > CF entry [0] is in "Waiting on Author", I'll move it to July CF. Hmm, is that productive? This patch has been waiting on author since the 1st of February,

Re: Why is parula failing?

2024-04-08 Thread David Rowley
On Mon, 8 Apr 2024 at 23:56, Robins Tharakan wrote: > #3 0x0083ed84 in WaitLatch (latch=, > wakeEvents=wakeEvents@entry=41, timeout=60, > wait_event_info=wait_event_info@entry=150994946) at latch.c:538 > #4 0x00907404 in pg_sleep (fcinfo=) at misc.c:406 > #17

Re: Weird test mixup

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 12:29:43PM +0300, Andrey M. Borodin wrote: > On 8 Apr 2024, at 11:55, Michael Paquier wrote: >> Uh, I did not understand this. Because commit message was about >> stabiilzizing tests, not extending coverage. Okay, it is about stabilizing an existing test. > Also, should

Re: Fixup a few 2023 copyright years

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 10:36:41PM -0400, Tom Lane wrote: > We *should* do this sometime before branching v17, but I'm not > in any hurry. My thought here is that some of these late changes > might end up getting reverted, in which case touching those files > would add a bit more complexity to

Re: Security lessons from liblzma

2024-04-08 Thread Bruce Momjian
On Sat, Mar 30, 2024 at 04:50:26PM -0400, Robert Haas wrote: > An awful lot of what we do operates on the principle that we know the > people who are involved and trust them, and I'm glad we do trust them, > but the world is full of people who trusted somebody too much and > regretted it

Experimental prefetching of buffer memory

2024-04-08 Thread Thomas Munro
Hi, Here is an experimental patch for read_stream.c. The basic idea is that when read_stream_next_buffer() gives you a page P1, it should also tell the CPU to prefetch the header of the next page P2, and so on. However, I recognise that its lack of timing control may be a fundamental flaw (see

Re: Fixup a few 2023 copyright years

2024-04-08 Thread Tom Lane
David Rowley writes: > Attached is a patch which adjusts the copyright years of 2023 that > have crept in this year from patches that were written last year and > committed without adjusting this to 2024. > The patch isn't produced by src/tools/copyright.pl as that'll > transform files which are

Re: Fixup some StringInfo usages

2024-04-08 Thread Tom Lane
David Rowley writes: > Similar to f736e188c, I've attached a patch that fixes up a few > misusages of the StringInfo functions. These just swap one function > call for another function that is more suited to the use case. > I feel like it's a good idea to fix these soon while they're new >

Fixup a few 2023 copyright years

2024-04-08 Thread David Rowley
Attached is a patch which adjusts the copyright years of 2023 that have crept in this year from patches that were written last year and committed without adjusting this to 2024. The patch isn't produced by src/tools/copyright.pl as that'll transform files which are new and only contain "2023" to

Re: post-freeze damage control

2024-04-08 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 8, 2024 at 10:42 AM Heikki Linnakangas wrote: >> Can you elaborate, which patches you think were not ready? Let's make >> sure to capture any concrete concerns in the Open Items list. > Hi, > I'm moving this topic to a new thread for better visibility and less

Re: pgsql: Fix the intermittent buildfarm failures in 040_standby_failover_

2024-04-08 Thread Amit Kapila
On Mon, Apr 8, 2024 at 9:24 PM Robert Haas wrote: > > Hi, > > I'm concerned that the failover slots feature may not be in > sufficiently good shape for us to ship it. Since this test file was > introduced at the end of January, it's been touched by a total of 16 > commits, most of which seem to

Re: Fixup some StringInfo usages

2024-04-08 Thread Nathan Bossart
On Tue, Apr 09, 2024 at 12:53:21PM +1200, David Rowley wrote: > Similar to f736e188c, I've attached a patch that fixes up a few > misusages of the StringInfo functions. These just swap one function > call for another function that is more suited to the use case. > > I've also attached the patch

Fixup some StringInfo usages

2024-04-08 Thread David Rowley
Similar to f736e188c, I've attached a patch that fixes up a few misusages of the StringInfo functions. These just swap one function call for another function that is more suited to the use case. I've also attached the patch that I used to find these. That's not intended for commit. I feel like

Re: WIP Incremental JSON Parser

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 05:42:35PM -0400, Andrew Dunstan wrote: > Arguably the fact that it points nowhere is a good thing. But feel free to > replace it with something else. It doesn't have to be URLs at all. That > happened simply because it was easy to extract from a very large piece of > JSON

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Amit Kapila
On Mon, Apr 8, 2024 at 9:49 PM Andres Freund wrote: > > On 2024-04-08 16:01:41 +0530, Amit Kapila wrote: > > Pushed. > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27 > > This unfortunately is a commit after > Right, and thanks for the report. Hou-San

Re: enhance the efficiency of migrating particularly large tables

2024-04-08 Thread David Rowley
On Tue, 9 Apr 2024 at 11:02, Tom Lane wrote: > > David Rowley writes: > > Unsure if such a feature is worthwhile. I think maybe not for just > > min(ctid)/max(ctid). However, there could be other reasons, such as > > the transform OR to UNION stuff that Tom worked on a few years ago. > > That

Re: post-freeze damage control

2024-04-08 Thread Michael Paquier
On Tue, Apr 09, 2024 at 01:16:02AM +0200, Tomas Vondra wrote: > I don't feel too particularly worried about this. Yes, backups are super > important because it's often the only thing you have left when things go > wrong, and the incremental aspect is all new. The code I've seen while > doing the

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tom Lane
Andrew Dunstan writes: > I quite like the triage idea. But I think there's also a case for being > more a bit more flexible with those patches we don't throw out. A case > close to my heart: I'd have been very sad if the NESTED piece of > JSON_TABLE hadn't made the cut, which it did with a few

Re: post-freeze damage control

2024-04-08 Thread Tomas Vondra
On 4/8/24 21:47, Robert Haas wrote: > On Mon, Apr 8, 2024 at 10:42 AM Heikki Linnakangas wrote: >> Can you elaborate, which patches you think were not ready? Let's make >> sure to capture any concrete concerns in the Open Items list. > > ... > > - Incremental backup could have all sorts of

Re: enhance the efficiency of migrating particularly large tables

2024-04-08 Thread Tom Lane
David Rowley writes: > Unsure if such a feature is worthwhile. I think maybe not for just > min(ctid)/max(ctid). However, there could be other reasons, such as > the transform OR to UNION stuff that Tom worked on a few years ago. > That needed to eliminate duplicate rows that matched both OR

Re: post-freeze damage control

2024-04-08 Thread Michael Paquier
On Tue, Apr 09, 2024 at 09:35:00AM +1200, Thomas Munro wrote: > Mr Paquier this year announced his personal code freeze a few weeks > back on social media, which seemed like an interesting idea I might > adopt. Perhaps that is what some other people are doing without > saying so, and perhaps the

Re: CASE control block broken by a single line comment

2024-04-08 Thread Tom Lane
Erik Wienhold writes: > On 2024-04-07 06:33 +0200, Tom Lane wrote: >> I suspect it'd be much more robust if we could remove the comment from >> the expr->query string. No idea how hard that is. > I slept on it and I think this can be fixed by tracking the end of the > last token before THEN and

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Andrew Dunstan
On 2024-04-08 Mo 12:07, Alvaro Herrera wrote: On 2024-Apr-08, Robert Haas wrote: And maybe we need to think of a way to further mitigate this crush of last minute commits. e.g. In the last week, you can't have more feature commits, or more lines of insertions in your commits, than you did in

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tomas Vondra
On 4/8/24 21:32, Jelte Fennema-Nio wrote: > On Mon, 8 Apr 2024 at 20:15, Tomas Vondra > wrote: >> I 100% understand how frustrating the lack of progress can be, and I >> agree we need to do better. I tried to move a number of stuck patches >> this CF, and I hope (and plan) to do more of this

Re: enhance the efficiency of migrating particularly large tables

2024-04-08 Thread David Rowley
On Tue, 9 Apr 2024 at 09:52, David Zhang wrote: > However, when executing SELECT min(ctid) and max(ctid), it performs a > Seq Scan, which can be slow for a large table. Is there a way to > retrieve the minimum and maximum ctid other than using the system > functions min() and max()? Finding the

Re: WIP Incremental JSON Parser

2024-04-08 Thread Andrew Dunstan
On 2024-04-08 Mo 09:29, Andrew Dunstan wrote: On 2024-04-07 Su 20:58, Tom Lane wrote: Coverity complained that this patch leaks memory: /srv/coverity/git/pgsql-git/postgresql/src/bin/pg_combinebackup/load_manifest.c: 212 in load_backup_manifest() 206 bytes_left -= rc; 207   

enhance the efficiency of migrating particularly large tables

2024-04-08 Thread David Zhang
Hi Postgres hackers, I'm reaching out to gather some comments on enhancing the efficiency of migrating particularly large tables with significant data volumes in PostgreSQL. When migrating a particularly large table with a significant amount of data, users sometimes tend to split the table

Re: WIP Incremental JSON Parser

2024-04-08 Thread Andrew Dunstan
On 2024-04-08 Mo 14:24, Jacob Champion wrote: Michael pointed out over at [1] that the new tiny.json is pretty inscrutable given its size, and I have to agree. Attached is a patch to pare it down 98% or so. I think people wanting to run the performance comparisons will need to come up with

Re: post-freeze damage control

2024-04-08 Thread Thomas Munro
On Tue, Apr 9, 2024 at 7:47 AM Robert Haas wrote: > - The streaming read API stuff was all committed very last minute. I > think this should have been committed much sooner. It's probably not > going to break the world; it's more likely to have performance > consequences. But if it had gone in

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-04-08 Thread Alena Rybakina
Hi! Thank you for your work on this issue! Your patch required a little revision. I did this and attached the patch. Also, I think you should add some clarification to the comments about printing 'exact' and 'loosy' pages in show_hashagg_info function, which you get from planstate->stats,

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > wrote: > > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > > significant changes just before commit. > > I'll revert this later today. It appears to be a

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-08 Thread Dmitry Koval
Hi! Attached fix for the problems found by Alexander Lakhin. About grammar errors. Unfortunately, I don't know English well. Therefore, I plan (in the coming days) to show the text to specialists who perform technical translation of documentation. -- With best regards, Dmitry Koval Postgres

Re: Converting README documentation to Markdown

2024-04-08 Thread Daniel Gustafsson
> On 8 Apr 2024, at 22:30, Erik Wienhold wrote: > On 2024-04-08 21:29 +0200, Daniel Gustafsson wrote: > I've only peeked at a couple of those READMEs, but they look alright so > far (at least on GitHub). Should we settle on a specific Markdown > flavor[1]? Because I'm never sure if some

Re: Converting README documentation to Markdown

2024-04-08 Thread Erik Wienhold
On 2024-04-08 21:29 +0200, Daniel Gustafsson wrote: > Over in [0] I asked whether it would be worthwhile converting all our README > files to Markdown, and since it wasn't met with pitchforks I figured it would > be an interesting excercise to see what it would take (my honest gut feeling > was

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Tom Lane
Alexander Lakhin writes: > 08.04.2024 18:08, Tom Lane wrote: >> Hmm, the point about recursion is still valid isn't it? I agree the >> reference to ExecQueryUsingCursor is obsolete, but I think we need to >> reconstruct what this comment is actually talking about. It's >> certainly pretty

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Daniel Verite
Alexander Lakhin wrote: > >> Now that ExecQueryUsingCursor() is gone, it's not clear, what does > >> the following comment mean:? > >> * We must turn off gexec_flag to avoid infinite recursion. Note that > >> * this allows ExecQueryUsingCursor to be applied to the individual >

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 3:32 PM Jelte Fennema-Nio wrote: > Maybe a better solution to this problem would be to spread impactful > reviews by committers more evenly throughout the year. Then there > wouldn't be such a rush to address them in the last commit fest. Spreading activity of all sorts

post-freeze damage control

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 10:42 AM Heikki Linnakangas wrote: > Can you elaborate, which patches you think were not ready? Let's make > sure to capture any concrete concerns in the Open Items list. Hi, I'm moving this topic to a new thread for better visibility and less admixture of concerns. I'd

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 20:15, Tomas Vondra wrote: > I 100% understand how frustrating the lack of progress can be, and I > agree we need to do better. I tried to move a number of stuck patches > this CF, and I hope (and plan) to do more of this in the future. > > But I don't quite see how would

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jesper Pedersen
Hi, On 4/8/24 14:15, Tomas Vondra wrote: I think we need to fix & improve that - not to rework/push it at the very end. This is going to be very extreme... Either a patch is ready for merge or it isn't - when 2 or more Committers agree on it then it can be merged - the policy have to be

Re: Improve eviction algorithm in ReorderBuffer

2024-04-08 Thread Jeff Davis
On Mon, 2024-04-08 at 21:29 +0900, Masahiko Sawada wrote: > For v17, changes for #2 are smaller, but I'm concerned > that the new API that requires a hash function to be able to use > binaryheap_update_{up|down} might not be user friendly. The only API change in 02 is accepting a hash callback

Re: Table AM Interface Enhancements

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov wrote: > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > significant changes just before commit. > I'll revert this later today. Alexander, Exactly how much is getting reverted here? I see these, all since March

Re: Security lessons from liblzma

2024-04-08 Thread Jacob Champion
On Fri, Apr 5, 2024 at 5:14 PM Michael Paquier wrote: > Saying that, my spidey sense tingles at the recent commit > 3311ea86edc7, that had the idea to introduce a 20k line output file > based on a 378 line input file full of random URLs. In my experience, > tests don't require to be that large

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tomas Vondra
On 4/8/24 17:48, Matthias van de Meent wrote: > On Mon, 8 Apr 2024 at 17:21, Tomas Vondra > wrote: >> >> ... >> >> For me the main problem with the pre-freeze crush is that it leaves >> pretty much no practical chance to do meaningful review/testing, and >> some of the patches likely went

Re: Allow non-superuser to cancel superuser tasks.

2024-04-08 Thread Leung, Anthony
>>> There is pg_read_all_stats as well, so I don't see a big issue in >>> requiring to be a member of this role as well for the sake of what's >>> proposing here. >> >> Well, that tells you quite a bit more than just which PIDs correspond to >> autovacuum workers, but maybe that's good enough for

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Masahiko Sawada
On Tue, Apr 9, 2024 at 12:30 AM Andres Freund wrote: > > Hi, > > On 2024-04-08 09:26:09 -0400, Robert Haas wrote: > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > > And maybe we need to think of a way to further mitigate this crush of > > last minute commits. e.g. In the last week,

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024, 19:08 Andres Freund wrote: > On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > > I was under the impression there are not so many out-of-core table > > > >

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Alvaro Herrera
On 2024-Apr-08, Robert Haas wrote: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in your commits, than you > did in the prior 3 weeks combined. I don't know.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 11:48 AM Matthias van de Meent wrote: > I also think there is already a big issue with a lack of interest in > getting existing patches from non-committers committed, reducing the > set of patches that could be considered is just cheating the numbers > and discouraging

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jonathan S. Katz
On 4/8/24 8:29 AM, Andres Freund wrote: Hi, On 2024-04-08 09:26:09 -0400, Robert Haas wrote: On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: And maybe we need to think of a way to further mitigate this crush of last minute commits. e.g. In the last week, you can't have more feature

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 16:01:41 +0530, Amit Kapila wrote: > Pushed. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27 This unfortunately is a commit after commit 6f3d8d5e7cc Author: Amit Kapila Date: 2024-04-08 13:21:55 +0530 Fix the intermittent

Re: LogwrtResult contended spinlock

2024-04-08 Thread Jeff Davis
On Mon, 2024-04-08 at 10:24 +0200, Alvaro Herrera wrote: > My trouble with the "copy" term is that we don't use that term > anywhere > in relation to WAL. I got the term from CopyXLogRecordToWAL(). > This "copy" is in > reality just the insertion, after it's finished.  The "Result" suffix > is

Re: Logging parallel worker draught

2024-04-08 Thread Benoit Lobréau
On 4/8/24 10:05, Andrey M. Borodin wrote: Hi Benoit! This is kind reminder that this thread is waiting for your response. CF entry [0] is in "Waiting on Author", I'll move it to July CF. Hi thanks for the reminder, The past month as been hectic for me. It should calm down by next week at

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > I was under the impression there are not so many out-of-core table > > > AMs, which have non-dummy analysis implementations. And even

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Pavel Borisov
On Mon, 8 Apr 2024 at 19:48, Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > On Mon, 8 Apr 2024 at 17:21, Tomas Vondra > wrote: > > > > > > > > On 4/8/24 16:59, Tom Lane wrote: > > > Heikki Linnakangas writes: > > >> On 08/04/2024 16:43, Tom Lane wrote: > > >>> I was just about

Re: pgsql: Fix the intermittent buildfarm failures in 040_standby_failover_

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 4:04 AM Amit Kapila wrote: > Fix the intermittent buildfarm failures in 040_standby_failover_slots_sync. > > It is possible that even if the primary waits for the subscriber to catch > up and then disables the subscription, the XLOG_RUNNING_XACTS record gets > inserted

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Matthias van de Meent
On Mon, 8 Apr 2024 at 17:21, Tomas Vondra wrote: > > > > On 4/8/24 16:59, Tom Lane wrote: > > Heikki Linnakangas writes: > >> On 08/04/2024 16:43, Tom Lane wrote: > >>> I was just about to pen an angry screed along the same lines. > >>> The commit flux over the past couple days, and even the

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > I was under the impression there are not so many out-of-core table > > AMs, which have non-dummy analysis implementations. And even if there > > are some, duplicating

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 09:26:09 -0400, Robert Haas wrote: > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Joe Conway
On 4/8/24 11:05, Tom Lane wrote: Pavel Borisov writes: IMO the fact that people struggle to work on patches, and make them better, etc. is an immense blessing for the Postgres community. Is the peak of commits really a big problem provided we have 6 months before actual release? I doubt March

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tomas Vondra
On 4/8/24 16:59, Tom Lane wrote: > Heikki Linnakangas writes: >> On 08/04/2024 16:43, Tom Lane wrote: >>> I was just about to pen an angry screed along the same lines. >>> The commit flux over the past couple days, and even the last >>> twelve hours, was flat-out ridiculous. These patches

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Alexander Lakhin
08.04.2024 18:08, Tom Lane wrote: Alexander Lakhin writes: Now that ExecQueryUsingCursor() is gone, it's not clear, what does the following comment mean:?    * We must turn off gexec_flag to avoid infinite recursion.  Note that    * this allows ExecQueryUsingCursor to be applied to the

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Tom Lane
Alexander Lakhin writes: > Now that ExecQueryUsingCursor() is gone, it's not clear, what does > the following comment mean:? >    * We must turn off gexec_flag to avoid infinite recursion.  Note that >    * this allows ExecQueryUsingCursor to be applied to the individual query >    * results.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tom Lane
Pavel Borisov writes: > IMO the fact that people struggle to work on patches, and make them better, > etc. is an immense blessing for the Postgres community. Is the peak of > commits really a big problem provided we have 6 months before actual > release? I doubt March patches tend to be worse

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Alexander Lakhin
Hello Daniel and Tom, 08.04.2024 17:25, Daniel Verite wrote: So I whacked the patch around till I liked it better, and pushed it. Thanks for taking care of this! Now that ExecQueryUsingCursor() is gone, it's not clear, what does the following comment mean:?    * We must turn off gexec_flag

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tom Lane
Heikki Linnakangas writes: > On 08/04/2024 16:43, Tom Lane wrote: >> I was just about to pen an angry screed along the same lines. >> The commit flux over the past couple days, and even the last >> twelve hours, was flat-out ridiculous. These patches weren't >> ready a week ago, and I doubt they

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Pavel Borisov
On Mon, 8 Apr 2024 at 18:42, Heikki Linnakangas wrote: > On 08/04/2024 16:43, Tom Lane wrote: > > Robert Haas writes: > >> And maybe we need to think of a way to further mitigate this crush of > >> last minute commits. e.g. In the last week, you can't have more > >> feature commits, or more

Re: CI and test improvements

2024-04-08 Thread Andrey M. Borodin
> On 19 Feb 2024, at 11:33, Peter Eisentraut wrote: > > Ok, I didn't see that my feedback had been addressed. I have committed this > patch. Justin, Peter, I can't determine actual status of the CF entry [0]. May I ask someone of you to move patch to next CF or close as committed? Thanks!

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Melanie Plageman
On Mon, Apr 8, 2024 at 10:41 AM Robert Treat wrote: > > On Mon, Apr 8, 2024 at 10:27 AM Melanie Plageman > wrote: > > On Mon, Apr 8, 2024 at 9:26 AM Robert Haas wrote: > > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier > > > wrote: > > > > And, as of the moment of typing this email, I get:

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Heikki Linnakangas
On 08/04/2024 16:43, Tom Lane wrote: Robert Haas writes: And maybe we need to think of a way to further mitigate this crush of last minute commits. e.g. In the last week, you can't have more feature commits, or more lines of insertions in your commits, than you did in the prior 3 weeks

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Andrey M. Borodin
> On 8 Apr 2024, at 17:26, Melanie Plageman wrote: > > What if we pick the actual feature freeze time randomly? That is, > starting on March 15th (or whenever but more than a week before), each > night someone from RMT generates a random number between $current_day > and April 8th. If the

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Laurenz Albe
On Mon, 2024-04-08 at 09:26 -0400, Robert Haas wrote: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in your commits, than you > did in the prior 3 weeks

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Treat
On Mon, Apr 8, 2024 at 10:27 AM Melanie Plageman wrote: > On Mon, Apr 8, 2024 at 9:26 AM Robert Haas wrote: > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > > > And, as of the moment of typing this email, I get: > > > =# select '2024-04-08 00:00-12:00' - now() as time_remaining; > >

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Melanie Plageman
On Mon, Apr 8, 2024 at 9:26 AM Robert Haas wrote: > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > > And, as of the moment of typing this email, I get: > > =# select '2024-04-08 00:00-12:00' - now() as time_remaining; > > time_remaining > > - > > 13:10:35.688134 > >

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Daniel Verite
Tom Lane wrote: > I've reconsidered after realizing that implementing FETCH_COUNT > atop traditional single-row mode would require either merging > single-row results into a bigger PGresult or persuading psql's > results-printing code to accept an array of PGresults not just > one.

Re: documentation structure

2024-04-08 Thread Peter Eisentraut
On 05.04.24 17:11, Robert Haas wrote: 4. Consolidate the "Generic WAL Records" and "Custom WAL Resource Managers" chapters, which cover related topics, into a single one. I didn't see anyone object to this, but David Johnston pointed out that the patch I posted was a few bricks short of a load,

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tom Lane
Robert Haas writes: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in your commits, than you > did in the prior 3 weeks combined. I don't know. I think this

RE: Synchronizing slots from primary to standby

2024-04-08 Thread Zhijie Hou (Fujitsu)
On Monday, April 8, 2024 6:32 PM Amit Kapila wrote: > > On Mon, Apr 8, 2024 at 12:19 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Saturday, April 6, 2024 12:43 PM Amit Kapila > wrote: > > > On Fri, Apr 5, 2024 at 8:05 PM Bertrand Drouvot > > > wrote: > > > > > > Yeah, that could be the first

Re: WIP Incremental JSON Parser

2024-04-08 Thread Andrew Dunstan
On 2024-04-07 Su 20:58, Tom Lane wrote: Coverity complained that this patch leaks memory: /srv/coverity/git/pgsql-git/postgresql/src/bin/pg_combinebackup/load_manifest.c: 212 in load_backup_manifest() 206 bytes_left -= rc; 207

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Haas
On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > And, as of the moment of typing this email, I get: > =# select '2024-04-08 00:00-12:00' - now() as time_remaining; > time_remaining > - > 13:10:35.688134 > (1 row) > > So there is just a bit more than half a day remaining

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread John Naylor
On Mon, Apr 8, 2024 at 7:42 PM Pavel Borisov wrote: > >> I pushed both of these and see that mylodon complains that anonymous >> unions are a C11 feature. I'm not actually sure that the union with >> uintptr_t is actually needed, though, since that's not accessed as >> such here. The simplest

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread Pavel Borisov
On Mon, 8 Apr 2024 at 16:27, John Naylor wrote: > On Sun, Apr 7, 2024 at 9:08 AM John Naylor > wrote: > > > > I've attached a mostly-polished update on runtime embeddable values, > > storing up to 3 offsets in the child pointer (1 on 32-bit platforms). > > As discussed, this includes a macro to

Re: Improve eviction algorithm in ReorderBuffer

2024-04-08 Thread Masahiko Sawada
On Sat, Apr 6, 2024 at 5:44 AM Jeff Davis wrote: > > > > > It sounds like a data structure that mixes the hash table and the > > binary heap and we use it as the main storage (e.g. for > > ReorderBufferTXN) instead of using the binary heap as the secondary > > data structure. IIUC with your idea,

Re: Flushing large data immediately in pqcomm

2024-04-08 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 11:34, David Rowley wrote: > That seems to require modifying the following function signatures: > secure_write(), be_tls_write(), be_gssapi_write(). That's not an area > I'm familiar with, however. Attached is a new patchset where 0003 does exactly that. The only place

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread John Naylor
On Sun, Apr 7, 2024 at 9:08 AM John Naylor wrote: > > I've attached a mostly-polished update on runtime embeddable values, > storing up to 3 offsets in the child pointer (1 on 32-bit platforms). > As discussed, this includes a macro to cap max possible offset that > can be stored in the bitmap,

  1   2   >