Re: Why is parula failing?

2024-04-08 Thread David Rowley
On Mon, 8 Apr 2024 at 23:56, Robins Tharakan wrote: > #3 0x0083ed84 in WaitLatch (latch=, > wakeEvents=wakeEvents@entry=41, timeout=60, > wait_event_info=wait_event_info@entry=150994946) at latch.c:538 > #4 0x00907404 in pg_sleep (fcinfo=) at misc.c:406 > #17

Re: Vectored I/O in bulk_write.c

2024-04-08 Thread Thomas Munro
Here's a rebase. I decided against committing this for v17 in the end. There's not much wrong with it AFAIK, except perhaps an unprincipled chopping up of writes with large io_combine_limit due to simplistic flow control, and I liked the idea of having a decent user of smgrwritev() in the tree,

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Amit Kapila
On Mon, Apr 8, 2024 at 7:01 PM Zhijie Hou (Fujitsu) wrote: > > Thanks for pushing. > > I checked the BF status, and noticed one BF failure, which I think is related > to > a miss in the test code. > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27 > > From

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Thomas Munro
On Tue, Apr 9, 2024 at 5:01 PM Michael Paquier wrote: > On Mon, Apr 08, 2024 at 12:23:56PM +0300, Nazir Bilal Yavuz wrote: > > On Mon, 8 Apr 2024 at 11:00, Alexander Lakhin wrote: > >> As I wrote in [1], I didn't observe the issue with clang-18, so maybe it > >> is fixed already. > >> Perhaps

Re: WIP Incremental JSON Parser

2024-04-08 Thread Michael Paquier
On Tue, Apr 09, 2024 at 09:48:18AM +0900, Michael Paquier wrote: > There is no direct check on test_json_parser_perf.c, either, only a > custom rule in the Makefile without specifying something for meson. > So it looks like you could do short execution check in a TAP test, at > least. While

Issue with the PRNG used by Postgres

2024-04-08 Thread Parag Paul
hi all, We have an interesting problem, where PG went to PANIC due to stuck spinlock case. On careful analysis and hours of trying to reproduce this(something that showed up in production after almost 2 weeks of stress run), I did some statistical analysis on the RNG generator that PG uses to

Re: post-freeze damage control

2024-04-08 Thread Peter Eisentraut
On 09.04.24 00:58, Michael Paquier wrote: That's more linked to the fact that I was going silent without a laptop for a few weeks before the end of the release cycle, and a way to say to not count on me, while I was trying to keep my room clean to avoid noise for others who would rush patches.

Fixup a few 2023 copyright years

2024-04-08 Thread David Rowley
Attached is a patch which adjusts the copyright years of 2023 that have crept in this year from patches that were written last year and committed without adjusting this to 2024. The patch isn't produced by src/tools/copyright.pl as that'll transform files which are new and only contain "2023" to

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 12:23:56PM +0300, Nazir Bilal Yavuz wrote: > On Mon, 8 Apr 2024 at 11:00, Alexander Lakhin wrote: >> As I wrote in [1], I didn't observe the issue with clang-18, so maybe it >> is fixed already. >> Perhaps it's worth rechecking... >> >> [1] >>

Re: Fixup a few 2023 copyright years

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 10:36:41PM -0400, Tom Lane wrote: > We *should* do this sometime before branching v17, but I'm not > in any hurry. My thought here is that some of these late changes > might end up getting reverted, in which case touching those files > would add a bit more complexity to

Re: Weird test mixup

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 12:29:43PM +0300, Andrey M. Borodin wrote: > On 8 Apr 2024, at 11:55, Michael Paquier wrote: >> Uh, I did not understand this. Because commit message was about >> stabiilzizing tests, not extending coverage. Okay, it is about stabilizing an existing test. > Also, should

Re: CASE control block broken by a single line comment

2024-04-08 Thread Pavel Stehule
út 9. 4. 2024 v 0:55 odesílatel Tom Lane napsal: > Erik Wienhold writes: > > On 2024-04-07 06:33 +0200, Tom Lane wrote: > >> I suspect it'd be much more robust if we could remove the comment from > >> the expr->query string. No idea how hard that is. > > > I slept on it and I think this can be

Re: post-freeze damage control

2024-04-08 Thread Andrei Lepikhov
On 9/4/2024 09:12, Tom Lane wrote: I have another one that I'm not terribly happy about: Author: Alexander Korotkov Branch: master [72bd38cc9] 2024-04-08 01:27:52 +0300 Transform OR clauses to ANY expression Because I'm primary author of the idea, let me answer. I don't

Re: Allow non-superuser to cancel superuser tasks.

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 05:42:05PM +, Leung, Anthony wrote: > Are you suggesting that we check if the backend is B_AUTOVAC in > pg_cancel/ terminate_backend? That seems a bit unclean to me since > pg_cancel_backend & pg_cancel_backend does not access to the > procNumber to check the type of

Re: pgsql: Fix the intermittent buildfarm failures in 040_standby_failover_

2024-04-08 Thread Amit Kapila
On Mon, Apr 8, 2024 at 9:24 PM Robert Haas wrote: > > Hi, > > I'm concerned that the failover slots feature may not be in > sufficiently good shape for us to ship it. Since this test file was > introduced at the end of January, it's been touched by a total of 16 > commits, most of which seem to

Re: Fixup some StringInfo usages

2024-04-08 Thread Tom Lane
David Rowley writes: > Similar to f736e188c, I've attached a patch that fixes up a few > misusages of the StringInfo functions. These just swap one function > call for another function that is more suited to the use case. > I feel like it's a good idea to fix these soon while they're new >

Re: Fixup a few 2023 copyright years

2024-04-08 Thread Tom Lane
David Rowley writes: > Attached is a patch which adjusts the copyright years of 2023 that > have crept in this year from patches that were written last year and > committed without adjusting this to 2024. > The patch isn't produced by src/tools/copyright.pl as that'll > transform files which are

Re: "an SQL" vs. "a SQL"

2024-04-08 Thread David Rowley
On Tue, 11 Apr 2023 at 17:43, David Rowley wrote: > > On Fri, 11 Jun 2021 at 13:44, David Rowley wrote: > > Anyway, I'll set an alarm for this time next year so I can check on > > how many inconsistencies have crept back in over the development > > cycle. > > That alarm went off today. > > There

Re: libpq.sgml: "server ejectes GSS" -> server rejects GSS

2024-04-08 Thread Heikki Linnakangas
On 09/04/2024 07:40, Erik Rijkers wrote: Typo. fix: -attempted first. If the server ejectes GSS encryption, SSL is +attempted first. If the server rejects GSS encryption, SSL is Fixed, thanks! -- Heikki Linnakangas Neon (https://neon.tech)

Re: post-freeze damage control

2024-04-08 Thread Tom Lane
Andrei Lepikhov writes: > On 9/4/2024 09:12, Tom Lane wrote: >> I have another one that I'm not terribly happy about: >> Author: Alexander Korotkov >> Branch: master [72bd38cc9] 2024-04-08 01:27:52 +0300 >> Transform OR clauses to ANY expression >> * What the medical community would call

Re: post-freeze damage control

2024-04-08 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 8, 2024 at 10:42 AM Heikki Linnakangas wrote: >> Can you elaborate, which patches you think were not ready? Let's make >> sure to capture any concrete concerns in the Open Items list. > Hi, > I'm moving this topic to a new thread for better visibility and less

Re: GenBKI emits useless open;close for catalogs without rows

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 11:11:07AM +0300, Andrey M. Borodin wrote: > This is kind reminder that this thread is waiting for your response. > CF entry [0] is in "Waiting on Author", I'll move it to July CF. Hmm, is that productive? This patch has been waiting on author since the 1st of February,

Streaming relation data out of order

2024-04-08 Thread Thomas Munro
Hi This idea is due to Robert Haas, who complained that he feared that the streaming I/O API already worked like this. It doesn't, but it could! Here is a concept patch to try it out. Normally, read_stream_next_buffer() spits out buffers in the order that the user's callback generated block

Experimental prefetching of buffer memory

2024-04-08 Thread Thomas Munro
Hi, Here is an experimental patch for read_stream.c. The basic idea is that when read_stream_next_buffer() gives you a page P1, it should also tell the CPU to prefetch the header of the next page P2, and so on. However, I recognise that its lack of timing control may be a fundamental flaw (see

Re: Security lessons from liblzma

2024-04-08 Thread Bruce Momjian
On Sat, Mar 30, 2024 at 04:50:26PM -0400, Robert Haas wrote: > An awful lot of what we do operates on the principle that we know the > people who are involved and trust them, and I'm glad we do trust them, > but the world is full of people who trusted somebody too much and > regretted it

libpq.sgml: "server ejectes GSS" -> server rejects GSS

2024-04-08 Thread Erik Rijkers
Typo. fix: -attempted first. If the server ejectes GSS encryption, SSL is +attempted first. If the server rejects GSS encryption, SSL is Erik--- doc/src/sgml/libpq.sgml.orig 2024-04-09 06:28:36.254541932 +0200 +++ doc/src/sgml/libpq.sgml 2024-04-09 06:30:55.818541454 +0200 @@

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread John Naylor
On Mon, Apr 8, 2024 at 7:42 PM Pavel Borisov wrote: > >> I pushed both of these and see that mylodon complains that anonymous >> unions are a C11 feature. I'm not actually sure that the union with >> uintptr_t is actually needed, though, since that's not accessed as >> such here. The simplest

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Haas
On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > And, as of the moment of typing this email, I get: > =# select '2024-04-08 00:00-12:00' - now() as time_remaining; > time_remaining > - > 13:10:35.688134 > (1 row) > > So there is just a bit more than half a day remaining

Re: WIP Incremental JSON Parser

2024-04-08 Thread Andrew Dunstan
On 2024-04-07 Su 20:58, Tom Lane wrote: Coverity complained that this patch leaks memory: /srv/coverity/git/pgsql-git/postgresql/src/bin/pg_combinebackup/load_manifest.c: 212 in load_backup_manifest() 206 bytes_left -= rc; 207

RE: Synchronizing slots from primary to standby

2024-04-08 Thread Zhijie Hou (Fujitsu)
On Monday, April 8, 2024 6:32 PM Amit Kapila wrote: > > On Mon, Apr 8, 2024 at 12:19 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Saturday, April 6, 2024 12:43 PM Amit Kapila > wrote: > > > On Fri, Apr 5, 2024 at 8:05 PM Bertrand Drouvot > > > wrote: > > > > > > Yeah, that could be the first

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tom Lane
Robert Haas writes: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in your commits, than you > did in the prior 3 weeks combined. I don't know. I think this

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 11:48 AM Matthias van de Meent wrote: > I also think there is already a big issue with a lack of interest in > getting existing patches from non-committers committed, reducing the > set of patches that could be considered is just cheating the numbers > and discouraging

Re: Allow non-superuser to cancel superuser tasks.

2024-04-08 Thread Leung, Anthony
>>> There is pg_read_all_stats as well, so I don't see a big issue in >>> requiring to be a member of this role as well for the sake of what's >>> proposing here. >> >> Well, that tells you quite a bit more than just which PIDs correspond to >> autovacuum workers, but maybe that's good enough for

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tomas Vondra
On 4/8/24 17:48, Matthias van de Meent wrote: > On Mon, 8 Apr 2024 at 17:21, Tomas Vondra > wrote: >> >> ... >> >> For me the main problem with the pre-freeze crush is that it leaves >> pretty much no practical chance to do meaningful review/testing, and >> some of the patches likely went

Re: Security lessons from liblzma

2024-04-08 Thread Jacob Champion
On Fri, Apr 5, 2024 at 5:14 PM Michael Paquier wrote: > Saying that, my spidey sense tingles at the recent commit > 3311ea86edc7, that had the idea to introduce a 20k line output file > based on a 378 line input file full of random URLs. In my experience, > tests don't require to be that large

Re: LogwrtResult contended spinlock

2024-04-08 Thread Jeff Davis
On Mon, 2024-04-08 at 10:24 +0200, Alvaro Herrera wrote: > My trouble with the "copy" term is that we don't use that term > anywhere > in relation to WAL. I got the term from CopyXLogRecordToWAL(). > This "copy" is in > reality just the insertion, after it's finished.  The "Result" suffix > is

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024, 19:08 Andres Freund wrote: > On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > > I was under the impression there are not so many out-of-core table > > > >

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Masahiko Sawada
On Tue, Apr 9, 2024 at 12:30 AM Andres Freund wrote: > > Hi, > > On 2024-04-08 09:26:09 -0400, Robert Haas wrote: > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > > And maybe we need to think of a way to further mitigate this crush of > > last minute commits. e.g. In the last week,

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > I was under the impression there are not so many out-of-core table > > > AMs, which have non-dummy analysis implementations. And even

Re: Logging parallel worker draught

2024-04-08 Thread Benoit Lobréau
On 4/8/24 10:05, Andrey M. Borodin wrote: Hi Benoit! This is kind reminder that this thread is waiting for your response. CF entry [0] is in "Waiting on Author", I'll move it to July CF. Hi thanks for the reminder, The past month as been hectic for me. It should calm down by next week at

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 16:01:41 +0530, Amit Kapila wrote: > Pushed. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27 This unfortunately is a commit after commit 6f3d8d5e7cc Author: Amit Kapila Date: 2024-04-08 13:21:55 +0530 Fix the intermittent

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jonathan S. Katz
On 4/8/24 8:29 AM, Andres Freund wrote: Hi, On 2024-04-08 09:26:09 -0400, Robert Haas wrote: On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: And maybe we need to think of a way to further mitigate this crush of last minute commits. e.g. In the last week, you can't have more feature

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Alvaro Herrera
On 2024-Apr-08, Robert Haas wrote: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in your commits, than you > did in the prior 3 weeks combined. I don't know.

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-08 Thread Dmitry Koval
Alexander Lakhin, thanks for the problems you found! Unfortunately I can't watch them immediately (event [1]). I will try to start solving them in 12-14 hours. [1] https://pgconf.ru/2024 -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: Improve tab completion for ALTER DEFAULT PRIVILEGE and ALTER TABLE

2024-04-08 Thread vignesh C
On Mon, 8 Apr 2024 at 10:29, Masahiko Sawada wrote: > > On Fri, Apr 5, 2024 at 1:18 AM vignesh C wrote: > > > > On Tue, 2 Apr 2024 at 13:08, Masahiko Sawada wrote: > > > > > > On Mon, Apr 1, 2024 at 10:41 PM vignesh C wrote: > > > > > > > > On Thu, 28 Mar 2024 at 13:05, Masahiko Sawada > > >

Re: Table AM Interface Enhancements

2024-04-08 Thread Pavel Borisov
Hi, Alexander and Andres! On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov wrote: > Hi, > > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I

NLS doesn't work for pg_combinebackup

2024-04-08 Thread Kyotaro Horiguchi
Hello. I noticed that NLS doesn't work for pg_combinebackup. The cause is that the tool forgets to call set_pglocale_pgservice(). This issue is fixed by the following chage. diff --git a/src/bin/pg_combinebackup/pg_combinebackup.c b/src/bin/pg_combinebackup/pg_combinebackup.c index

Re: Weird test mixup

2024-04-08 Thread Andrey M. Borodin
> On 8 Apr 2024, at 10:33, Michael Paquier wrote: > > Thoughts? As an alternative we can make local injection points mutually exclusive. Best regards, Andrey Borodin.

Re: NLS doesn't work for pg_combinebackup

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 04:27:02PM +0900, Kyotaro Horiguchi wrote: > I noticed that NLS doesn't work for pg_combinebackup. The cause is > that the tool forgets to call set_pglocale_pgservice(). > > This issue is fixed by the following chage. Indeed. Good catch. -- Michael signature.asc

RE: Synchronizing slots from primary to standby

2024-04-08 Thread Zhijie Hou (Fujitsu)
On Saturday, April 6, 2024 12:43 PM Amit Kapila wrote: > On Fri, Apr 5, 2024 at 8:05 PM Bertrand Drouvot > wrote: > > > > On Fri, Apr 05, 2024 at 06:23:10PM +0530, Amit Kapila wrote: > > > On Fri, Apr 5, 2024 at 5:17 PM Amit Kapila > wrote: > > > Thinking more on this, it doesn't seem related

Re: Weird test mixup

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 10:22:40AM +0900, Michael Paquier wrote: > For now I have applied 997db123c054 to make the GIN tests with > injection points repeatable as it was an independent issue, and > f587338dec87 to add the local function pieces. Bharath has reported me offlist that one of the new

Re: NLS doesn't work for pg_combinebackup

2024-04-08 Thread Kyotaro Horiguchi
At Mon, 08 Apr 2024 16:27:02 +0900 (JST), Kyotaro Horiguchi wrote in > Hello. > > I noticed that NLS doesn't work for pg_combinebackup. The cause is > that the tool forgets to call set_pglocale_pgservice(). > > This issue is fixed by the following chage. > > diff --git

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Alexander Lakhin
Hello Michael, 08.04.2024 08:23, Michael Paquier wrote: On Fri, Apr 05, 2024 at 06:19:20PM +0200, Jelte Fennema-Nio wrote: Agreed. While not a full solution, I think this patch is still good to address some of the pain: Waiting 10 seconds at the end of my build with only 1 of my 10 cores doing

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Peter Eisentraut
On 05.04.24 18:19, Jelte Fennema-Nio wrote: On Fri, 5 Apr 2024 at 17:24, Andres Freund wrote: I recommend opening a bug report for clang, best with an already preprocessed input file. We're going to need to do something about this from our side as well, I suspect. The times aren't great

Re: remaining sql/json patches

2024-04-08 Thread Amit Langote
On Mon, Apr 8, 2024 at 2:02 PM jian he wrote: > On Mon, Apr 8, 2024 at 11:21 AM jian he wrote: > > > > On Mon, Apr 8, 2024 at 12:34 AM jian he wrote: > > > > > > On Sun, Apr 7, 2024 at 9:36 PM Amit Langote > > > wrote: > > > > 0002 needs an expanded commit message but I've run out of energy

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Nazir Bilal Yavuz
Hi, On Mon, 8 Apr 2024 at 11:00, Alexander Lakhin wrote: > > Hello Michael, > > 08.04.2024 08:23, Michael Paquier wrote: > > On Fri, Apr 05, 2024 at 06:19:20PM +0200, Jelte Fennema-Nio wrote: > >> Agreed. While not a full solution, I think this patch is still good to > >> address some of the

Re: Weird test mixup

2024-04-08 Thread Andrey M. Borodin
> On 8 Apr 2024, at 11:55, Michael Paquier wrote: > > the point of the test is to make sure that the local > cleanup happens Uh, I did not understand this. Because commit message was about stabiilzizing tests, not extending coverage. Also, should we drop function wait_pid() at the end of a

Re: Should consider materializing the cheapest inner path in consider_parallel_nestloop()

2024-04-08 Thread Andrey M. Borodin
> On 27 Sep 2023, at 16:06, tender wang wrote: > >Do you have any comments or suggestions on this issue? Thanks. Hi Tender, there are some review comments in the thread, that you might be interested in. I'll mark this [0] entry "Waiting on Author" and move to next CF. Thanks! Best

Re: Logging parallel worker draught

2024-04-08 Thread Andrey M. Borodin
> On 29 Feb 2024, at 11:24, Benoit Lobréau wrote: > > Yes, thanks for the proposal, I'll work on it on report here. Hi Benoit! This is kind reminder that this thread is waiting for your response. CF entry [0] is in "Waiting on Author", I'll move it to July CF. Thanks! Best regards,

Re: GenBKI emits useless open;close for catalogs without rows

2024-04-08 Thread Andrey M. Borodin
> On 22 Sep 2023, at 18:50, Matthias van de Meent > wrote: Hi Matthias! This is kind reminder that this thread is waiting for your response. CF entry [0] is in "Waiting on Author", I'll move it to July CF. Thanks! Best regards, Andrey Borodin. [0]

Re: LogwrtResult contended spinlock

2024-04-08 Thread Alvaro Herrera
On 2024-Apr-07, Jeff Davis wrote: > On Sun, 2024-04-07 at 14:19 +0200, Alvaro Herrera wrote: > > I pushed the "copy" pointer now, except that I renamed it to > > "insert", > > which is what we call the operation being tracked.  I also added some > > comments. > > The "copy" pointer, as I called

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 10:00, Alexander Lakhin wrote: > As I wrote in [1], I didn't observe the issue with clang-18, so maybe it > is fixed already. > Perhaps it's worth rechecking... Using the attached script I got these timings. Clang is significantly slower in all of them. But especially with

Re: Experiments with Postgres and SSL

2024-04-08 Thread Heikki Linnakangas
On 08/04/2024 04:25, Heikki Linnakangas wrote: One important open item now is that we need to register a proper ALPN protocol ID with IANA. I sent a request for that: https://mailarchive.ietf.org/arch/msg/tls-reg-review/9LWPzQfOpbc8dTT7vc9ahNeNaiw/ -- Heikki Linnakangas Neon

Re: generic plans and "initial" pruning

2024-04-08 Thread Amit Langote
Hi Andrey, On Sun, Mar 31, 2024 at 2:03 PM Andrey M. Borodin wrote: > > On 6 Dec 2023, at 23:52, Robert Haas wrote: > > > > I hope that it's at least somewhat useful. > > > On 5 Jan 2024, at 15:46, vignesh C wrote: > > > > There is a leak reported > > Hi Amit, > > this is a kind reminder that

Re: Weird test mixup

2024-04-08 Thread Michael Paquier
On Mon, Apr 08, 2024 at 10:42:08AM +0300, Andrey M. Borodin wrote: > As an alternative we can make local injection points mutually exclusive. Sure. Now, the point of the test is to make sure that the local cleanup happens, so I'd rather keep it as-is and use the same names across reloads. --

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 10:02, Peter Eisentraut wrote: > I have tested this with various compilers, and I can confirm that this > shaves off about 5 seconds from the build wall-clock time, which > represents about 10%-20% of the total time. I think this is a good patch. Great to hear. >

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Alexander Lakhin
Hello Jelte, 08.04.2024 11:36, Jelte Fennema-Nio wrote: On Mon, 8 Apr 2024 at 10:00, Alexander Lakhin wrote: As I wrote in [1], I didn't observe the issue with clang-18, so maybe it is fixed already. Perhaps it's worth rechecking... Using the attached script I got these timings. Clang is

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 10:18 AM Pavel Borisov wrote: > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov wrote: >> >> Hi, >> >> On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: >> > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: >> > > I've pushed 0001, 0002 and 0006. >> > >> > I

Re: Catalog domain not-null constraints

2024-04-08 Thread Peter Eisentraut
On 21.03.24 12:23, Peter Eisentraut wrote: All the examples in the tests append "value" to this, presumably by analogy with CHECK constraints, but it looks as though anything works, and is simply ignored: ALTER DOMAIN d ADD CONSTRAINT nn NOT NULL xxx; -- works That doesn't seem particularly

Re: Table AM Interface Enhancements

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov wrote: > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > significant changes just before commit. > I'll revert this later today. Alexander, Exactly how much is getting reverted here? I see these, all since March

post-freeze damage control

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 10:42 AM Heikki Linnakangas wrote: > Can you elaborate, which patches you think were not ready? Let's make > sure to capture any concrete concerns in the Open Items list. Hi, I'm moving this topic to a new thread for better visibility and less admixture of concerns. I'd

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Daniel Verite
Alexander Lakhin wrote: > >> Now that ExecQueryUsingCursor() is gone, it's not clear, what does > >> the following comment mean:? > >> * We must turn off gexec_flag to avoid infinite recursion. Note that > >> * this allows ExecQueryUsingCursor to be applied to the individual >

Re: Converting README documentation to Markdown

2024-04-08 Thread Daniel Gustafsson
> On 8 Apr 2024, at 22:30, Erik Wienhold wrote: > On 2024-04-08 21:29 +0200, Daniel Gustafsson wrote: > I've only peeked at a couple of those READMEs, but they look alright so > far (at least on GitHub). Should we settle on a specific Markdown > flavor[1]? Because I'm never sure if some

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-04-08 Thread Alena Rybakina
Hi! Thank you for your work on this issue! Your patch required a little revision. I did this and attached the patch. Also, I think you should add some clarification to the comments about printing 'exact' and 'loosy' pages in show_hashagg_info function, which you get from planstate->stats,

Re: Improve eviction algorithm in ReorderBuffer

2024-04-08 Thread Jeff Davis
On Mon, 2024-04-08 at 21:29 +0900, Masahiko Sawada wrote: > For v17, changes for #2 are smaller, but I'm concerned > that the new API that requires a hash function to be able to use > binaryheap_update_{up|down} might not be user friendly. The only API change in 02 is accepting a hash callback

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jesper Pedersen
Hi, On 4/8/24 14:15, Tomas Vondra wrote: I think we need to fix & improve that - not to rework/push it at the very end. This is going to be very extreme... Either a patch is ready for merge or it isn't - when 2 or more Committers agree on it then it can be merged - the policy have to be

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 20:15, Tomas Vondra wrote: > I 100% understand how frustrating the lack of progress can be, and I > agree we need to do better. I tried to move a number of stuck patches > this CF, and I hope (and plan) to do more of this in the future. > > But I don't quite see how would

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Tom Lane
Alexander Lakhin writes: > 08.04.2024 18:08, Tom Lane wrote: >> Hmm, the point about recursion is still valid isn't it? I agree the >> reference to ExecQueryUsingCursor is obsolete, but I think we need to >> reconstruct what this comment is actually talking about. It's >> certainly pretty

Re: Converting README documentation to Markdown

2024-04-08 Thread Erik Wienhold
On 2024-04-08 21:29 +0200, Daniel Gustafsson wrote: > Over in [0] I asked whether it would be worthwhile converting all our README > files to Markdown, and since it wasn't met with pitchforks I figured it would > be an interesting excercise to see what it would take (my honest gut feeling > was

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Robert Haas
On Mon, Apr 8, 2024 at 3:32 PM Jelte Fennema-Nio wrote: > Maybe a better solution to this problem would be to spread impactful > reviews by committers more evenly throughout the year. Then there > wouldn't be such a rush to address them in the last commit fest. Spreading activity of all sorts

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-08 Thread Dmitry Koval
Hi! Attached fix for the problems found by Alexander Lakhin. About grammar errors. Unfortunately, I don't know English well. Therefore, I plan (in the coming days) to show the text to specialists who perform technical translation of documentation. -- With best regards, Dmitry Koval Postgres

Re: Table AM Interface Enhancements

2024-04-08 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > wrote: > > Yes, it was my mistake. I got rushing trying to fit this to FF, even doing > > significant changes just before commit. > > I'll revert this later today. It appears to be a

Re: Why is parula failing?

2024-04-08 Thread Robins Tharakan
On Tue, 2 Apr 2024 at 15:01, Tom Lane wrote: > "Tharakan, Robins" writes: > > So although HEAD ran fine, but I saw multiple failures (v12, v13, v16) all of which passed on subsequent-tries, > > of which some were even"signal 6: Aborted". > > Ugh... parula didn't send any reports to buildfarm

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-08 Thread Alexander Lakhin
Hi Tender Wang, 08.04.2024 13:43, Tender Wang wrote: Hi all,   I went through the MERGE/SPLIT partition codes today, thanks for the works.  I found some grammar errors:  i. in error messages(Users can see this grammar errors, not friendly). ii. in codes comments On a quick glance, I saw

Re: Flushing large data immediately in pqcomm

2024-04-08 Thread Ranier Vilela
Em seg., 8 de abr. de 2024 às 07:42, Jelte Fennema-Nio escreveu: > On Sun, 7 Apr 2024 at 14:41, David Rowley wrote: > > Looking at the code in socket_putmessage_noblock(), I don't understand > > why it's ok for PqSendBufferSize to be int but "required" must be > > size_t. There's a line that

Re: Add last_commit_lsn to pg_stat_database

2024-04-08 Thread Kirill Reshke
Hi James, There are some review in the thread that need to be addressed. it seems that we need to mark this entry "Waiting on Author" and move to the next CF [0]. Thanks [0] https://commitfest.postgresql.org/47/4355/ On Sat, 10 Jun 2023 at 05:27, James Coleman wrote: > I've previously noted

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread Pavel Borisov
Hi, John! On Mon, 8 Apr 2024 at 03:13, John Naylor wrote: > On Mon, Apr 8, 2024 at 2:07 AM Andres Freund wrote: > > > > Looking at the code, the failure isn't suprising anymore: > > chardata[MaxBlocktableEntrySize]; > > BlocktableEntry *page = (BlocktableEntry *)

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread John Naylor
On Sun, Apr 7, 2024 at 9:08 AM John Naylor wrote: > > I've attached a mostly-polished update on runtime embeddable values, > storing up to 3 offsets in the child pointer (1 on 32-bit platforms). > As discussed, this includes a macro to cap max possible offset that > can be stored in the bitmap,

Re: Flushing large data immediately in pqcomm

2024-04-08 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 11:34, David Rowley wrote: > That seems to require modifying the following function signatures: > secure_write(), be_tls_write(), be_gssapi_write(). That's not an area > I'm familiar with, however. Attached is a new patchset where 0003 does exactly that. The only place

Re: Improve eviction algorithm in ReorderBuffer

2024-04-08 Thread Masahiko Sawada
On Sat, Apr 6, 2024 at 5:44 AM Jeff Davis wrote: > > > > > It sounds like a data structure that mixes the hash table and the > > binary heap and we use it as the main storage (e.g. for > > ReorderBufferTXN) instead of using the binary heap as the secondary > > data structure. IIUC with your idea,

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-08 Thread Pavel Borisov
On Mon, 8 Apr 2024 at 16:27, John Naylor wrote: > On Sun, Apr 7, 2024 at 9:08 AM John Naylor > wrote: > > > > I've attached a mostly-polished update on runtime embeddable values, > > storing up to 3 offsets in the child pointer (1 on 32-bit platforms). > > As discussed, this includes a macro to

Re: WIP Incremental JSON Parser

2024-04-08 Thread Andrew Dunstan
On 2024-04-08 Mo 14:24, Jacob Champion wrote: Michael pointed out over at [1] that the new tiny.json is pretty inscrutable given its size, and I have to agree. Attached is a patch to pare it down 98% or so. I think people wanting to run the performance comparisons will need to come up with

Re: enhance the efficiency of migrating particularly large tables

2024-04-08 Thread David Rowley
On Tue, 9 Apr 2024 at 09:52, David Zhang wrote: > However, when executing SELECT min(ctid) and max(ctid), it performs a > Seq Scan, which can be slow for a large table. Is there a way to > retrieve the minimum and maximum ctid other than using the system > functions min() and max()? Finding the

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Tomas Vondra
On 4/8/24 21:32, Jelte Fennema-Nio wrote: > On Mon, 8 Apr 2024 at 20:15, Tomas Vondra > wrote: >> I 100% understand how frustrating the lack of progress can be, and I >> agree we need to do better. I tried to move a number of stuck patches >> this CF, and I hope (and plan) to do more of this

Re: post-freeze damage control

2024-04-08 Thread Michael Paquier
On Tue, Apr 09, 2024 at 09:35:00AM +1200, Thomas Munro wrote: > Mr Paquier this year announced his personal code freeze a few weeks > back on social media, which seemed like an interesting idea I might > adopt. Perhaps that is what some other people are doing without > saying so, and perhaps the

Re: enhance the efficiency of migrating particularly large tables

2024-04-08 Thread Tom Lane
David Rowley writes: > Unsure if such a feature is worthwhile. I think maybe not for just > min(ctid)/max(ctid). However, there could be other reasons, such as > the transform OR to UNION stuff that Tom worked on a few years ago. > That needed to eliminate duplicate rows that matched both OR

Re: post-freeze damage control

2024-04-08 Thread Thomas Munro
On Tue, Apr 9, 2024 at 7:47 AM Robert Haas wrote: > - The streaming read API stuff was all committed very last minute. I > think this should have been committed much sooner. It's probably not > going to break the world; it's more likely to have performance > consequences. But if it had gone in

enhance the efficiency of migrating particularly large tables

2024-04-08 Thread David Zhang
Hi Postgres hackers, I'm reaching out to gather some comments on enhancing the efficiency of migrating particularly large tables with significant data volumes in PostgreSQL. When migrating a particularly large table with a significant amount of data, users sometimes tend to split the table

Re: WIP Incremental JSON Parser

2024-04-08 Thread Andrew Dunstan
On 2024-04-08 Mo 09:29, Andrew Dunstan wrote: On 2024-04-07 Su 20:58, Tom Lane wrote: Coverity complained that this patch leaks memory: /srv/coverity/git/pgsql-git/postgresql/src/bin/pg_combinebackup/load_manifest.c: 212 in load_backup_manifest() 206 bytes_left -= rc; 207   

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Andrew Dunstan
On 2024-04-08 Mo 12:07, Alvaro Herrera wrote: On 2024-Apr-08, Robert Haas wrote: And maybe we need to think of a way to further mitigate this crush of last minute commits. e.g. In the last week, you can't have more feature commits, or more lines of insertions in your commits, than you did in

  1   2   >