Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 02:49, Andres Freund wrote: > On 2024-04-07 00:19:35 +0200, Daniel Gustafsson wrote: >>> On 6 Apr 2024, at 23:44, Andres Freund wrote: >> The non-context aware fix would be to just print the last 1024 (or something) >> bytes from the logfile: > > That'd be better, yes. I'd

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 08:21, Andres Freund wrote: > I added WITH BINARY, SET STORAGE EXTERNAL and tested both unix socket and > localhost. I also reduced row counts and iteration counts, because I am > impatient, and I don't think it matters much here. Attached the modified > version. Thanks for

Re: [PATCH] Exponential backoff for auth_delay

2024-04-07 Thread Michael Banck
Hi, On Wed, Mar 20, 2024 at 11:22:12PM +0100, Daniel Gustafsson wrote: > > On 20 Mar 2024, at 22:21, Jacob Champion > > wrote: > > > > On Wed, Mar 20, 2024 at 2:15 PM Jacob Champion > > wrote: > >> I think solutions for case 1 and case 2 are necessarily at odds under > >> the current design,

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 12:07:22 +0700, John Naylor wrote: > Just in time ;-) The commit message should also have "reviewed by > Zhang Mingli" and "tested by Quan Zongliang", who shared results in a > thread for a withrawn CF entry with a similar idea but covering fewer > cases: Good call. Added and

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-04-07 Thread Nazir Bilal Yavuz
Hi Andrey, On Sun, 7 Apr 2024 at 08:29, Andrey M. Borodin wrote: > > > > > On 15 Mar 2024, at 17:12, Nazir Bilal Yavuz wrote: > > > > I did not have the time to check other things you mentioned but I > > tested the read performance. The table size is 5.5GB, I did 20 runs in > > total. > > Hi

Re: Table AM Interface Enhancements

2024-04-07 Thread Pavel Borisov
Hi, Alexander! On Sun, 7 Apr 2024 at 07:33, Alexander Korotkov wrote: > Hi, Pavel! > > On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov > wrote: > > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: > >> > >> On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: > >> > I don't like the idea

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 10:10 AM Tomas Vondra wrote: > > On 4/7/24 15:11, Melanie Plageman wrote: > > > Also, the iterators in the TableScanDescData might be something I > > could live with in the source code for a couple months before we make > > the rest of the changes in July+. But, adding them

Re: ❓ JSON Path Dot Precedence

2024-04-07 Thread Erik Wienhold
On 2024-04-07 18:13 +0200, David E. Wheeler wrote: > A question about the behavior of the JSON Path parser. The docs[1] > have this to say about numbers: > > > Numeric literals in SQL/JSON path expressions follow JavaScript > > rules, which are different from both SQL and JSON in some minor > >

Re: LogwrtResult contended spinlock

2024-04-07 Thread Jeff Davis
On Sun, 2024-04-07 at 14:19 +0200, Alvaro Herrera wrote: > I pushed the "copy" pointer now, except that I renamed it to > "insert", > which is what we call the operation being tracked.  I also added some > comments. The "copy" pointer, as I called it, is not the same as the "insert" pointer (as

Re: LogwrtResult contended spinlock

2024-04-07 Thread Alvaro Herrera
I pushed the "copy" pointer now, except that I renamed it to "insert", which is what we call the operation being tracked. I also added some comments. One perhaps significant change from what Bharath submitted is that we now use the return value from monotonic advance to return an updated value

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra wrote: > > On 4/7/24 06:17, Melanie Plageman wrote: > >> What bothers me on 0006-0008 is that the justification in the commit > >> messages is "future commit will do something". I think it's fine to have > >> a separate "prepareatory" patches (I really

Re: remaining sql/json patches

2024-04-07 Thread Amit Langote
On Sun, Apr 7, 2024 at 10:21 PM jian he wrote: > On Sun, Apr 7, 2024 at 12:30 PM jian he wrote: > > > > other than that, it looks good to me. > while looking at it again. > > + | NESTED path_opt Sconst > + COLUMNS '(' json_table_column_definition_list ')' > + { > + JsonTableColumn *n =

Re: MultiXact\SLRU buffers configuration

2024-04-07 Thread Alvaro Herrera
On 2024-Feb-03, Andrey M. Borodin wrote: > Here's the test draft. This test reliably reproduces sleep on CV when waiting > next multixact to be filled into "members" SLRU. > Cost of having this test: > 1. We need a new injection point type "wait" (in addition to "error" and > "notice"). It

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread Andres Freund
Hi, On 2024-04-01 11:53:28 +0900, Masahiko Sawada wrote: > On Fri, Mar 29, 2024 at 4:21 PM John Naylor wrote: > > I've marked it Ready for Committer. > > Thank you! I've attached the patch that I'm going to push tomorrow. Locally I ran a 32bit build with ubsan enabled (by accident actually),

Re: EXCLUDE COLLATE in CREATE/ALTER TABLE document

2024-04-07 Thread Tom Lane
"Daniel Verite" writes: > Also the use of "and/or" in the previous version conveys the fact > that operator class and ordering options are not mutually > exclusive. But when using "any of the following" in the new text, > doesn't it loose that meaning? Yeah; and/or is perfectly fine here and

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tomas Vondra
On 4/7/24 14:37, David Rowley wrote: > On Sun, 7 Apr 2024 at 22:05, John Naylor wrote: >> >> On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: >>> >> I'm planning on pushing these, pending a final look at 0002 and 0003 >>> on Sunday morning NZ time (UTC+12), likely in about 10 hours time. >>

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread John Naylor
On Sun, Apr 7, 2024 at 9:08 AM John Naylor wrote: > I've attached a mostly-polished update on runtime embeddable values, > storing up to 3 offsets in the child pointer (1 on 32-bit platforms). And...since there's a new bump context patch, I wanted to anticipate squeezing an update on top of

LDAP & Kerberos test woes

2024-04-07 Thread Heikki Linnakangas
While refactoring the Kerberos test module in preparation for adding libpq encryption negotiation tests [1], I noticed that if the test script die()s during setup, the whole test is marked as SKIPped rather than failed. The cleanup END section is missing this trick: ---

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Tomas Vondra
On 4/7/24 06:17, Melanie Plageman wrote: > On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: >> On 4/6/24 23:34, Melanie Plageman wrote: >>> ... I realized it makes more sense to add a FIXME (I used XXX. I'm not when to use what) with a link to the message where

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread Ranier Vilela
Em sáb., 6 de abr. de 2024 às 22:39, Andres Freund escreveu: > Hi, > > On 2024-04-07 00:45:31 +0200, Jelte Fennema-Nio wrote: > > On Sat, 6 Apr 2024 at 22:21, Andres Freund wrote: > > > The small regression for small results is still kinda visible, I > haven't yet > > > tested the patch

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Peter Geoghegan
On Sun, Apr 7, 2024 at 1:00 PM Alexander Lakhin wrote: > Please look at an assertion failure (reproduced starting from 5bf748b86), > triggered by the following query: > CREATE TABLE t (a int, b int); > CREATE INDEX t_idx ON t (a, b); > INSERT INTO t (a, b) SELECT g, g FROM generate_series(0, 999)

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 1:33 PM Nazir Bilal Yavuz wrote: > > Hi, > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > I had been planning to commit v14 this morning but got cold feet with > > the BMR-based interface. Heikki didn't like it much, and in the end, > > neither did I. I have

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra wrote: > > > > On 4/7/24 06:17, Melanie Plageman wrote: > > On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: > >> On 4/6/24 23:34, Melanie Plageman wrote: > >>> ... > > I realized it makes more sense to add a FIXME (I used XXX.

Re: remaining sql/json patches

2024-04-07 Thread jian he
On Sun, Apr 7, 2024 at 9:36 PM Amit Langote wrote: > > > 0002 needs an expanded commit message but I've run out of energy today. > some cosmetic issues in v51, 0002. in struct JsonTablePathScan, /* ERROR/EMPTY ON ERROR behavior */ bool errorOnError; the comments seem not right. I think

Re: pg_combinebackup --copy-file-range

2024-04-07 Thread Tomas Vondra
On 4/7/24 19:46, Tomas Vondra wrote: > On 4/5/24 21:43, Tomas Vondra wrote: >> Hi, >> >> ... >> >> 2) The prefetching is not a huge improvement, at least not for these >> three filesystems (btrfs, ext4, xfs). From the color scale it might seem >> like it helps, but those values are relative to the

Re: Trigger violates foreign key constraint

2024-04-07 Thread Tom Lane
Laurenz Albe writes: > Patch v3 is attached. I agree with documenting this hazard, but I think it'd be better to do so in the "Triggers" chapter. There is no hazard unless you are writing user-defined triggers, which is surely far fewer people than use foreign keys. So I suggest something like

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 22:05, Jelte Fennema-Nio wrote: > > On Sun, 7 Apr 2024 at 03:39, Andres Freund wrote: > > Changing the global vars to size_t seems mildly bogus to me. All it's > > achieving is to use slightly more memory. It also just seems unrelated to > > the > > change. > > I took a

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Tomas Vondra
On 4/7/24 15:11, Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra > wrote: >> >> On 4/7/24 06:17, Melanie Plageman wrote: What bothers me on 0006-0008 is that the justification in the commit messages is "future commit will do something". I think it's fine to have

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > On 2024-04-06 Sa 20:49, Andres Freund wrote: >> That's probably unnecessary optimization, but it seems a tad silly to read an >> entire, potentially sizable, file to just use the last 1k. Not sure if the >> way >> slurp_file() uses seek

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 18:28, Andres Freund wrote: > > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: >>> On 2024-04-06 Sa 20:49, Andres Freund wrote: >> That's probably unnecessary optimization, but it seems a tad silly to read

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Alexander Lakhin
Hello Peter, 03.04.2024 22:53, Peter Geoghegan wrote: On Mon, Apr 1, 2024 at 6:33 PM Peter Geoghegan wrote: Note: v18 doesn't have any adjustments to the costing, as originally planned. I'll probably need to post a revised patch with improved (or at least polished) costing in the next few

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 18:51:40 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 18:28, Andres Freund wrote: > > > > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > >>> On 2024-04-06 Sa 20:49, Andres Freund wrote: > >> > That's

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > I had been planning to commit v14 this morning but got cold feet with > the BMR-based interface. Heikki didn't like it much, and in the end, > neither did I. I have now removed it, and it seems much better. No > other significant

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-07 Thread Alexander Lakhin
Hi Alexander and Dmitry, 07.04.2024 01:22, Alexander Korotkov wrote: I've pushed 0001 and 0002. I didn't push 0003 for the following reasons. Please try the following (erroneous) query: CREATE TABLE t1(i int, t text) PARTITION BY LIST (t); CREATE TABLE t1pa PARTITION OF t1 FOR VALUES IN

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tomas Vondra
On 4/7/24 22:35, Tomas Vondra wrote: > On 4/7/24 14:37, David Rowley wrote: >> On Sun, 7 Apr 2024 at 22:05, John Naylor wrote: >>> >>> On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: >>> I'm planning on pushing these, pending a final look at 0002 and 0003 on Sunday morning NZ

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread John Naylor
On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: > I'm planning on pushing these, pending a final look at 0002 and 0003 > on Sunday morning NZ time (UTC+12), likely in about 10 hours time. +1 I haven't looked at v6, but I've tried using it in situ, and it seems to work as well as hoped:

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 03:39, Andres Freund wrote: > Changing the global vars to size_t seems mildly bogus to me. All it's > achieving is to use slightly more memory. It also just seems unrelated to the > change. I took a closer look at this. I agree that changing PqSendBufferSize to size_t is

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andrew Dunstan
On 2024-04-06 Sa 20:49, Andres Freund wrote: That's probably unnecessary optimization, but it seems a tad silly to read an entire, potentially sizable, file to just use the last 1k. Not sure if the way slurp_file() uses seek supports negative ofsets, the docs read to me like that may only be

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 10:42 AM Tomas Vondra wrote: > > > > On 4/7/24 16:24, Melanie Plageman wrote: > >>> Thanks! I thought about it a bit more, and I got worried about the > >>> > >>> Assert(scan->rs_empty_tuples_pending == 0); > >>> > >>> in heap_rescan() and heap_endscan(). > >>> > >>>

Re: CASE control block broken by a single line comment

2024-04-07 Thread Erik Wienhold
I wrote: > Attached v2 tries to do that. Hit send too soon. Attached now. -- Erik >From 23a20a410dc92946141b6c6fa5100473eac482cf Mon Sep 17 00:00:00 2001 From: Erik Wienhold Date: Sat, 6 Apr 2024 22:36:54 +0200 Subject: [PATCH v2] plpgsql: accept trailing line comment in CASE WHEN Expression

Re: CASE control block broken by a single line comment

2024-04-07 Thread Erik Wienhold
On 2024-04-07 06:33 +0200, Tom Lane wrote: > Erik Wienhold writes: > > I'm surprised that the comment is not skipped by the scanner at this > > point. Maybe because the parser just reads the raw expression between > > WHEN and THEN with plpgsql_append_source_text via read_sql_construct. > > >

Re: Table AM Interface Enhancements

2024-04-07 Thread Pavel Borisov
Hi, Alexander! On Sun, 7 Apr 2024 at 12:34, Pavel Borisov wrote: > Hi, Alexander! > > On Sun, 7 Apr 2024 at 07:33, Alexander Korotkov > wrote: > >> Hi, Pavel! >> >> On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov >> wrote: >> > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: >> >> >> >> On Tue,

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-04-07 Thread Tom Lane
Corey Huinker writes: >> I've also used the technique quite a lot, but only using the PK, >> didn't know about the ctid trick, so many thanks for documenting it. > tid-scans only became a thing a few versions ago (12?). Prior to that, PK > was the only way to go. I think we had TID scans for

Re: Security lessons from liblzma

2024-04-07 Thread Ranier Vilela
Hi, Sorry, without any connection with the technical part of the thread. But I couldn't help but record this, and congratulate Andres Freund, for the excellent work. It's not every day that a big press, in Brazil or around the world, publishes something related to technology people. Today I came

Re: Can't find not null constraint, but \d+ shows that

2024-04-07 Thread Tender Wang
It has been several days since the last email. Do you have any suggestions, please? What I'm concerned about is that adding a new AT_PASS is good fix? Is it a big try? More concerned is that it can cover all ALTER TABLE cases? Any thoughts. -- Tender Wang OpenPie: https://en.openpie.com/

Re: MultiXact\SLRU buffers configuration

2024-04-07 Thread Andrey M. Borodin
> On 6 Apr 2024, at 14:24, Andrey M. Borodin wrote: > > What do you think? OK, I'll follow this plan. As long as most parts of this thread were committed, I'll mark CF item as "committed". Thanks to everyone involved! See you in a followup thread about sleeping on CV. Best regards, Andrey

Re: remaining sql/json patches

2024-04-07 Thread jian he
On Sun, Apr 7, 2024 at 12:30 PM jian he wrote: > > other than that, it looks good to me. while looking at it again. + | NESTED path_opt Sconst + COLUMNS '(' json_table_column_definition_list ')' + { + JsonTableColumn *n = makeNode(JsonTableColumn); + + n->coltype = JTC_NESTED; + n->pathspec =

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Tomas Vondra
On 4/7/24 16:24, Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra > wrote: >> >> >> >> On 4/7/24 06:17, Melanie Plageman wrote: >>> On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: On 4/6/24 23:34, Melanie Plageman wrote: > ... >> >> I

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > > On 2024-04-06 Sa 20:49, Andres Freund wrote: > > >> That's probably unnecessary optimization, but it seems a tad silly to read > >> an > >> entire, potentially sizable, file to just use

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Thu, Apr 04, 2024 at 02:03:30PM +0300, Nazir Bilal Yavuz wrote: > > On Wed, 3 Apr 2024 at 23:44, Melanie Plageman > wrote: > > > > I don't see the point of BlockSampler_HasMore() anymore. I removed it in > > the attached and made BlockSampler_Next() return InvalidBlockNumber > > under the

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tomas Vondra
On 4/7/24 23:09, Andres Freund wrote: > Hi, > > On 2024-04-07 22:35:47 +0200, Tomas Vondra wrote: >> I haven't investigated, but I'd considering it works on 64-bit, I guess >> it's not considering alignment somewhere. I can dig more if needed. > > I think I may the problem: > > > #define

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > I've pushed 0001, 0002 and 0006. I briefly looked at 27bc1772fc81 and I don't think the state post this commit makes sense. Before this commit another block based AM could implement analyze without much code duplication. Now a large

Re: ❓ JSON Path Dot Precedence

2024-04-07 Thread David E. Wheeler
On Apr 7, 2024, at 15:46, Erik Wienhold wrote: > I guess jsonpath assumes that hex, octal, and binary literals are > integers. So there's no ambiguity about any fractional part that might > follow. Yeah, that’s what the comment in the flex file says:

Re: POC, WIP: OR-clause support for indexes

2024-04-07 Thread Alexander Korotkov
Hi! On Mon, Apr 1, 2024 at 9:38 AM Andrei Lepikhov wrote: > On 28/3/2024 16:54, Alexander Korotkov wrote: > > The current patch has a boolean guc enable_or_transformation. > > However, when we have just a few ORs to be transformated, then we > > should get less performance gain from the

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 00:55:42 +0200, Daniel Gustafsson wrote: > > On 8 Apr 2024, at 00:41, Tom Lane wrote: > > > > Tomas Vondra writes: > >> Yup, changing it to this: > > > >> #define KeeperBlock(set) ((BumpBlock *) ((char *) (set) + > >> MAXALIGN(sizeof(BumpContext > > > >> fixes the issue

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread John Naylor
On Mon, Apr 8, 2024 at 2:07 AM Andres Freund wrote: > > Looking at the code, the failure isn't suprising anymore: > chardata[MaxBlocktableEntrySize]; > BlocktableEntry *page = (BlocktableEntry *) data; > > 'char' doesn't enforce any alignment, but you're storing a

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Daniel Gustafsson
> On 8 Apr 2024, at 01:04, Andres Freund wrote: > What makes you think that's unrelated? To me that looks like the same issue? Nvm, I misread the assert, ETOOLITTLESLEEP. Sorry for the noise. -- Daniel Gustafsson

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 02:25:17 +0300, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I don't think the state post this >

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 11:07:58 +1200, Thomas Munro wrote: > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I > festooned it with warnings about why exactly it's inherently racy and > only for testing use. I still

Re: Popcount optimization using AVX512

2024-04-07 Thread Tom Lane
Nathan Bossart writes: > Here is what I have staged for commit, which I intend to do shortly. Today's Coverity run produced this warning, which seemingly was triggered by one of these commits, but I can't make much sense of it: *** CID 1596255: Uninitialized variables (UNINIT)

Re: Popcount optimization using AVX512

2024-04-07 Thread Nathan Bossart
On Sun, Apr 07, 2024 at 08:42:12PM -0400, Tom Lane wrote: > Today's Coverity run produced this warning, which seemingly was > triggered by one of these commits, but I can't make much sense > of it: > > *** CID 1596255: Uninitialized variables (UNINIT) >

Re: Weird test mixup

2024-04-07 Thread Michael Paquier
On Sat, Apr 06, 2024 at 10:34:46AM +0500, Andrey M. Borodin wrote: > I find name of the function "injection_points_local()" strange, > because there is no verb in the name. How about > "injection_points_set_local"? That makes sense. > I'm not sure if we should refactor anything here, but >

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 14:43:21 +1200, David Rowley wrote: > On Sun, 7 Apr 2024 at 19:30, Andres Freund wrote: > > Good call. Added and pushed. > > I understand you're already aware of the reference in the comment to > heapgetpage(), which no longer exists as of 44086b097. Yea,

Re: pub/sub - specifying optional parameters without values.

2024-04-07 Thread Tom Lane
Peter Smith writes: > Here is a similar update for another page: "55.4 Streaming Replication > Protocol" [0]. This patch was prompted by a review comment reply at > [1] (#2). > I've used text almost the same as the boilerplate text added by the > previous commit [2] Pushed, except I put it at

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Mon, 8 Apr 2024 at 00:01, Nazir Bilal Yavuz wrote: > > Hi, > > On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > > > I had been planning to commit v14 this morning but got cold feet with > > > the BMR-based

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 18:51, Daniel Gustafsson wrote: >> On 7 Apr 2024, at 18:28, Andres Freund wrote: >> I'm ok with printing path + some content or just the path. > > I think printing the last 512 bytes or so would be a good approach, I'll take > care of it later tonight. That would be a

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tom Lane
Tomas Vondra writes: > Yup, changing it to this: > #define KeeperBlock(set) ((BumpBlock *) ((char *) (set) + > MAXALIGN(sizeof(BumpContext > fixes the issue for me. Mamba is happy with that change, too. regards, tom lane

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread David Rowley
On Mon, 8 Apr 2024 at 09:09, Andres Freund wrote: > I suspect that KeeperBlock() isn't returning true, because IsKeeperBlock > misses > the MAXALIGN(). I think that about fits with: Thanks for investigating that. I've just pushed a fix for the macro and also adjusted a location which was

Re: Table AM Interface Enhancements

2024-04-07 Thread Alexander Korotkov
Hi, On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > I've pushed 0001, 0002 and 0006. > > I briefly looked at 27bc1772fc81 and I don't think the state post this commit > makes sense. Before this commit another block based AM could

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-04-07 Thread Andres Freund
Hi, On 2024-03-15 18:36:55 +0900, Sutou Kouhei wrote: > +warning_level = get_option('warning_level') > +# See https://mesonbuild.com/Builtin-options.html#details-for-warning_level > for > +# warning_level values. > +if warning_level == '1' > + common_builtin_flags += ['-Wall', '/W2'] > +elif

Re: Experiments with Postgres and SSL

2024-04-07 Thread Heikki Linnakangas
On 08/04/2024 04:28, Tom Lane wrote: Heikki Linnakangas writes: Committed this. Thank you to everyone involved! Looks like perlcritic isn't too happy with the test code: koel and crake say ./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored -

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Peter Geoghegan
On Sun, Apr 7, 2024 at 9:57 PM Peter Geoghegan wrote: > On Sun, Apr 7, 2024 at 9:50 PM Tom Lane wrote: > > those first two Asserts are redundant with the "if" as well. > > I'll get rid of those other two assertions as well, then. Done that way. -- Peter Geoghegan

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 19:30, Andres Freund wrote: > Good call. Added and pushed. I understand you're already aware of the reference in the comment to heapgetpage(), which no longer exists as of 44086b097. Melanie and I had discussed the heap_prepare_pagescan() name while I was reviewing that

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Thomas Munro
On Mon, Apr 8, 2024 at 10:26 AM Melanie Plageman wrote: > On Sun, Apr 07, 2024 at 03:00:00PM -0700, Andres Freund wrote: > > On 2024-04-07 16:59:26 -0400, Melanie Plageman wrote: > > > From 862b7ac81cdafcda7b525e02721da14e46265509 Mon Sep 17 00:00:00 2001 > > > From: Melanie Plageman > > > Date:

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 15:43:12 +1200, David Rowley wrote: > On Mon, 8 Apr 2024 at 15:13, Andres Freund wrote: > > Off-list Melanie suggested heap_page_collect_visible_tuples(). Given the > > separate callsites (making long names annoying) and the fact that it's > > really > > specific to one

Re: Allow non-superuser to cancel superuser tasks.

2024-04-07 Thread Michael Paquier
On Fri, Apr 05, 2024 at 08:07:51PM -0500, Nathan Bossart wrote: > On Sat, Apr 06, 2024 at 08:56:04AM +0900, Michael Paquier wrote: >> There is pg_read_all_stats as well, so I don't see a big issue in >> requiring to be a member of this role as well for the sake of what's >> proposing here. > >

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Thomas Munro
On Mon, Apr 8, 2024 at 12:10 PM Andres Freund wrote: > On 2024-04-07 11:07:58 +1200, Thomas Munro wrote: > > I thought of a better name for the bufmgr.c function though: > > InvalidateUnpinnedBuffer(). That name seemed better to me after I > > festooned it with warnings about why exactly it's

Replace FunctionCall2Coll with FunctionCallInvoke

2024-04-07 Thread Andy Fan
Hello, During the review of using extended statistics to improve join estimates [1], I found some code level optimization opportunities which apply to existing code as well. To make the discussion easier, open this new thread. I don't know how to name the thread name, just use the patch 1 for

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-07 Thread Alexander Korotkov
Hi, Alexander! On Sun, Apr 7, 2024 at 10:00 PM Alexander Lakhin wrote: > 07.04.2024 01:22, Alexander Korotkov wrote: > > I've pushed 0001 and 0002. I didn't push 0003 for the following reasons. > > Please try the following (erroneous) query: > CREATE TABLE t1(i int, t text) PARTITION BY LIST

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-07 Thread Michael Paquier
On Sat, Apr 06, 2024 at 07:47:43PM +0200, Daniel Gustafsson wrote: > My apologies, I thought I did but clearly failed. My point was that this is a > special/corner case where we try to find one of two different libraries (with > different ideas about backwards compatability etc) for supporting a

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-07 Thread Michael Paquier
On Thu, Apr 04, 2024 at 03:10:27PM +1300, David Rowley wrote: > Someone asked me about this, so thought it might be useful to post here. I've received the same question. > To express this as UTC, It's: > > postgres=# select '2024-04-08 00:00-12:00' at time zone 'UTC'; > timezone >

Re: POC, WIP: OR-clause support for indexes

2024-04-07 Thread Justin Pryzby
On Mon, Apr 08, 2024 at 01:34:37AM +0300, Alexander Korotkov wrote: > Hi! > > On Mon, Apr 1, 2024 at 9:38 AM Andrei Lepikhov > wrote: > > On 28/3/2024 16:54, Alexander Korotkov wrote: > > > The current patch has a boolean guc enable_or_transformation. > > > However, when we have just a few ORs

Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Tom Lane
Today's Coverity run produced this: /srv/coverity/git/pgsql-git/postgresql/src/include/lib/simplehash.h: 1138 in bh_nodeidx_stat() 1132 avg_collisions = 0; 1133 } 1134 1135 sh_log("size: " UINT64_FORMAT ", members: %u, filled: %f, total chain: %u, max chain: %u,

Re: Experiments with Postgres and SSL

2024-04-07 Thread Tom Lane
Heikki Linnakangas writes: > Committed this. Thank you to everyone involved! Looks like perlcritic isn't too happy with the test code: koel and crake say ./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored - chmod at line 138, column 2. See

Re: Popcount optimization using AVX512

2024-04-07 Thread Nathan Bossart
On Sun, Apr 07, 2024 at 08:23:32PM -0500, Nathan Bossart wrote: > The Intel documentation for _mm256_undefined_si256() [0] > indicates that it is intended to return "undefined elements," so it seems > like the use of an uninitialized variable might be intentional. See also

Re: Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Tom Lane
Andres Freund writes: > On 2024-04-07 21:03:53 -0400, Tom Lane wrote: >> I realize this function is only debug support, but wouldn't it >> be appropriate to pfree(collisions) before exiting? > It indeed looks like that memory should be freed. Very odd that coverity > started to complain about

Re: remaining sql/json patches

2024-04-07 Thread jian he
On Mon, Apr 8, 2024 at 12:34 AM jian he wrote: > > On Sun, Apr 7, 2024 at 9:36 PM Amit Langote wrote: > > 0002 needs an expanded commit message but I've run out of energy today. > > > +/* + * Fetch next row from a JsonTablePlan's path evaluation result and from + * any child nested path(s). + *

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread David Rowley
On Mon, 8 Apr 2024 at 15:13, Andres Freund wrote: > I kinda don't like heap_prepare_pagescan(), but heapgetpage() is worse. And I > don't have a great alternative suggestion. It came around from having nothing better. I was keen not to have the name indicate it was only for checking visibility

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread David Rowley
On Mon, 8 Apr 2024 at 16:08, Andres Freund wrote: > > On 2024-04-08 15:43:12 +1200, David Rowley wrote: > > I understand wanting to avoid the long name. I'd rather stay clear of > > "visible", but don't feel as strongly about this as it's static. > > I think visible would be ok, the

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 22:05, John Naylor wrote: > > On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: > > > I'm planning on pushing these, pending a final look at 0002 and 0003 > > on Sunday morning NZ time (UTC+12), likely in about 10 hours time. > > +1 I've now pushed all 3 patches. Thank

❓ JSON Path Dot Precedence

2024-04-07 Thread David E. Wheeler
Hello Hackers, A question about the behavior of the JSON Path parser. The docs[1] have this to say about numbers: > Numeric literals in SQL/JSON path expressions follow JavaScript rules, which > are different from both SQL and JSON in some minor details. For example, > SQL/JSON path allows

Re: LDAP & Kerberos test woes

2024-04-07 Thread Heikki Linnakangas
On 07/04/2024 13:19, Heikki Linnakangas wrote: 1st patch fixes the LDAP setup tests, and 2nd patch fixes the error handling in the END blocks. Committed and backpatched these test fixes. -- Heikki Linnakangas Neon (https://neon.tech)

Re: MultiXact\SLRU buffers configuration

2024-04-07 Thread Andrey M. Borodin
> On 7 Apr 2024, at 21:41, Alvaro Herrera wrote: > > Well, it would be nice to have *some* test, but as you say it is way > more complex than the thing being tested, and it zooms in on the > functioning of the multixact creation in insane quantum-physics ways ... > to the point that you can

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 3:57 PM Melanie Plageman wrote: > > On Thu, Apr 04, 2024 at 02:03:30PM +0300, Nazir Bilal Yavuz wrote: > > > > On Wed, 3 Apr 2024 at 23:44, Melanie Plageman > > wrote: > > > > > > I don't see the point of BlockSampler_HasMore() anymore. I removed it in > > > the attached

Re: Streaming read-ready sequential scan code

2024-04-07 Thread Thomas Munro
On Sun, Apr 7, 2024 at 1:34 PM Melanie Plageman wrote: > Attached v13 0001 is your fix and 0002 is a new version of the > sequential scan streaming read user. Off-list Andres mentioned that I > really ought to separate the parallel and serial sequential scan users > into two different callbacks.

Re: Streaming read-ready sequential scan code

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 09:36:59 +1200, Thomas Munro wrote: > I've been on the fence about that flag for sequential scan... Some > days I want to consider changing to READ_STREAM_DEFAULT and relying on > our "anti-heuristics" to suppress advice, which would work out the > same in most cases but might

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Melanie Plageman
On Sat, Apr 6, 2024 at 7:08 PM Thomas Munro wrote: > > On second thoughts, I think the original "invalidate" terminology was > fine, no need to invent a new term. > > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Thomas Munro
On Mon, Apr 8, 2024 at 10:26 AM Melanie Plageman wrote: > On Sun, Apr 07, 2024 at 03:00:00PM -0700, Andres Freund wrote: > > > src/backend/commands/analyze.c | 89 ++ > > > 1 file changed, 26 insertions(+), 63 deletions(-) > > > > That's a very nice demonstration

Re: Popcount optimization using AVX512

2024-04-07 Thread Tom Lane
Nathan Bossart writes: > On Sun, Apr 07, 2024 at 08:23:32PM -0500, Nathan Bossart wrote: >> The Intel documentation for _mm256_undefined_si256() [0] >> indicates that it is intended to return "undefined elements," so it seems >> like the use of an uninitialized variable might be intentional. >

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Peter Geoghegan
On Sun, Apr 7, 2024 at 9:50 PM Tom Lane wrote: > If you're doing that, then surely > > if (j != (BTEqualStrategyNumber - 1) || > !(xform[j].skey->sk_flags & SK_SEARCHARRAY)) > { > ... > } >

  1   2   >