Doc Rework: Section 9.16.13 SQL/JSON Query Functions

2024-06-24 Thread David G. Johnston
Hey! Lots of SQL/JSON threads going about. This one is less about technical correctness and more about usability of the documentation. Though in writing this I am finding some things that aren't quite clear. I'm going to come back with those on a follow-on post once I get a chance to make my se

Re: Add pg_get_acl() function get the ACL for a database object

2024-06-24 Thread Michael Paquier
On Tue, Jun 25, 2024 at 08:06:41AM +0200, Joel Jacobson wrote: > Not sure if I see how to implement it for pg_get_acl() though. > > I've had a look at how pg_describe_object() works for this case: > > SELECT pg_describe_object(0,'t'::regclass::oid,0); > ERROR: unsupported object class: 0 > > I

Re: New standby_slot_names GUC in PG 17

2024-06-24 Thread Bertrand Drouvot
Hi, On Tue, Jun 25, 2024 at 10:24:41AM +0530, Amit Kapila wrote: > On Tue, Jun 25, 2024 at 8:20 AM Masahiko Sawada wrote: > > > > On Tue, Jun 25, 2024 at 11:21 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > I agree the current name seems too generic and the suggested ' > > > synchronized_standb

RE: Partial aggregates pushdown

2024-06-24 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Jelte, hackers. Thank you for explanations. Actually, I have other tasks about "PARTIAL_AGGREAGATE" keyword to respond Requirement1 and Requirement2 in the following mail. https://www.postgresql.org/message-id/TYAPR01MB3088755F2281D41F5EEF06D495F92%40TYAPR01MB3088.jpnprd01.prod.outlook.com Af

Re: Pgoutput not capturing the generated columns

2024-06-24 Thread Peter Smith
Here are some review comments for the patch v10-0002. == Commit Message 1. Note that we don't copy columns when the subscriber-side column is also generated. Those will be filled as normal with the subscriber-side computed or default data. ~ Now this patch also introduced some errors etc, s

Re: speed up a logical replica setup

2024-06-24 Thread Amit Kapila
On Tue, Jun 25, 2024 at 3:38 AM Noah Misch wrote: > > On Mon, Jun 24, 2024 at 05:20:21PM +0530, Amit Kapila wrote: > > On Sun, Jun 23, 2024 at 11:52 AM Noah Misch wrote: > > > > > > > +static void > > > > +create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo) > > > > +{ > > > > > > > +

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Tue, Jun 25, 2024, at 07:11, Michael Paquier wrote: > On Tue, Jun 25, 2024 at 12:20:20AM +0200, Joel Jacobson wrote: >> Thanks, much better. New version attached. > > + The PostgreSQL documentation, and code, > refers > + to the specifications within the ACL as "privileges". This has

Re: Add pg_get_acl() function get the ACL for a database object

2024-06-24 Thread Joel Jacobson
On Tue, Jun 25, 2024, at 03:57, Michael Paquier wrote: > On Tue, Jun 25, 2024 at 01:21:14AM +0200, Joel Jacobson wrote: >> Good idea, I've started a separate thread for this: >> >> https://postgr.es/m/9253b872-dbb1-42a6-a79e-b1e96effc857%40app.fastmail.com >> >> This patch now assumes ACL will be

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Michael Paquier
On Tue, Jun 25, 2024 at 12:20:20AM +0200, Joel Jacobson wrote: > Thanks, much better. New version attached. + The PostgreSQL documentation, and code, refers + to the specifications within the ACL as "privileges". This has the same + meaning as "permissions" on the linked page. Gen

Re: long-standing data loss bug in initial sync of logical replication

2024-06-24 Thread Amit Kapila
On Mon, Jun 24, 2024 at 8:06 PM Tomas Vondra wrote: > > On 6/24/24 12:54, Amit Kapila wrote: > > ... > >> > I'm not sure there are any cases where using SRE instead of AE would > cause > problems for logical decoding, but it seems very hard to prove. I'd be > very > surp

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-06-24 Thread Alexander Lakhin
24.06.2024 01:59, Jelte Fennema-Nio wrote: On Sat, 22 Jun 2024 at 17:00, Alexander Lakhin wrote: @@ -2775,6 +2775,7 @@ SET LOCAL statement_timeout = '10ms'; select count(*) from ft1 CROSS JOIN ft2 CROSS JOIN ft4 CROSS JOIN ft5; -- this takes very long ERROR: canceling statement due t

Re: New standby_slot_names GUC in PG 17

2024-06-24 Thread Amit Kapila
On Tue, Jun 25, 2024 at 8:20 AM Masahiko Sawada wrote: > > On Tue, Jun 25, 2024 at 11:21 AM Zhijie Hou (Fujitsu) > wrote: > > > > I agree the current name seems too generic and the suggested ' > > synchronized_standby_slots ' > > is better than the current one. > > > > Some other ideas could be:

Re: sql/json miscellaneous issue

2024-06-24 Thread jian he
On Tue, Jun 25, 2024 at 11:23 AM Amit Langote wrote: > > On Tue, Jun 25, 2024 at 12:18 PM jian he wrote: > > On Mon, Jun 24, 2024 at 7:46 PM Amit Langote > > wrote: > > > On Mon, Jun 24, 2024 at 7:04 PM jian he > > > wrote: > > > > > > > > hi. > > > > the following two queries should return t

Re: [PATCH] Fix type redefinition build errors with macOS SDK 15.0

2024-06-24 Thread Stan Hu
Thanks, Tom and Michael. I've submitted a bug report via Apple's Feedback Assistant. It's filed under FB14047412. If anyone happens to know the right person at Apple to look at this, please direct them there. On Mon, Jun 24, 2024 at 7:15 PM Tom Lane wrote: > > Michael Paquier writes: > > Ugh.

Re: Patch bug: Fix jsonpath .* on Arrays

2024-06-24 Thread Степан Неретин
  >Вторник, 25 июня 2024, 11:17 +07:00 от David E. Wheeler >: >  >On Jun 7, 2024, at 10:23, David E. Wheeler < da...@justatheory.com > wrote: >  >> Rebased and moved the new tests to the end of the file. >Bah, sorry, that was the previous patch. Here’s v3. > >D >      Hi! Looks good to me, bu

Re: sql/json miscellaneous issue

2024-06-24 Thread Amit Langote
On Tue, Jun 25, 2024 at 12:18 PM jian he wrote: > On Mon, Jun 24, 2024 at 7:46 PM Amit Langote wrote: > > On Mon, Jun 24, 2024 at 7:04 PM jian he wrote: > > > > > > hi. > > > the following two queries should return the same result? > > > > > > SELECT * FROM JSON_query (jsonb 'null', '$' returnin

Re: sql/json miscellaneous issue

2024-06-24 Thread jian he
On Mon, Jun 24, 2024 at 7:46 PM Amit Langote wrote: > > Hi, > > On Mon, Jun 24, 2024 at 7:04 PM jian he wrote: > > > > hi. > > the following two queries should return the same result? > > > > SELECT * FROM JSON_query (jsonb 'null', '$' returning jsonb); > > SELECT * FROM JSON_value (jsonb 'null',

Re: Support "Right Semi Join" plan shapes

2024-06-24 Thread wenhui qiu
Hi Japin Li Thank you for your reviewing ,This way the notes are more accurate and complete. Thanks also to the author for updating the patch ,I also tested the new patch ,It looks good to me Regrads Japin Li 于2024年6月25日周二 08:51写道: > On Mon, 24 Jun 2024 at 17:59, Richard Guo wrote: > > T

Re: New standby_slot_names GUC in PG 17

2024-06-24 Thread Masahiko Sawada
On Tue, Jun 25, 2024 at 11:21 AM Zhijie Hou (Fujitsu) wrote: > > On Saturday, June 22, 2024 5:47 PM Amit Kapila > wrote: > > > > On Sat, Jun 22, 2024 at 1:49 AM Nathan Bossart > > wrote: > > > > > > On Fri, Jun 21, 2024 at 03:50:00PM -0400, Tom Lane wrote: > > > > Allow specification of phy

Re: Injection point locking

2024-06-24 Thread Michael Paquier
On Mon, Jun 24, 2024 at 11:03:09AM -0400, Tom Lane wrote: > Given your point that the existing locking is just a fig leaf > anyway, maybe we could simply not have any? Then it's on the > test author to be sure that the injection point won't be > getting invoked when it's about to be removed. That

Re: Injection point locking

2024-06-24 Thread Noah Misch
On Mon, Jun 24, 2024 at 11:03:09AM -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > ... I can't do that, because InjectionPointRun() requires a PGPROC > > entry, because it uses an LWLock. That also makes it impossible to use > > injection points in the postmaster. Any chance we could all

RE: New standby_slot_names GUC in PG 17

2024-06-24 Thread Zhijie Hou (Fujitsu)
On Saturday, June 22, 2024 5:47 PM Amit Kapila wrote: > > On Sat, Jun 22, 2024 at 1:49 AM Nathan Bossart > wrote: > > > > On Fri, Jun 21, 2024 at 03:50:00PM -0400, Tom Lane wrote: > > > Allow specification of physical standbys that must be > > > synchronized before they are visible to su

Re: [PATCH] Fix type redefinition build errors with macOS SDK 15.0

2024-06-24 Thread Tom Lane
Michael Paquier writes: > Ugh. Which means that you are testing macOS Sequoia still in beta > phase? Thanks for the report. > Perhaps we should wait for the actual release before seeing if this is > still an issue and see if this is still a problem? Tom is a heavy > macOS user, I'm still under

Re: Injection point locking

2024-06-24 Thread Michael Paquier
On Mon, Jun 24, 2024 at 01:29:38PM +0300, Heikki Linnakangas wrote: > InjectionPointRun() acquires InjectionPointLock, looks up the hash entry, > and releases the lock: > > > LWLockAcquire(InjectionPointLock, LW_SHARED); > > entry_by_name = (InjectionPointEntry *) > > hash_sear

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Noah Misch
On Mon, Jun 24, 2024 at 09:49:53PM -0400, Peter Geoghegan wrote: > On Mon, Jun 24, 2024 at 9:30 PM Noah Misch wrote: > > On Mon, Jun 24, 2024 at 03:23:39PM -0400, Melanie Plageman wrote: > > > Right now, in master, we do use a single horizon when determining what > > > is pruned -- that from Globa

Re: [PATCH] Fix type redefinition build errors with macOS SDK 15.0

2024-06-24 Thread Michael Paquier
On Mon, Jun 24, 2024 at 02:58:47PM -0700, Stan Hu wrote: > Prior to macOS SDK 15, there were include guards in > $SDK_ROOT/usr/include/xlocale/_regex.h: > > #ifndef _REGEX_H_ > #include <_regex.h> > #endif // _REGEX_H_ > #include <_xlocale.h> Ugh. Which means that you are testing macOS S

Re: Add pg_get_acl() function get the ACL for a database object

2024-06-24 Thread Michael Paquier
On Tue, Jun 25, 2024 at 01:21:14AM +0200, Joel Jacobson wrote: > Good idea, I've started a separate thread for this: > > https://postgr.es/m/9253b872-dbb1-42a6-a79e-b1e96effc857%40app.fastmail.com > > This patch now assumes ACL will be supported. Thanks for doing that! That helps in making revi

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 9:30 PM Noah Misch wrote: > On Mon, Jun 24, 2024 at 03:23:39PM -0400, Melanie Plageman wrote: > > Right now, in master, we do use a single horizon when determining what > > is pruned -- that from GlobalVisState. OldestXmin is only used for > > freezing and full page visibil

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Noah Misch
On Mon, Jun 24, 2024 at 04:51:24PM -0400, Peter Geoghegan wrote: > On Mon, Jun 24, 2024 at 4:36 PM Robert Haas wrote: > > I'm not sure I understand. The most important thing here is fixing the > > bug. But if we have a choice of how to fix the bug, I'd prefer to do > > it by having the pruning cod

Re: Issues with ON CONFLICT UPDATE and REINDEX CONCURRENTLY

2024-06-24 Thread Michael Paquier
On Fri, Jun 21, 2024 at 11:31:21AM +0200, Michail Nikolaev wrote: > Yes, I also have tried that approach, but it doesn't work, unfortunately. > You may fail test increasing number of connections: > > '--no-vacuum --client=10 -j 2 --transactions=1000', > > The source of the issue is not the swap o

Re: Track the amount of time waiting due to cost_delay

2024-06-24 Thread Imseih (AWS), Sami
>> 2. the leader being interrupted while waiting is also already happening on >> master >> due to the pgstat_progress_parallel_incr_param() calls in >> parallel_vacuum_process_one_index() (that have been added in >> 46ebdfe164). It has been the case "only" 36 times during my test case. 46ebdfe164

Re: 001_rep_changes.pl fails due to publisher stuck on shutdown

2024-06-24 Thread Kyotaro Horiguchi
At Fri, 21 Jun 2024 11:48:22 +0530, Amit Kapila wrote in > On Wed, Jun 19, 2024 at 10:44 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Horiguchi-san, > > > > Thanks for sharing the patch! I agree this approach (ensure WAL records are > > flushed) > > Is more proper than others. > > > > I ha

Re: Support "Right Semi Join" plan shapes

2024-06-24 Thread Japin Li
On Mon, 24 Jun 2024 at 17:59, Richard Guo wrote: > Thank you for reviewing. > > On Mon, Jun 24, 2024 at 1:27 PM Li Japin wrote: >> + /* >> +* For now we do not support RIGHT_SEMI join in mergejoin or nestloop >> +* join. >> +*/ >> + if (jointype == JOIN_RIGHT_S

Re: Add pg_get_acl() function get the ACL for a database object

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 01:46, Michael Paquier wrote: > Rather unrelated to this patch, still this patch makes the situation > more complicated in the docs, but wouldn't it be better to add ACL as > a term in acronyms.sql, and reuse it here? It would be a doc-only > patch that applies on top of t

Re: Apparent bug in WAL summarizer process (hung state)

2024-06-24 Thread Michael Paquier
On Mon, Jun 24, 2024 at 02:56:00PM -0300, Israel Barth Rubio wrote: > I've been playing a bit with the incremental backup feature which might > come as > part of the 17 release, and I think I hit a possible bug in the WAL > summarizer > process. Thanks for testing new features and for this report!

Re: improve predefined roles documentation

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 2:53 PM Nathan Bossart wrote: > On Mon, Jun 24, 2024 at 02:44:33PM -0400, Robert Haas wrote: > > > I don't know what to do about pg_database_owner. I almost wonder if > > that should be moved out of the table and documented as a special > > case. Or maybe some more wordsmi

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 23:15, David G. Johnston wrote: > I really dislike "For avoidance of doubt and clarity" - and in terms of > being equivalent the following seems like a more accurate description > of reality. > > The PostgreSQL documentation, and code, refers to the specifications > with

[PATCH v3] Fix type redefinition build errors with macOS SDK 15.0

2024-06-24 Thread Stan Hu
Prior to macOS SDK 15, there were include guards in $SDK_ROOT/usr/include/xlocale/_regex.h: #ifndef _REGEX_H_ #include <_regex.h> #endif // _REGEX_H_ #include <_xlocale.h> In macOS SDK 15, these include guards are gone: #include <_regex.h> #include <_xlocale.h> Because _REGEX_H_ was

Re: speed up a logical replica setup

2024-06-24 Thread Noah Misch
On Mon, Jun 24, 2024 at 05:20:21PM +0530, Amit Kapila wrote: > On Sun, Jun 23, 2024 at 11:52 AM Noah Misch wrote: > > > > > +static void > > > +create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo) > > > +{ > > > > > + appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", >

[PATCH] Fix type redefinition build errors with macOS SDK 15.0

2024-06-24 Thread Stan Hu
Prior to macOS SDK 15, there were include guards in $SDK_ROOT/usr/include/xlocale/_regex.h: #ifndef _REGEX_H_ #include <_regex.h> #endif // _REGEX_H_ #include <_xlocale.h> In macOS SDK 15, these include guards are gone: #include <_regex.h> #include <_xlocale.h> Because _REGEX_H_ was

Re: improve predefined roles documentation

2024-06-24 Thread Nathan Bossart
On Mon, Jun 24, 2024 at 02:44:33PM -0400, Robert Haas wrote: > I think the first two cases could be made more like each other by > changing the varlistentires that are just about one setting to use the > second format instead of the first, e.g. pg_checkpoint allows > executing the CHECKPOINT comman

[PATCH] Fix type redefinition build errors with macOS SDK 15.0

2024-06-24 Thread Stan Hu
Prior to macOS SDK 15, there were include guards in $SDK_ROOT/usr/include/xlocale/_regex.h: #ifndef _REGEX_H_ #include <_regex.h> #endif // _REGEX_H_ #include <_xlocale.h> In macOS SDK 15.5, these include guards are gone: #include <_regex.h> #include <_xlocale.h> Because _REGEX_H_ w

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 22:42, David E. Wheeler wrote: > >> BINDIR > >> DOCDIR > >> HTMLDIR > >> PKGINCLUDEDIR > >> LOCALEDIR > >> MANDIR > >> > >> I can imagine an extension wanting or needing to use any and all of these. > > > > Are these really all relevant to backend code? > > Oh I think so. Es

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 17:17, Jelte Fennema-Nio wrote: > If you want to only change $libdir during CREATE EXTENSION (or ALTER > EXTENSION UPDATE), then why not just change it there. And really you'd > only want to change it when creating an extension from which the > control file is coming from exte

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 18:11, Nathan Bossart wrote: > At first glance, the general idea seems reasonable to me. I'm wondering > whether there is a requirement for this directory to be prepended or if it > could be appended to the end. That way, the existing ones would take > priority, which migh

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Thu, 11 Apr 2024 at 19:52, David E. Wheeler wrote: > I realize this probably isn’t going to happen for 17, given the freeze, but I > would very much welcome feedback and pointers to address concerns about > providing a second directory for extensions and DSOs. Quite a few people have > talke

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 1:49 PM Joel Jacobson wrote: > How about? > > + > + The linked page uses "permissions" while we consistently use the > synonym > + "privileges", to describe the contents of the list. For avoidance of > + doubt and clarity, these two terms are equivalent

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:51 PM Peter Geoghegan wrote: > > On Mon, Jun 24, 2024 at 4:36 PM Robert Haas wrote: > > I thought the idea was that the GlobalVisTest stuff would force a > > recalculation now and then, but maybe it doesn't actually do that? > > It definitely can do that. Just not in a w

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Nikolay Shaplov
Hi! Let me join the review process. I am no expert in execution plans, so there would not be much help in doing even better optimization. But I can read the code, as a person who is not familiar with this area and help making it clear even to a person like me. So, I am reading v25-0001-Transf

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 4:36 PM Robert Haas wrote: > I'm not sure I understand. The most important thing here is fixing the > bug. But if we have a choice of how to fix the bug, I'd prefer to do > it by having the pruning code test one horizon that is always correct, > rather than (as I think the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:42 PM Peter Geoghegan wrote: > > On Mon, Jun 24, 2024 at 3:36 PM Melanie Plageman > wrote: > > One thing I don't understand is why it is okay to freeze the xmax of a > > dead tuple just because it is from an aborted update. > > We don't do that with XID-based xmaxs. Thou

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 21:51, David G. Johnston wrote: > On Mon, Jun 24, 2024 at 12:46 PM Joel Jacobson wrote: >> On Mon, Jun 24, 2024, at 18:02, David G. Johnston wrote: >> >> > The page we link to uses "permissions" while we consistently use >> > "privileges" to describe the contents of the

Re: Partial aggregates pushdown

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 15:03, fujii.y...@df.mitsubishielectric.co.jp wrote: > I see. I maybe got your proposal. > Refer to your proposal, for avg(int8), > I create a new native type like state_int8_avg > with the new typsend/typreceive functions > and use them to transmit the state value, right?

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 3:36 PM Melanie Plageman wrote: > One thing I don't understand is why it is okay to freeze the xmax of a > dead tuple just because it is from an aborted update. We don't do that with XID-based xmaxs. Though perhaps we should, since we'll already prune-away the successor tu

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 4:28 PM, Robert Haas wrote: > As long as the GUC is superuser-only, I'm not sure what else there is > to do here. The only question is whether there's some reason to > disallow this even from the superuser, but I'm not quite seeing such a > reason. I can switch it back from r

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 3:23 PM Melanie Plageman wrote: > Are you more concerned about having a single horizon for pruning or > about having a horizon that does not move backwards after being > established at the beginning of vacuuming the relation? I'm not sure I understand. The most important t

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Nikolay Shaplov
Hi! Let me join the review process. I am no expert in execution plans, so there would not be much help in doing even better optimization. But I can read the code, as a person who is not familiar with this area and help making it clear even to a person like me. So, I am reading v25-0001-Transf

Re: Direct SSL connection and ALPN loose ends

2024-06-24 Thread Heikki Linnakangas
On 21/06/2024 02:32, Jacob Champion wrote: On Thu, Jun 20, 2024 at 4:13 PM Heikki Linnakangas wrote: By "negotiation" I mean the server's response to the startup packet. I.e. "supported"/"not supported"/"error". Ok, I'm still a little confused, probably a terminology issue. The server doesn't

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 3:37 PM David E. Wheeler wrote: > I guess the question then is what security controls are appropriate for this > feature, which after all tells the postmaster what directories to read files > from. It feels a little outside the scope of a regular user to even be aware >

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread David E. Wheeler
On Jun 19, 2024, at 05:41, Peter Eisentraut wrote: > This is probably a bit confusing. This might as well mean client application > code against libpq. Better something like "server plugin code that uses the > PostgreSQL server APIs". That works. > But now we're talking about API. That mig

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 12:46 PM Joel Jacobson wrote: > On Mon, Jun 24, 2024, at 18:02, David G. Johnston wrote: > > > The page we link to uses "permissions" while we consistently use > > "privileges" to describe the contents of the list. This seems like an > > obvious synonym, but as the point

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread David E . Wheeler
On Jun 24, 2024, at 14:51, Robert Haas wrote: > I suppose that it's true that we try to avoid gratuitous breakage, but > I feel like it would be weird to document that. I see how that can seem weird to a committer deeply familiar with the development process and how things happen. But people ou

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread David E. Wheeler
On Jun 19, 2024, at 05:42, Peter Eisentraut wrote: >>> https://postgr.es/m/CAH2-Wzm-W6hSn71sUkz0Rem=qdeu7tnfmc7_jg2djrlfef_...@mail.gmail.com >>> >>> Theoretically anybody can do this themselves. In practice they don't. >>> So something as simple as providing automated reports about ABI >>> chan

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 18:02, David G. Johnston wrote: > On Mon, Jun 24, 2024 at 8:44 AM Nathan Bossart > wrote: >> I think we could omit "i.e. privileges list." >> > > Agreed. Between the docs and code we say "privileges list" once and > that refers to the dumputIls description of the argum

Unusually long checkpoint time on version 16, and 17beta1 running in a docker container

2024-06-24 Thread Dave Cramer
Greetings, While testing pgjdbc I noticed the following pgdb-1 | Will execute command on database postgres: pgdb-1 | SELECT pg_drop_replication_slot(slot_name) FROM pg_replication_slots WHERE slot_name = 'replica_one'; pgdb-1 | DROP USER IF EXISTS replica_one; pgdb-1 |

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 1:53 PM, Robert Haas wrote: > Is "tighten up what the superuser can do" on our list of objectives? > Personally, I think we should be focusing mostly, and maybe entirely, > on letting non-superusers do more things, with appropriate security > controls. The superuser can ultima

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman wrote: > > If vacuum fails to remove a tuple with xmax older than > VacuumCutoffs->OldestXmin and younger than > GlobalVisState->maybe_needed, it will ERROR out when determining > whether or not to freeze the tuple with "cannot freeze committed > xm

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 3:23 PM Melanie Plageman wrote: > I had always thought it was because the vacuuming backend's > GlobalVisState will get updated periodically throughout vacuum and so, > assuming the oldest running transaction changes, our horizon for > vacuum would change. I believe that i

Re: PostgreSQL does not compile on macOS SDK 15.0

2024-06-24 Thread Stan Hu
It appears in macOS SDK 14.5, there were include guards in $SDK_ROOT/usr/include/xlocale/_regex.h: #ifndef _XLOCALE__REGEX_H_ #define _XLOCALE__REGEX_H_ #ifndef _REGEX_H_ #include <_regex.h> #endif // _REGEX_H_ #include <_xlocale.h> In macOS SDK 15.5, these include guards are gone: #ifndef _XLO

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 1:05 PM Robert Haas wrote: > > On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > > The problem here is that OldestXmin is supposed to be more > > conservative than vistest, which it almost always is, except in this > > one edge case. I don't think that plugging tha

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-24 Thread Jeff Davis
On Wed, 2024-06-19 at 08:53 +0530, Ashutosh Sharma wrote: > For standalone functions, users can easily adjust the search_path > settings as needed. However, managing this becomes challenging for > functions created via extensions. Extensions are relocatable, making > it difficult to determine and a

Re: strange context message in spi.c?

2024-06-24 Thread Daniel Gustafsson
> On 24 Jun 2024, at 11:14, Stepan Neretin wrote: > > Hi! Looks good to me! Thanks for review. I have this on my TODO for when the tree branches, it doesn't seem like anything worth squeezing in before then. -- Daniel Gustafsson

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 2:28 PM Robert Haas wrote: > On Mon, Jun 24, 2024 at 1:47 PM Peter Geoghegan wrote: > > I agree, with the proviso that "avoid gratuitous failures" should > > include cases where a query that got the optimization suddenly fails > > to get the optimization, due only to some

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread Robert Haas
On Mon, Jun 17, 2024 at 6:38 PM David E. Wheeler wrote: > Is it? ISTM that there is the intention not to break things that don’t need > to be broken, though that doesn’t rule out interface improvements. I suppose that it's true that we try to avoid gratuitous breakage, but I feel like it would b

Re: improve predefined roles documentation

2024-06-24 Thread Robert Haas
On Fri, Jun 21, 2024 at 11:40 AM Nathan Bossart wrote: > Done. If you look at how the varlistentries begin, there are three separate patterns: * Some document a single role and start with "Allow doing blah blah blah". * Some document a couple of rolls so there are several paragraphs, each begin

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 1:47 PM Peter Geoghegan wrote: > I agree, with the proviso that "avoid gratuitous failures" should > include cases where a query that got the optimization suddenly fails > to get the optimization, due only to some very innocuous looking > change. Such as a change from using

PostgreSQL does not compile on macOS SDK 15.0

2024-06-24 Thread Stan Hu
Xcode 16 beta was released on 2024-06-10 and ships with macOS SDK 15.0 [1]. It appears PostgreSQL does not compile due to typedef redefiniton errors in the regex library: /tmp/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -Wall -Wmissing-prototypes -Wpointer-a

Re: Apparent bug in WAL summarizer process (hung state)

2024-06-24 Thread Israel Barth Rubio
I'm attaching the files which I missed in the original email. > 19:34:17.437626 epoll_wait(5, [], 1, 8161) = 0 <8.171542> 19:34:25.610176 rt_sigprocmask(SIG_SETMASK, [URG], NULL, 8) = 0 <0.000334> 19:34:25.611012 openat(AT_FDCWD, "pg_wal/0001000200B3", O_RDONLY) = -1 ENOENT (No such f

Apparent bug in WAL summarizer process (hung state)

2024-06-24 Thread Israel Barth Rubio
Hello, Hope you are doing well. I've been playing a bit with the incremental backup feature which might come as part of the 17 release, and I think I hit a possible bug in the WAL summarizer process. The issue that I face refers to the summarizer process getting into a hung state. When the issue

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 1:46 PM Peter Geoghegan wrote: > BTW, did you know MySQL has long supported the latter? It has a <=> > operator, which is basically a non-standard spelling of IS NOT > DISTINCT FROM. Importantly, it is indexable, whereas right now > Postgres doesn't support indexing IS NOT

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Robert Haas
On Wed, Apr 3, 2024 at 3:13 AM Alvaro Herrera wrote: > I support the idea of there being a second location from where to load > shared libraries ... but I don't like the idea of making it > runtime-configurable. If we want to continue to tighten up what > superuser can do, then one of the things

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 1:29 PM Robert Haas wrote: > I am not against handling this kind of case if we can do it, but it's > more important that the patch doesn't cause gratuitous failures than > that it handles more cases. I agree, with the proviso that "avoid gratuitous failures" should include

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 1:05 PM Robert Haas wrote: > On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > > The problem here is that OldestXmin is supposed to be more > > conservative than vistest, which it almost always is, except in this > > one edge case. I don't think that plugging that

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 12:09 PM Peter Geoghegan wrote: > But what about cases like this: > > SELECT * FROM mytable WHERE columna = 1_000_000_000 or columna = > 5_000_000_000; -- columna is int4 > > This is using two types, of course. 1_000_000_000 is int4, while > 5_000_000_000 is bigint. If the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > The problem here is that OldestXmin is supposed to be more > conservative than vistest, which it almost always is, except in this > one edge case. I don't think that plugging that hole changes the basic > fact that there is one source of tr

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 11:44 AM Robert Haas wrote: > I don't have a great feeling about this fix. It's not that I think > it's wrong. It's just that the underlying problem here is that we have > heap_page_prune_and_freeze() getting both GlobalVisState *vistest and > struct VacuumCutoffs *cutoffs,

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 01:52:26PM -0400, David E. Wheeler wrote: > I realize this probably isn´t going to happen for 17, given the freeze, > but I would very much welcome feedback and pointers to address concerns > about providing a second directory for extensions and DSOs. Quite a few > people ha

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 11:28 AM Robert Haas wrote: > > It needs to transform all similar constants to one type, because some > > constants of "OR" expressions can belong others, like the numeric and int > > types. Due to the fact that array structure demands that all types must be > > belonged

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 8:44 AM Nathan Bossart wrote: > On Mon, Jun 24, 2024 at 02:32:27PM +0200, Joel Jacobson wrote: > > This patch is based on a suggestion from a separate thread [1]: > > > > On Mon, Jun 24, 2024, at 01:46, Michael Paquier wrote: > >> Rather unrelated to this patch, still this

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Matthias van de Meent
On Mon, 24 Jun 2024 at 14:42, Jelte Fennema-Nio wrote: > > On Mon, 24 Jun 2024 at 13:02, Matthias van de Meent > wrote: > > It does not really behave similar: index scan keys (such as the > > id3=101 scankey) don't require visibility checks in the btree code, > > while the Filter condition _does_

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Nathan Bossart
On Mon, Jun 24, 2024 at 02:32:27PM +0200, Joel Jacobson wrote: > This patch is based on a suggestion from a separate thread [1]: > > On Mon, Jun 24, 2024, at 01:46, Michael Paquier wrote: >> Rather unrelated to this patch, still this patch makes the situation >> more complicated in the docs, but w

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Robert Haas
On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman wrote: > We can fix this by always removing tuples considered dead before > VacuumCutoffs->OldestXmin. I don't have a great feeling about this fix. It's not that I think it's wrong. It's just that the underlying problem here is that we have heap_pa

Re: basebackups seem to have serious issues with FILE_COPY in CREATE DATABASE

2024-06-24 Thread Tomas Vondra
On 6/24/24 17:14, Nathan Bossart wrote: > On Mon, Jun 24, 2024 at 04:12:38PM +0200, Tomas Vondra wrote: >> The important observation is that this only happens if a database is >> created while the backup is running, and that it only happens with the >> FILE_COPY strategy - I've never seen this with

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Robert Haas
On Fri, Jun 21, 2024 at 6:52 PM Alena Rybakina wrote: > It's hard to tell, but I think it might be one of the good places to apply > transformation. Let me describe a brief conclusion on the two approaches. This explanation is somewhat difficult for me to follow. For example: > In the first app

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Tomas Vondra
On 6/24/24 16:53, Melanie Plageman wrote: > On Mon, Jun 24, 2024 at 4:27 AM Heikki Linnakangas wrote: >> >> On 21/06/2024 03:02, Peter Geoghegan wrote: >>> On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman >>> wrote: >>> The repro forces a round of index vacuuming after the standby r

Re: basebackups seem to have serious issues with FILE_COPY in CREATE DATABASE

2024-06-24 Thread Nathan Bossart
On Mon, Jun 24, 2024 at 04:12:38PM +0200, Tomas Vondra wrote: > The important observation is that this only happens if a database is > created while the backup is running, and that it only happens with the > FILE_COPY strategy - I've never seen this with WAL_LOG (which is the > default since PG15).

Re: scalability bottlenecks with (many) partitions (and more)

2024-06-24 Thread Robert Haas
On Sun, Jan 28, 2024 at 4:57 PM Tomas Vondra wrote: > For NUM_LOCK_PARTITIONS this is pretty simple (see 0001 patch). The > LWLock table has 16 partitions by default - it's quite possible that on > machine with many cores and/or many partitions, we can easily hit this. > So I bumped this 4x to 64

Re: Injection point locking

2024-06-24 Thread Tom Lane
Heikki Linnakangas writes: > ... I can't do that, because InjectionPointRun() requires a PGPROC > entry, because it uses an LWLock. That also makes it impossible to use > injection points in the postmaster. Any chance we could allow injection > points to be triggered without a PGPROC entry? Cou

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:27 AM Heikki Linnakangas wrote: > > On 21/06/2024 03:02, Peter Geoghegan wrote: > > On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman > > wrote: > > > >> The repro forces a round of index vacuuming after the standby > >> reconnects and before pruning a dead tuple whose xm

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:10 AM Alena Rybakina wrote: > > We can fix this by always removing tuples considered dead before > VacuumCutoffs->OldestXmin. This is okay even if a reconnected standby > has a transaction that sees that tuple as alive, because it will > simply wait to replay the removal

  1   2   >