Re: 001_rep_changes.pl fails due to publisher stuck on shutdown

2024-06-24 Thread Kyotaro Horiguchi
At Fri, 21 Jun 2024 11:48:22 +0530, Amit Kapila wrote in > On Wed, Jun 19, 2024 at 10:44 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Horiguchi-san, > > > > Thanks for sharing the patch! I agree this approach (ensure WAL records are > > flushed) > > Is more proper than others. > > > > I

Re: Support "Right Semi Join" plan shapes

2024-06-24 Thread Japin Li
On Mon, 24 Jun 2024 at 17:59, Richard Guo wrote: > Thank you for reviewing. > > On Mon, Jun 24, 2024 at 1:27 PM Li Japin wrote: >> + /* >> +* For now we do not support RIGHT_SEMI join in mergejoin or nestloop >> +* join. >> +*/ >> + if (jointype ==

Re: Add pg_get_acl() function get the ACL for a database object

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 01:46, Michael Paquier wrote: > Rather unrelated to this patch, still this patch makes the situation > more complicated in the docs, but wouldn't it be better to add ACL as > a term in acronyms.sql, and reuse it here? It would be a doc-only > patch that applies on top of

Re: Apparent bug in WAL summarizer process (hung state)

2024-06-24 Thread Michael Paquier
for this report! > FWIW, once I restart Postgres the WAL summarizer process gets back to normal > functioning. It seems to me there is some race condition between when a WAL > file > is removed and when `summarize_wal` is re-enabled, causing the process to > keep > looking for a

Re: improve predefined roles documentation

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 2:53 PM Nathan Bossart wrote: > On Mon, Jun 24, 2024 at 02:44:33PM -0400, Robert Haas wrote: > > > I don't know what to do about pg_database_owner. I almost wonder if > > that should be moved out of the table and documented as a special > > case. Or maybe some more

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 23:15, David G. Johnston wrote: > I really dislike "For avoidance of doubt and clarity" - and in terms of > being equivalent the following seems like a more accurate description > of reality. > > The PostgreSQL documentation, and code, refers to the specifications >

Re: speed up a logical replica setup

2024-06-24 Thread Noah Misch
On Mon, Jun 24, 2024 at 05:20:21PM +0530, Amit Kapila wrote: > On Sun, Jun 23, 2024 at 11:52 AM Noah Misch wrote: > > > > > +static void > > > +create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo) > > > +{ > > > > > + appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", >

Re: improve predefined roles documentation

2024-06-24 Thread Nathan Bossart
On Mon, Jun 24, 2024 at 02:44:33PM -0400, Robert Haas wrote: > I think the first two cases could be made more like each other by > changing the varlistentires that are just about one setting to use the > second format instead of the first, e.g. pg_checkpoint allows > executing the CHECKPOINT

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 22:42, David E. Wheeler wrote: > >> BINDIR > >> DOCDIR > >> HTMLDIR > >> PKGINCLUDEDIR > >> LOCALEDIR > >> MANDIR > >> > >> I can imagine an extension wanting or needing to use any and all of these. > > > > Are these really all relevant to backend code? > > Oh I think so.

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 17:17, Jelte Fennema-Nio wrote: > If you want to only change $libdir during CREATE EXTENSION (or ALTER > EXTENSION UPDATE), then why not just change it there. And really you'd > only want to change it when creating an extension from which the > control file is coming from

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 18:11, Nathan Bossart wrote: > At first glance, the general idea seems reasonable to me. I'm wondering > whether there is a requirement for this directory to be prepended or if it > could be appended to the end. That way, the existing ones would take > priority, which

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Jelte Fennema-Nio
On Thu, 11 Apr 2024 at 19:52, David E. Wheeler wrote: > I realize this probably isn’t going to happen for 17, given the freeze, but I > would very much welcome feedback and pointers to address concerns about > providing a second directory for extensions and DSOs. Quite a few people have >

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 1:49 PM Joel Jacobson wrote: > How about? > > + > + The linked page uses "permissions" while we consistently use the > synonym > + "privileges", to describe the contents of the list. For avoidance of > + doubt and clarity, these two terms are

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:51 PM Peter Geoghegan wrote: > > On Mon, Jun 24, 2024 at 4:36 PM Robert Haas wrote: > > I thought the idea was that the GlobalVisTest stuff would force a > > recalculation now and then, but maybe it doesn't actually do that? > > It definitely can do that. Just not in a

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Nikolay Shaplov
Hi! Let me join the review process. I am no expert in execution plans, so there would not be much help in doing even better optimization. But I can read the code, as a person who is not familiar with this area and help making it clear even to a person like me. So, I am reading

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 4:36 PM Robert Haas wrote: > I'm not sure I understand. The most important thing here is fixing the > bug. But if we have a choice of how to fix the bug, I'd prefer to do > it by having the pruning code test one horizon that is always correct, > rather than (as I think the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:42 PM Peter Geoghegan wrote: > > On Mon, Jun 24, 2024 at 3:36 PM Melanie Plageman > wrote: > > One thing I don't understand is why it is okay to freeze the xmax of a > > dead tuple just because it is from an aborted update. > > We don't do that with XID-based xmaxs.

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 21:51, David G. Johnston wrote: > On Mon, Jun 24, 2024 at 12:46 PM Joel Jacobson wrote: >> On Mon, Jun 24, 2024, at 18:02, David G. Johnston wrote: >> >> > The page we link to uses "permissions" while we consistently use >> > "privileges" to describe the contents of the

Re: Partial aggregates pushdown

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 15:03, fujii.y...@df.mitsubishielectric.co.jp wrote: > I see. I maybe got your proposal. > Refer to your proposal, for avg(int8), > I create a new native type like state_int8_avg > with the new typsend/typreceive functions > and use them to transmit the state value, right?

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 3:36 PM Melanie Plageman wrote: > One thing I don't understand is why it is okay to freeze the xmax of a > dead tuple just because it is from an aborted update. We don't do that with XID-based xmaxs. Though perhaps we should, since we'll already prune-away the successor

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 4:28 PM, Robert Haas wrote: > As long as the GUC is superuser-only, I'm not sure what else there is > to do here. The only question is whether there's some reason to > disallow this even from the superuser, but I'm not quite seeing such a > reason. I can switch it back from

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 3:23 PM Melanie Plageman wrote: > Are you more concerned about having a single horizon for pruning or > about having a horizon that does not move backwards after being > established at the beginning of vacuuming the relation? I'm not sure I understand. The most important

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Nikolay Shaplov
Hi! Let me join the review process. I am no expert in execution plans, so there would not be much help in doing even better optimization. But I can read the code, as a person who is not familiar with this area and help making it clear even to a person like me. So, I am reading

Re: Direct SSL connection and ALPN loose ends

2024-06-24 Thread Heikki Linnakangas
On 21/06/2024 02:32, Jacob Champion wrote: On Thu, Jun 20, 2024 at 4:13 PM Heikki Linnakangas wrote: By "negotiation" I mean the server's response to the startup packet. I.e. "supported"/"not supported"/"error". Ok, I'm still a little confused, probably a terminology issue. The server

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 3:37 PM David E. Wheeler wrote: > I guess the question then is what security controls are appropriate for this > feature, which after all tells the postmaster what directories to read files > from. It feels a little outside the scope of a regular user to even be aware >

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread David E. Wheeler
On Jun 19, 2024, at 05:41, Peter Eisentraut wrote: > This is probably a bit confusing. This might as well mean client application > code against libpq. Better something like "server plugin code that uses the > PostgreSQL server APIs". That works. > But now we're talking about API. That

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 12:46 PM Joel Jacobson wrote: > On Mon, Jun 24, 2024, at 18:02, David G. Johnston wrote: > > > The page we link to uses "permissions" while we consistently use > > "privileges" to describe the contents of the list. This seems like an > > obvious synonym, but as the point

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread David E . Wheeler
On Jun 24, 2024, at 14:51, Robert Haas wrote: > I suppose that it's true that we try to avoid gratuitous breakage, but > I feel like it would be weird to document that. I see how that can seem weird to a committer deeply familiar with the development process and how things happen. But people

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread David E. Wheeler
On Jun 19, 2024, at 05:42, Peter Eisentraut wrote: >>> https://postgr.es/m/CAH2-Wzm-W6hSn71sUkz0Rem=qdeu7tnfmc7_jg2djrlfef_...@mail.gmail.com >>> >>> Theoretically anybody can do this themselves. In practice they don't. >>> So something as simple as providing automated reports about ABI >>>

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Joel Jacobson
On Mon, Jun 24, 2024, at 18:02, David G. Johnston wrote: > On Mon, Jun 24, 2024 at 8:44 AM Nathan Bossart > wrote: >> I think we could omit "i.e. privileges list." >> > > Agreed. Between the docs and code we say "privileges list" once and > that refers to the dumputIls description of the

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread David E. Wheeler
On Jun 24, 2024, at 1:53 PM, Robert Haas wrote: > Is "tighten up what the superuser can do" on our list of objectives? > Personally, I think we should be focusing mostly, and maybe entirely, > on letting non-superusers do more things, with appropriate security > controls. The superuser can

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman wrote: > > If vacuum fails to remove a tuple with xmax older than > VacuumCutoffs->OldestXmin and younger than > GlobalVisState->maybe_needed, it will ERROR out when determining > whether or not to freeze the tuple with "cannot freeze committed >

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 3:23 PM Melanie Plageman wrote: > I had always thought it was because the vacuuming backend's > GlobalVisState will get updated periodically throughout vacuum and so, > assuming the oldest running transaction changes, our horizon for > vacuum would change. I believe that

Re: PostgreSQL does not compile on macOS SDK 15.0

2024-06-24 Thread Stan Hu
It appears in macOS SDK 14.5, there were include guards in $SDK_ROOT/usr/include/xlocale/_regex.h: #ifndef _XLOCALE__REGEX_H_ #define _XLOCALE__REGEX_H_ #ifndef _REGEX_H_ #include <_regex.h> #endif // _REGEX_H_ #include <_xlocale.h> In macOS SDK 15.5, these include guards are gone: #ifndef

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 1:05 PM Robert Haas wrote: > > On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > > The problem here is that OldestXmin is supposed to be more > > conservative than vistest, which it almost always is, except in this > > one edge case. I don't think that plugging

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-24 Thread Jeff Davis
On Wed, 2024-06-19 at 08:53 +0530, Ashutosh Sharma wrote: > For standalone functions, users can easily adjust the search_path > settings as needed. However, managing this becomes challenging for > functions created via extensions. Extensions are relocatable, making > it difficult to determine and

Re: strange context message in spi.c?

2024-06-24 Thread Daniel Gustafsson
> On 24 Jun 2024, at 11:14, Stepan Neretin wrote: > > Hi! Looks good to me! Thanks for review. I have this on my TODO for when the tree branches, it doesn't seem like anything worth squeezing in before then. -- Daniel Gustafsson

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 2:28 PM Robert Haas wrote: > On Mon, Jun 24, 2024 at 1:47 PM Peter Geoghegan wrote: > > I agree, with the proviso that "avoid gratuitous failures" should > > include cases where a query that got the optimization suddenly fails > > to get the optimization, due only to some

Re: Proposal: Document ABI Compatibility

2024-06-24 Thread Robert Haas
On Mon, Jun 17, 2024 at 6:38 PM David E. Wheeler wrote: > Is it? ISTM that there is the intention not to break things that don’t need > to be broken, though that doesn’t rule out interface improvements. I suppose that it's true that we try to avoid gratuitous breakage, but I feel like it would

Re: improve predefined roles documentation

2024-06-24 Thread Robert Haas
On Fri, Jun 21, 2024 at 11:40 AM Nathan Bossart wrote: > Done. If you look at how the varlistentries begin, there are three separate patterns: * Some document a single role and start with "Allow doing blah blah blah". * Some document a couple of rolls so there are several paragraphs, each

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 1:47 PM Peter Geoghegan wrote: > I agree, with the proviso that "avoid gratuitous failures" should > include cases where a query that got the optimization suddenly fails > to get the optimization, due only to some very innocuous looking > change. Such as a change from

Re: Apparent bug in WAL summarizer process (hung state)

2024-06-24 Thread Israel Barth Rubio
I'm attaching the files which I missed in the original email. > 19:34:17.437626 epoll_wait(5, [], 1, 8161) = 0 <8.171542> 19:34:25.610176 rt_sigprocmask(SIG_SETMASK, [URG], NULL, 8) = 0 <0.000334> 19:34:25.611012 openat(AT_FDCWD, "pg_wal/0001000200B3", O_RDONLY) = -1 ENOENT (No such

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 1:46 PM Peter Geoghegan wrote: > BTW, did you know MySQL has long supported the latter? It has a <=> > operator, which is basically a non-standard spelling of IS NOT > DISTINCT FROM. Importantly, it is indexable, whereas right now > Postgres doesn't support indexing IS NOT

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Robert Haas
On Wed, Apr 3, 2024 at 3:13 AM Alvaro Herrera wrote: > I support the idea of there being a second location from where to load > shared libraries ... but I don't like the idea of making it > runtime-configurable. If we want to continue to tighten up what > superuser can do, then one of the things

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 1:29 PM Robert Haas wrote: > I am not against handling this kind of case if we can do it, but it's > more important that the patch doesn't cause gratuitous failures than > that it handles more cases. I agree, with the proviso that "avoid gratuitous failures" should

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 1:05 PM Robert Haas wrote: > On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > > The problem here is that OldestXmin is supposed to be more > > conservative than vistest, which it almost always is, except in this > > one edge case. I don't think that plugging that

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 12:09 PM Peter Geoghegan wrote: > But what about cases like this: > > SELECT * FROM mytable WHERE columna = 1_000_000_000 or columna = > 5_000_000_000; -- columna is int4 > > This is using two types, of course. 1_000_000_000 is int4, while > 5_000_000_000 is bigint. If the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Robert Haas
On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > The problem here is that OldestXmin is supposed to be more > conservative than vistest, which it almost always is, except in this > one edge case. I don't think that plugging that hole changes the basic > fact that there is one source of

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 11:44 AM Robert Haas wrote: > I don't have a great feeling about this fix. It's not that I think > it's wrong. It's just that the underlying problem here is that we have > heap_page_prune_and_freeze() getting both GlobalVisState *vistest and > struct VacuumCutoffs

Re: RFC: Additional Directory for Extensions

2024-06-24 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 01:52:26PM -0400, David E. Wheeler wrote: > I realize this probably isn´t going to happen for 17, given the freeze, > but I would very much welcome feedback and pointers to address concerns > about providing a second directory for extensions and DSOs. Quite a few > people

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Peter Geoghegan
On Mon, Jun 24, 2024 at 11:28 AM Robert Haas wrote: > > It needs to transform all similar constants to one type, because some > > constants of "OR" expressions can belong others, like the numeric and int > > types. Due to the fact that array structure demands that all types must be > >

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread David G. Johnston
On Mon, Jun 24, 2024 at 8:44 AM Nathan Bossart wrote: > On Mon, Jun 24, 2024 at 02:32:27PM +0200, Joel Jacobson wrote: > > This patch is based on a suggestion from a separate thread [1]: > > > > On Mon, Jun 24, 2024, at 01:46, Michael Paquier wrote: > >> Rather unrelated to this patch, still

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Matthias van de Meent
On Mon, 24 Jun 2024 at 14:42, Jelte Fennema-Nio wrote: > > On Mon, 24 Jun 2024 at 13:02, Matthias van de Meent > wrote: > > It does not really behave similar: index scan keys (such as the > > id3=101 scankey) don't require visibility checks in the btree code, > > while the Filter condition

Re: [PATCH] Add ACL (Access Control List) acronym

2024-06-24 Thread Nathan Bossart
On Mon, Jun 24, 2024 at 02:32:27PM +0200, Joel Jacobson wrote: > This patch is based on a suggestion from a separate thread [1]: > > On Mon, Jun 24, 2024, at 01:46, Michael Paquier wrote: >> Rather unrelated to this patch, still this patch makes the situation >> more complicated in the docs, but

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Robert Haas
On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman wrote: > We can fix this by always removing tuples considered dead before > VacuumCutoffs->OldestXmin. I don't have a great feeling about this fix. It's not that I think it's wrong. It's just that the underlying problem here is that we have

Re: basebackups seem to have serious issues with FILE_COPY in CREATE DATABASE

2024-06-24 Thread Tomas Vondra
On 6/24/24 17:14, Nathan Bossart wrote: > On Mon, Jun 24, 2024 at 04:12:38PM +0200, Tomas Vondra wrote: >> The important observation is that this only happens if a database is >> created while the backup is running, and that it only happens with the >> FILE_COPY strategy - I've never seen this

Re: POC, WIP: OR-clause support for indexes

2024-06-24 Thread Robert Haas
On Fri, Jun 21, 2024 at 6:52 PM Alena Rybakina wrote: > It's hard to tell, but I think it might be one of the good places to apply > transformation. Let me describe a brief conclusion on the two approaches. This explanation is somewhat difficult for me to follow. For example: > In the first

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Tomas Vondra
On 6/24/24 16:53, Melanie Plageman wrote: > On Mon, Jun 24, 2024 at 4:27 AM Heikki Linnakangas wrote: >> >> On 21/06/2024 03:02, Peter Geoghegan wrote: >>> On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman >>> wrote: >>> The repro forces a round of index vacuuming after the standby

Re: basebackups seem to have serious issues with FILE_COPY in CREATE DATABASE

2024-06-24 Thread Nathan Bossart
On Mon, Jun 24, 2024 at 04:12:38PM +0200, Tomas Vondra wrote: > The important observation is that this only happens if a database is > created while the backup is running, and that it only happens with the > FILE_COPY strategy - I've never seen this with WAL_LOG (which is the > default since

Re: scalability bottlenecks with (many) partitions (and more)

2024-06-24 Thread Robert Haas
On Sun, Jan 28, 2024 at 4:57 PM Tomas Vondra wrote: > For NUM_LOCK_PARTITIONS this is pretty simple (see 0001 patch). The > LWLock table has 16 partitions by default - it's quite possible that on > machine with many cores and/or many partitions, we can easily hit this. > So I bumped this 4x to 64

Re: Injection point locking

2024-06-24 Thread Tom Lane
Heikki Linnakangas writes: > ... I can't do that, because InjectionPointRun() requires a PGPROC > entry, because it uses an LWLock. That also makes it impossible to use > injection points in the postmaster. Any chance we could allow injection > points to be triggered without a PGPROC entry?

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:27 AM Heikki Linnakangas wrote: > > On 21/06/2024 03:02, Peter Geoghegan wrote: > > On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman > > wrote: > > > >> The repro forces a round of index vacuuming after the standby > >> reconnects and before pruning a dead tuple whose

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:10 AM Alena Rybakina wrote: > > We can fix this by always removing tuples considered dead before > VacuumCutoffs->OldestXmin. This is okay even if a reconnected standby > has a transaction that sees that tuple as alive, because it will > simply wait to replay the removal

Re: long-standing data loss bug in initial sync of logical replication

2024-06-24 Thread Tomas Vondra
On 6/24/24 12:54, Amit Kapila wrote: > ... >> I'm not sure there are any cases where using SRE instead of AE would cause problems for logical decoding, but it seems very hard to prove. I'd be very surprised if just using SRE would not lead to corrupted cache contents in some

RE: Partial aggregates pushdown

2024-06-24 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi. Jelte, hackers. Thank you for your proposal and comments. > From: Jelte Fennema-Nio > Sent: Monday, June 24, 2024 6:09 PM > > 1. Generality > > I believe we should develop a method that can theoretically apply to any > aggregate function, even if we cannot implement it immediately. However,

Re: Pgoutput not capturing the generated columns

2024-06-24 Thread Shlok Kyal
On Fri, 21 Jun 2024 at 12:51, Peter Smith wrote: > > Hi, Here are some review comments for patch v9-0003 > > == > Commit Message > > /fix/fixes/ Fixed > == > 1. > General. Is tablesync enough? > > I don't understand why is the patch only concerned about tablesync? > Does it make sense to

Re: replace strtok()

2024-06-24 Thread Peter Eisentraut
On 24.06.24 02:34, Michael Paquier wrote: On Sat, Jun 22, 2024 at 11:48:21AM -0400, Tom Lane wrote: Peter Eisentraut writes: On 18.06.24 13:43, Ranier Vilela wrote: I found another implementation of strsep, it seems lighter to me. I will attach it for consideration, however, I have not done

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Ashutosh Bapat
On Mon, Jun 24, 2024 at 8:08 AM wrote: > > I am unable to decide whether reporting the bound quals is just enough > to decide the efficiency of index without knowing the difference in the > number of index tuples selectivity and heap tuple selectivity. The > difference seems to be a better

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Jelte Fennema-Nio
On Mon, 24 Jun 2024 at 13:02, Matthias van de Meent wrote: > It does not really behave similar: index scan keys (such as the > id3=101 scankey) don't require visibility checks in the btree code, > while the Filter condition _does_ require a visibility check, and > delegates the check to the table

Re: Conflict detection and logging in logical replication

2024-06-24 Thread Nisha Moond
On Mon, Jun 24, 2024 at 7:39 AM Zhijie Hou (Fujitsu) wrote: > > When testing the patch, I noticed a bug that when reporting the conflict > after calling ExecInsertIndexTuples(), we might find the tuple that we > just inserted and report it.(we should only report conflict if there are > other

Re: Changing default -march landscape

2024-06-24 Thread Christoph Berg
Re: To Thomas Munro > Or Debian might just bump the baseline. PostgreSQL asking for it might > just be the reason we wanted to hear to make it happen. Which level would PostgreSQL specifically want? x86-64-v2 or even x86-64-v3? Christoph

Re: Meson far from ready on Windows

2024-06-24 Thread Dave Page
On Sat, 22 Jun 2024 at 17:35, Andres Freund wrote: > Hi, > > On 2024-06-21 15:36:56 +0100, Dave Page wrote: > > For giggles, I took a crack at doing that, manually creating .pc files > for > > everything I've been working with so far. > > Cool! > > > > It seems to work as expected, except that

Re: Changing default -march landscape

2024-06-24 Thread Christoph Berg
Hi, sorry for the delayed reply, I suck at prioritizing things. Re: Thomas Munro > OK let me CC Christoph and ask the question this way: hypothetically, > if RHEL users' PostgreSQL packages became automatically faster than > Debian users' packages because of default -mar

Re: Inconsistent Parsing of Offsets with Seconds

2024-06-24 Thread David E. Wheeler
On Jun 22, 2024, at 14:10, David E. Wheeler wrote: > I believe the former issue is caused by the latter: The jsonpath > implementation uses the formatting strings to parse the timestamps[1], and > since there is no formatting to support offsets with seconds, it doesn’t work > at all in JSON

Re: speed up a logical replica setup

2024-06-24 Thread Amit Kapila
On Sun, Jun 23, 2024 at 11:52 AM Noah Misch wrote: > > > +static void > > +create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo) > > +{ > > > + appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", > > + ipubname_esc); > > This tool's

Re: sql/json miscellaneous issue

2024-06-24 Thread Amit Langote
Hi, On Mon, Jun 24, 2024 at 7:04 PM jian he wrote: > > hi. > the following two queries should return the same result? > > SELECT * FROM JSON_query (jsonb 'null', '$' returning jsonb); > SELECT * FROM JSON_value (jsonb 'null', '$' returning jsonb); I get this with HEAD: SELECT * FROM JSON_query

Re: Conflict detection and logging in logical replication

2024-06-24 Thread shveta malik
On Mon, Jun 24, 2024 at 7:39 AM Zhijie Hou (Fujitsu) wrote: > > When testing the patch, I noticed a bug that when reporting the conflict > after calling ExecInsertIndexTuples(), we might find the tuple that we > just inserted and report it.(we should only report conflict if there are > other

Re: Avoid incomplete copy string (src/backend/access/transam/xlog.c)

2024-06-24 Thread Ranier Vilela
Em seg., 24 de jun. de 2024 às 00:27, Yugo NAGATA escreveu: > On Sun, 23 Jun 2024 22:34:03 -0300 > Ranier Vilela wrote: > > > Em dom., 23 de jun. de 2024 às 22:14, Ranier Vilela > > > escreveu: > > > > > Em dom., 23 de jun. de 2024 às 22:05, Ranier Vilela < > ranier...@gmail.com> > > >

Re: Avoid incomplete copy string (src/backend/access/transam/xlog.c)

2024-06-24 Thread Ranier Vilela
Em dom., 23 de jun. de 2024 às 23:56, Richard Guo escreveu: > On Mon, Jun 24, 2024 at 7:51 AM Ranier Vilela wrote: > > In src/include/access/xlogbackup.h, the field *name* > > has one byte extra to store null-termination. > > > > But, in the function *do_pg_backup_start*, > > I think that is a

Re: sql/json miscellaneous issue

2024-06-24 Thread Amit Langote
Hi, On Mon, Jun 24, 2024 at 8:02 PM Stepan Neretin wrote: > Hi! > > I also noticed a very strange difference in behavior in these two queries, it > seems to me that although it returns a string by default, for the boolean > operator it is necessary to return true or false > SELECT * FROM

Re: sql/json miscellaneous issue

2024-06-24 Thread Stepan Neretin
On Mon, Jun 24, 2024 at 5:05 PM jian he wrote: > hi. > the following two queries should return the same result? > > SELECT * FROM JSON_query (jsonb 'null', '$' returning jsonb); > SELECT * FROM JSON_value (jsonb 'null', '$' returning jsonb); > > I've tried a patch to implement it. > (i raised

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Matthias van de Meent
On Mon, 24 Jun 2024 at 11:58, Jelte Fennema-Nio wrote: > > +1 for the idea. > > On Mon, 24 Jun 2024 at 11:11, Matthias van de Meent > wrote: > > I think this is too easy to confuse with the pre-existing 'Filter' > > condition, which you'll find on indexes with INCLUDE-d columns or > > filters on

Re: long-standing data loss bug in initial sync of logical replication

2024-06-24 Thread Amit Kapila
On Sun, Nov 19, 2023 at 7:48 AM Andres Freund wrote: > > On 2023-11-19 02:15:33 +0100, Tomas Vondra wrote: > > > > If understand correctly, with the current code (which only gets > > ShareUpdateExclusiveLock), we may end up in a situation like this > > (sessions A and B): > > > > A: starts

Re: Track the amount of time waiting due to cost_delay

2024-06-24 Thread Bertrand Drouvot
Hi, On Sat, Jun 22, 2024 at 12:48:33PM +, Bertrand Drouvot wrote: > 1. vacuuming indexes time has been longer on master because with v2, the > leader > has been interrupted 342605 times while waiting, then making v2 "faster". > > 2. the leader being interrupted while waiting is also already

Re: Buildfarm animal caiman showing a plperl test issue with newer Perl versions

2024-06-24 Thread Andrew Dunstan
On 2024-06-24 Mo 12:00 AM, Alexander Lakhin wrote: Hello hackers, As recent caiman failures ([1], [2], ...) show, plperl.sql is incompatible with Perl 5.40. (The last successful test runs took place when cayman had Perl 5.38.2 installed: [3].) FWIW, I've found an already-existing fix for

Re: Support "Right Semi Join" plan shapes

2024-06-24 Thread Richard Guo
Thank you for reviewing. On Mon, Jun 24, 2024 at 1:27 PM Li Japin wrote: > + /* > +* For now we do not support RIGHT_SEMI join in mergejoin or nestloop > +* join. > +*/ > + if (jointype == JOIN_RIGHT_SEMI) > + return; > + > > How about adding

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Jelte Fennema-Nio
+1 for the idea. On Mon, 24 Jun 2024 at 11:11, Matthias van de Meent wrote: > I think this is too easy to confuse with the pre-existing 'Filter' > condition, which you'll find on indexes with INCLUDE-d columns or > filters on non-index columns. Why not combine them? And both call them Filter?

Re: strange context message in spi.c?

2024-06-24 Thread Stepan Neretin
Hi! Looks good to me! Best regards, Stepan Neretin.

Re: Meson far from ready on Windows

2024-06-24 Thread Andres Freund
Hi, On 2024-06-24 09:54:51 +0100, Dave Page wrote: > > The old system was a major bottleneck. For one, there was no way to run all > > tests. And even the tests that one could run, would run serially, leading > > to > > exceedingly long tests times. While that could partially be addressed by > >

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-24 Thread Matthias van de Meent
On Mon, 24 Jun 2024 at 04:38, wrote: > > In my local PoC patch, I have modified the output as follows, what do you > think? > > =# EXPLAIN (VERBOSE, ANALYZE) SELECT * FROM test WHERE id1 = 1 AND id2 = 101; >QUERY PLAN >

Re: Partial aggregates pushdown

2024-06-24 Thread Jelte Fennema-Nio
On Sun, 23 Jun 2024 at 10:24, fujii.y...@df.mitsubishielectric.co.jp wrote: > I attached the POC patch(the second one) which supports avg(int8) whose > standard format is _numeric type. Okay, very interesting. So instead of defining the serialization/deserialization functions to text/binary,

Re: Meson far from ready on Windows

2024-06-24 Thread Dave Page
Hi On Sat, 22 Jun 2024 at 17:32, Andres Freund wrote: > > I don't think it's unreasonable to not support static linking, but I take > > your point. > > Separately from this thread: ISTM that on windows it'd be quite beneficial > to > link a few things statically, given how annoying dealing with

Re: Proposal: Division operator for (interval / interval => double precision)

2024-06-24 Thread Ashutosh Bapat
On Mon, Jun 24, 2024 at 2:04 PM Laurenz Albe wrote: > On Sun, 2024-06-23 at 17:57 -0700, Gurjeet Singh wrote: > > Is there a desire to have a division operator / that takes dividend > > and divisor of types interval, and results in a quotient of type > > double precision. > > > > This would be

Re: Meson far from ready on Windows

2024-06-24 Thread Dave Page
Hi Tristan, On Fri, 21 Jun 2024 at 18:16, Tristan Partin wrote: > > Hey Dave, > > I'm a maintainer for Meson, and am happy to help you in any way that > I reasonably can. > Thank you! > > Let's start with the state of Windows support in Meson. If I were to > rank Meson support for platforms,

Re: Proposal: Division operator for (interval / interval => double precision)

2024-06-24 Thread Laurenz Albe
On Sun, 2024-06-23 at 17:57 -0700, Gurjeet Singh wrote: > Is there a desire to have a division operator / that takes dividend > and divisor of types interval, and results in a quotient of type > double precision. > > This would be helpful in calculating how many times the divisor > interval can

Re: New function normal_rand_array function to contrib/tablefunc.

2024-06-24 Thread Stepan Neretin
It looks useful, for example, it can be used in sorting tests to make them more interesting. I just have one question. Why are you using SRF_IS_FIRST CALL and not _PG_init? Best regards, Stepan Neretin.

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Heikki Linnakangas
On 21/06/2024 03:02, Peter Geoghegan wrote: On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman wrote: If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it will ERROR out when determining whether or not to freeze the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Alena Rybakina
Hi, Melanie! I'm glad to hear you that you have found a root case of the problem) Thank you for that! On 21.06.2024 02:42, Melanie Plageman wrote: Hi, If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it will ERROR

Re: Logical Replication of sequences

2024-06-24 Thread vignesh C
On Thu, 20 Jun 2024 at 18:45, Amit Kapila wrote: > > On Wed, Jun 19, 2024 at 8:33 PM vignesh C wrote: > > > > On Tue, 18 Jun 2024 at 16:10, Amit Kapila wrote: > > > > > > > > > Agreed and I am not sure which is better because there is a value in > > > keeping the state name the same for both

Re: Issues with ON CONFLICT UPDATE and REINDEX CONCURRENTLY

2024-06-24 Thread Michael Paquier
On Mon, Jun 17, 2024 at 07:00:51PM +0200, Michail Nikolaev wrote: > The same issue may happen in case of CREATE/DROP INDEX CONCURRENTLY as well. While looking at all that, I've been also curious about this specific point, and it is indeed possible to finish in a state where a duplicate key would

  1   2   3   4   5   6   7   8   9   10   >