Re: Is there any documentation on how to correctly create extensions in HA(primary-standby) setup?

2022-01-17 Thread Julien Rouhaud
Hi, On Thu, Dec 09, 2021 at 08:19:06AM +0530, Bharath Rupireddy wrote: > > Thanks. Attaching v1 patch specifying the notes there. Please review. I think that the common terminology is "module", not "extension". That's especially important here as this information is also relevant for modules

Re: generic plans and "initial" pruning

2022-01-17 Thread Simon Riggs
On Tue, 11 Jan 2022 at 16:22, Robert Haas wrote: > This is just a relatively simple example and I think there are > probably a bunch of others. There are a lot of kinds of DDL that could > be performed on a partition that gets pruned away: DROP INDEX is just > one example. I haven't followed

Re: 32TB relation size make mdnblocks overflow

2022-01-17 Thread Julien Rouhaud
Hi, On Tue, Jan 18, 2022 at 02:21:14PM +0800, 陈佳昕(步真) wrote: > > We know that PostgreSQL doesn't support a single relation size over 32TB, > limited by the MaxBlockNumber. But if we just 'insert into' one relation over > 32TB, it will get an error message 'unexpected data beyond EOF in block 0

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-17 Thread Julien Rouhaud
Hi, On Sun, Jan 02, 2022 at 02:55:04PM +0100, Fabien COELHO wrote: > > > Here is my review about v32: > > I forgot to tell that doc generation for the cumulated changes also works. Unfortunately the patchset doesn't apply anymore: http://cfbot.cputube.org/patch_36_2377.log === Applying

Re: missing indexes in indexlist with partitioned tables

2022-01-17 Thread Julien Rouhaud
Hi, On Mon, Jan 17, 2022 at 08:32:40PM +, Arne Roland wrote: > > Afaiac the join pruning where the outer table is a partitioned table is the > relevant case. The last version of the patch now fails on all platform, with plan changes. For instance:

Re: BUFFERS enabled by default in EXPLAIN (ANALYZE)

2022-01-17 Thread Julien Rouhaud
Hi, On Wed, Dec 01, 2021 at 06:58:20PM -0600, Justin Pryzby wrote: > > The attached patch series now looks like this (some minor patches are not > included in this list): This version of the patchset doesn't apply anymore: http://cfbot.cputube.org/patch_36_3409.log === Applying patches on top

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Shruthi Gowda
On Tue, Jan 18, 2022 at 12:35 AM Robert Haas wrote: > > On Tue, Dec 14, 2021 at 1:21 PM Shruthi Gowda wrote: > > Thanks, Robert for the updated version. I reviewed the changes and it > > looks fine. > > I also tested the patch. The patch works as expected. > > Thanks. > > > > - I adjusted the

Re: Use generation context to speed up tuplesorts

2022-01-17 Thread Julien Rouhaud
Hi, On Fri, Jan 07, 2022 at 12:03:55PM +0100, Ronan Dunklau wrote: > > (Sorry for trying to merge back the discussion on the two sides of the thread) > > In https://www.postgresql.org/message-id/4776839.iZASKD2KPV%40aivenronan, I > expressed the idea of being able to tune glibc's malloc

Re: Support for NSS as a libpq TLS backend

2022-01-17 Thread Julien Rouhaud
Hi, On Mon, Jan 17, 2022 at 03:09:11PM +0100, Daniel Gustafsson wrote: > > I must've fat-fingered the "git add -p" for v50 as the fix was in configure.ac > but not configure. Fixed now. Thanks! Apparently this version now fails on all OS, e.g.: https://cirrus-ci.com/task/4643868095283200

Re: SQL/JSON: functions

2022-01-17 Thread Julien Rouhaud
Hi, The last version conflicts with recent c4cc2850f4d1 (Rename value node fields). Can you send a rebased version?

Re: SQL/JSON: JSON_TABLE

2022-01-17 Thread Julien Rouhaud
Hi, On Tue, Jan 04, 2022 at 09:03:05AM -0500, Andrew Dunstan wrote: > > rebased again. This version conflicts with recent c4cc2850f4d1 (Rename value node fields). Can you send a rebased version?

Re: simplifying foreign key/RI checks

2022-01-17 Thread Amit Langote
Thanks for the review. On Tue, Dec 21, 2021 at 5:54 PM Zhihong Yu wrote: > Hi, > > + int lockflags = 0; > + TM_Result test; > + > + lockflags = TUPLE_LOCK_FLAG_LOCK_UPDATE_IN_PROGRESS; > > The above assignment can be meged with the line where variable lockflags is > declared.

Re: In-placre persistance change of a relation

2022-01-17 Thread Julien Rouhaud
Hi, On Fri, Jan 14, 2022 at 11:43:10AM +0900, Kyotaro Horiguchi wrote: > I found a bug. > > mdmarkexists() didn't close the tentatively opend fd. This is a silent > leak on Linux and similars and it causes delete failure on Windows. > It was the reason of the CI failure. > >

Re: tweak to a few index tests to hits ambuildempty() routine.

2022-01-17 Thread Rushabh Lathia
On Mon, Nov 29, 2021 at 10:34 AM Amul Sul wrote: > Hi, > > Attached patch is doing small changes to brin, gin & gist index tests > to use an unlogged table without changing the original intention of > those tests and that is able to hit ambuildempty() routing which is > otherwise not reachable

32TB relation size make mdnblocks overflow

2022-01-17 Thread 陈佳昕(步真)
Hello We know that PostgreSQL doesn't support a single relation size over 32TB, limited by the MaxBlockNumber. But if we just 'insert into' one relation over 32TB, it will get an error message 'unexpected data beyond EOF in block 0 of relation' in ReadBuffer_common. The '0 block' is from

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Amit Langote
On Tue, Jan 18, 2022 at 2:41 PM Julien Rouhaud wrote: > On Tue, Jan 18, 2022 at 02:33:39PM +0900, Michael Paquier wrote: > > On Tue, Jan 18, 2022 at 12:16:23PM +0800, Julien Rouhaud wrote: > > > I'm not sure why this test failed as it doesn't seem like something > > > impacted by > > > the

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Masahiko Sawada
On Tue, Jan 18, 2022 at 2:37 PM osumi.takami...@fujitsu.com wrote: > > On Tuesday, January 18, 2022 1:39 PM Masahiko Sawada > wrote: > > I've attached an updated patch. All comments I got so far were incorporated > > into this patch unless I'm missing something. > > Hi, thank you for your new

Re: Push down time-related SQLValue functions to foreign server

2022-01-17 Thread Corey Huinker
> > Hmm ... not really, because for these particular functions, the > point is exactly that we *don't* translate them to some function > call on the remote end. We evaluate them locally and push the > resulting constant to the far side, thus avoiding issues like > clock skew. > Ah, my pattern

Re: Proposal: More structured logging

2022-01-17 Thread Ronan Dunklau
Le lundi 17 janvier 2022, 09:18:04 CET Ronan Dunklau a écrit : > Le samedi 15 janvier 2022, 07:09:59 CET Julien Rouhaud a écrit : > > Hi, > > > > On Tue, Jan 11, 2022 at 11:05:26AM +0100, Ronan Dunklau wrote: > > > Done, and I added anoher commit per your suggestion to add this comment. > > > >

Re: Push down time-related SQLValue functions to foreign server

2022-01-17 Thread Tom Lane
Corey Huinker writes: > I'm very late to the party, but it seems to me that this effort is > describing a small subset of what "routine mapping" seems to be for: > defining function calls that can be pushed down to the foreign server, and > the analogous function on the foreign side. We may want

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Julien Rouhaud
Hi, On Tue, Jan 18, 2022 at 02:33:39PM +0900, Michael Paquier wrote: > On Tue, Jan 18, 2022 at 12:16:23PM +0800, Julien Rouhaud wrote: > > I'm not sure why this test failed as it doesn't seem like something > > impacted by > > the patch, but I may have missed something as I only had a quick look

RE: Skipping logical replication transactions on subscriber side

2022-01-17 Thread osumi.takami...@fujitsu.com
On Tuesday, January 18, 2022 1:39 PM Masahiko Sawada wrote: > I've attached an updated patch. All comments I got so far were incorporated > into this patch unless I'm missing something. Hi, thank you for your new patch v7. For your information, I've encountered a failure to apply patch v7 on

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Michael Paquier
On Tue, Jan 18, 2022 at 12:16:23PM +0800, Julien Rouhaud wrote: > I'm not sure why this test failed as it doesn't seem like something impacted > by > the patch, but I may have missed something as I only had a quick look at the > patch and discussion. This issue is discussed here:

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-17 Thread Peter Geoghegan
On Mon, Jan 17, 2022 at 8:13 PM Robert Haas wrote: > On Mon, Jan 17, 2022 at 5:41 PM Peter Geoghegan wrote: > > That just seems like semantics to me. The very next sentence after the > > one you quoted in your reply was "And so it's highly unlikely that any > > given VACUUM will ever

Re: Push down time-related SQLValue functions to foreign server

2022-01-17 Thread Corey Huinker
> The implementation of converting now() to CURRENT_TIMESTAMP > seems like an underdocumented kluge, too. > I'm very late to the party, but it seems to me that this effort is describing a small subset of what "routine mapping" seems to be for: defining function calls that can be pushed down to

Re: TAP test to cover "EndOfLogTLI != replayTLI" case

2022-01-17 Thread Kyotaro Horiguchi
At Tue, 18 Jan 2022 12:25:15 +0800, Julien Rouhaud wrote in > Hi, > > On Mon, Jan 17, 2022 at 09:33:57PM +0800, Julien Rouhaud wrote: > > > > Thanks for the updated patch! Note that thanks to Andres and Thomas work, > > you > > can now easily rely on the exact same CI than the cfbot on your

Re: row filtering for logical replication

2022-01-17 Thread Greg Nancarrow
On Tue, Jan 18, 2022 at 2:31 PM Amit Kapila wrote: > > On Tue, Jan 18, 2022 at 8:41 AM Greg Nancarrow wrote: > > > > On Tue, Jan 18, 2022 at 2:31 AM houzj.f...@fujitsu.com > > wrote: > > > > > > > (2) GetTopMostAncestorInPublication > > > > Is there a reason why there is no "break" after

Re: Null commitTS bug

2022-01-17 Thread Kyotaro Horiguchi
At Tue, 18 Jan 2022 10:43:55 +0900, Michael Paquier wrote in > On Sun, Jan 16, 2022 at 11:01:25PM -0500, Tom Lane wrote: > > Isn't that a very bad way to write "i = j + 1"? > > > > I agree with Horiguchi-san that > > for (i = 0, headxid = xid;;) > > Okay. Horiguchi-san, would you like to

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Masahiko Sawada
On Tue, Jan 18, 2022 at 12:20 PM osumi.takami...@fujitsu.com wrote: > > On Monday, January 17, 2022 9:52 PM Masahiko Sawada > wrote: > > Thank you for the comments! > .. > > > (2) Minor improvement suggestion of comment in > > > src/backend/replication/logical/worker.c > > > > > > + * reset

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Masahiko Sawada
On Tue, Jan 18, 2022 at 12:04 PM tanghy.f...@fujitsu.com wrote: > > On Mon, Jan 17, 2022 2:18 PM Masahiko Sawada wrote: > > > > I've attached an updated patch. Please review it. > > > > Thanks for updating the patch. Few comments: > > 1) > /* Two_phase is only supported in v15

Re: libpq compression (part 2)

2022-01-17 Thread Justin Pryzby
On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > => Since March, errmsg doesn't need extra parenthesis around it (e3a87b4). > I’ve resolved the stuck tests and added zlib support for CI Windows builds to > patch 0003-*. Thanks > for the suggestion, all tests seem to be OK

Re: TAP test to cover "EndOfLogTLI != replayTLI" case

2022-01-17 Thread Julien Rouhaud
Hi, On Mon, Jan 17, 2022 at 09:33:57PM +0800, Julien Rouhaud wrote: > > Thanks for the updated patch! Note that thanks to Andres and Thomas work, you > can now easily rely on the exact same CI than the cfbot on your own github > repository, if you need to debug something on a platform you don't

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Julien Rouhaud
Hi, On Mon, Jan 17, 2022 at 08:40:54PM +0900, Amit Langote wrote: > > Okay, I created versions of the patch series for branches 13 and 14 > (.txt files). The one for HEAD is also re-attached. FYI The patch failed today on FreeBSD, while it was previously quite stable on all platforms

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 5:41 PM Peter Geoghegan wrote: > That just seems like semantics to me. The very next sentence after the > one you quoted in your reply was "And so it's highly unlikely that any > given VACUUM will ever *completely* fail to advance relfrozenxid". > It's continuous *within*

Re: ICU for global collation

2022-01-17 Thread Julien Rouhaud
Hi, On Thu, Jan 13, 2022 at 09:39:42AM +0100, Peter Eisentraut wrote: > On 11.01.22 12:08, Julien Rouhaud wrote: > > > So, unless there are concerns, I'm going to see about making a patch to > > > call > > > pg_newlocale_from_collation() even with the default collation. That would > > > make the

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Masahiko Sawada
On Tue, Jan 18, 2022 at 12:37 PM Amit Kapila wrote: > > On Tue, Jan 18, 2022 at 8:34 AM tanghy.f...@fujitsu.com > wrote: > > > > On Mon, Jan 17, 2022 2:18 PM Masahiko Sawada wrote: > > > > > > > 2) The following two places are not consistent in whether "= value" is > > surround > > with square

Re: ICU for global collation

2022-01-17 Thread Julien Rouhaud
Hi, On Mon, Jan 17, 2022 at 07:07:38PM +, Finnerty, Jim wrote: > On 10.01.22 12:49, Daniel Verite wrote: > > > I think some users would want their db-wide ICU collation to be > > case/accent-insensitive. > ... > > IIRC, that was the context for some questions where people were > >

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Amit Kapila
On Tue, Jan 18, 2022 at 8:34 AM tanghy.f...@fujitsu.com wrote: > > On Mon, Jan 17, 2022 2:18 PM Masahiko Sawada wrote: > > > > 2) The following two places are not consistent in whether "= value" is > surround > with square brackets. > > +ALTER SUBSCRIPTION name SKIP ( > skip_option [=

Re: row filtering for logical replication

2022-01-17 Thread Amit Kapila
On Tue, Jan 18, 2022 at 8:41 AM Greg Nancarrow wrote: > > On Tue, Jan 18, 2022 at 2:31 AM houzj.f...@fujitsu.com > wrote: > > > > > (2) GetTopMostAncestorInPublication > > > Is there a reason why there is no "break" after finding a > > > topmost_relid? Why keep searching and potentially

RE: Skipping logical replication transactions on subscriber side

2022-01-17 Thread osumi.takami...@fujitsu.com
On Monday, January 17, 2022 9:52 PM Masahiko Sawada wrote: > Thank you for the comments! .. > > (2) Minor improvement suggestion of comment in > > src/backend/replication/logical/worker.c > > > > + * reset during that. Also, we don't skip receiving the changes in > > + streaming > > + * cases,

Re: row filtering for logical replication

2022-01-17 Thread Greg Nancarrow
On Tue, Jan 18, 2022 at 2:31 AM houzj.f...@fujitsu.com wrote: > > > (2) GetTopMostAncestorInPublication > > Is there a reason why there is no "break" after finding a > > topmost_relid? Why keep searching and potentially overwrite a > > previously-found topmost_relid? If it's intentional, I think

RE: Skipping logical replication transactions on subscriber side

2022-01-17 Thread tanghy.f...@fujitsu.com
On Mon, Jan 17, 2022 2:18 PM Masahiko Sawada wrote: > > I've attached an updated patch. Please review it. > Thanks for updating the patch. Few comments: 1) /* Two_phase is only supported in v15 and higher */ if (pset.sversion >= 15)

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Amit Langote
On Tue, Jan 18, 2022 at 7:15 AM Alvaro Herrera wrote: > On 2022-Jan-17, Tom Lane wrote: > > But could we please do it in a way that is designed to keep the > > code readable, rather than to minimize the number of lines of diff? > > It makes zero sense to have the bits in AFTER_TRIGGER_TUP_BITS

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Amit Kapila
On Tue, Jan 18, 2022 at 8:02 AM Masahiko Sawada wrote: > > On Mon, Jan 17, 2022 at 10:15 PM Amit Kapila wrote: > > > > On Mon, Jan 17, 2022 at 6:22 PM Masahiko Sawada > > wrote: > > > > > > > > > > > (5) > > > > > > > > I can miss something here but, in one of > > > > the past discussions,

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Masahiko Sawada
On Tue, Jan 18, 2022 at 10:36 AM Greg Nancarrow wrote: > > On Mon, Jan 17, 2022 at 5:18 PM Masahiko Sawada wrote: > > > > I've attached an updated patch. Please review it. > > > > Some review comments for the v6 patch: Thank you for the comments! > > > doc/src/sgml/logical-replication.sgml > >

Re: row filtering for logical replication

2022-01-17 Thread Amit Kapila
On Mon, Jan 17, 2022 at 9:00 PM houzj.f...@fujitsu.com wrote: > > On Mon, Jan 17, 2022 12:34 PM Peter Smith wrote: > > > > Here are some review comments for v65-0001 (review of updates since > > v64-0001) > > Thanks for the comments! > > > ~~~ > > > > 1. src/include/commands/publicationcmds.h -

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Masahiko Sawada
On Mon, Jan 17, 2022 at 10:15 PM Amit Kapila wrote: > > On Mon, Jan 17, 2022 at 6:22 PM Masahiko Sawada wrote: > > > > > > > > (5) > > > > > > I can miss something here but, in one of > > > the past discussions, there seems a consensus that > > > if the user specifies XID of a subtransaction, >

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-01-17 Thread Michael Paquier
On Sat, Jan 15, 2022 at 01:52:39PM +0800, Julien Rouhaud wrote: > libpq environment variable PGHOST has a non-local server value: > C:/Users/ContainerAdministrator/AppData/Local/Temp/FhBIlsw6SV > Failure, exiting > not ok 3 - run of pg_upgrade for new instance There are two things here, as far

Re: Null commitTS bug

2022-01-17 Thread Michael Paquier
On Sun, Jan 16, 2022 at 11:01:25PM -0500, Tom Lane wrote: > Isn't that a very bad way to write "i = j + 1"? > > I agree with Horiguchi-san that > for (i = 0, headxid = xid;;) Okay. Horiguchi-san, would you like to write a patch? -- Michael signature.asc Description: PGP signature

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-17 Thread James Coleman
On Mon, Jan 17, 2022 at 4:34 PM Alvaro Herrera wrote: > > On 2022-Jan-14, James Coleman wrote: > > > The logical slot can't flush past the > > last commit, so even if there's 100s of megabytes of unflushed WAL on > > the slot there may be zero lag (in terms of what's possible to > > process). > >

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-17 Thread James Coleman
On Mon, Jan 17, 2022 at 4:20 PM Robert Haas wrote: > > On Fri, Jan 14, 2022 at 7:42 PM James Coleman wrote: > > I've attached a simple patch (sans tests and documentation) to get > > feedback early. After poking around this afternoon it seemed to me > > that the simplest approach was to hook

Re: Skipping logical replication transactions on subscriber side

2022-01-17 Thread Greg Nancarrow
On Mon, Jan 17, 2022 at 5:18 PM Masahiko Sawada wrote: > > I've attached an updated patch. Please review it. > Some review comments for the v6 patch: doc/src/sgml/logical-replication.sgml (1) Expanded output Since the view output is shown in "expanded output" mode, perhaps the doc should say

Re: Refactoring of compression options in pg_basebackup

2022-01-17 Thread Michael Paquier
On Mon, Jan 17, 2022 at 12:48:12PM -0500, Robert Haas wrote: > Alvaro's proposal is fine with me. I don't see any value in replacing > --compress with --compression. It's longer but not superior in any way > that I can see. Having both seems worst of all -- that's just > confusing. Okay, that

Re: generic plans and "initial" pruning

2022-01-17 Thread Amit Langote
On Fri, Jan 14, 2022 at 11:10 PM Amit Langote wrote: > On Thu, Jan 6, 2022 at 3:45 PM Amul Sul wrote: > > Here are few comments for v1 patch: > > Thanks Amul. I'm thinking about Robert's latest comments addressing > which may need some rethinking of this whole design, but I decided to > post a

Re: row filtering for logical replication

2022-01-17 Thread Peter Smith
Here are some review comments for v66-0001 (review of updates since v65-0001) ~~~ 1. src/backend/catalog/pg_publication.c - GetTopMostAncestorInPublication @@ -276,17 +276,45 @@ GetPubPartitionOptionRelations(List *result, PublicationPartOpt pub_partopt, } /* + * Returns the relid of the

Re: docs: pg_replication_origin_oid() description does not match behaviour

2022-01-17 Thread Michael Paquier
On Tue, Jan 18, 2022 at 10:19:41AM +0900, Ian Lawrence Barwick wrote: > Given that the code has remained unchanged since the function was > introduced in 9.5, it seems reasonable to change the documentation > to match the function behaviour rather than the other way round. Obviously. I'll go fix

docs: pg_replication_origin_oid() description does not match behaviour

2022-01-17 Thread Ian Lawrence Barwick
Hi >From the documentation for pg_replication_origin_oid() [1]: > Looks up a replication origin by name and returns the internal ID. > If no such replication origin is found an error is thrown. However, it actually returns NULL if the origin does not exist: postgres=# SELECT * FROM

Re: drop tablespace failed when location contains .. on win32

2022-01-17 Thread Michael Paquier
On Tue, Jan 18, 2022 at 01:08:01AM +, wangsh.f...@fujitsu.com wrote: > Yes, I will send a new version before next weekend Thanks! -- Michael signature.asc Description: PGP signature

RE: drop tablespace failed when location contains .. on win32

2022-01-17 Thread wangsh.f...@fujitsu.com
Hi > This patch is a wanted bugfix and has been waiting for an update for 2 months. > > Do you plan to send a new version soon? Yes, I will send a new version before next weekend Regards Shenhao Wang

Re: \d with triggers: more than one row returned by a subquery used as an expression

2022-01-17 Thread Tom Lane
I wrote: > Justin Pryzby writes: >> Is there any reason why WITH ORDINALITY can't work ? >> This is passing the smoke test. > How hard did you try to break it? It still seems to me that > this can be fooled by an unrelated trigger with the same tgname. Hmm ... no, it does work, because we'll

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2022-01-17 Thread Jelte Fennema
It seems the man page of TCP_USER_TIMEOUT does not align with reality then. When I use it on my local machine it is effectively used as a connection timeout too. The second command times out after two seconds: sudo iptables -A INPUT -p tcp --destination-port 5432 -j DROP psql 'host=localhost

Re: \d with triggers: more than one row returned by a subquery used as an expression

2022-01-17 Thread Tom Lane
Justin Pryzby writes: > On Mon, Jan 17, 2022 at 05:02:00PM -0500, Tom Lane wrote: >> ISTM the real problem is the assumption that only related triggers could >> share a tgname, which evidently isn't true. I think this query needs to >> actually match on tgparentid, rather than taking shortcuts.

Re: \d with triggers: more than one row returned by a subquery used as an expression

2022-01-17 Thread Justin Pryzby
On Mon, Jan 17, 2022 at 05:02:00PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Dec 17, 2021 at 09:43:56AM -0600, Justin Pryzby wrote: > >> I want to mention that the 2nd problem I mentioned here is still broken. > >>

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Tom Lane
Alvaro Herrera writes: > On 2022-Jan-17, Tom Lane wrote: >> It makes zero sense to have the bits in AFTER_TRIGGER_TUP_BITS not >> be adjacent. So what should happen here is to renumber the symbols >> in between to move their bits over one place. > Is it typical to enumerate bits starting from

Re: Push down time-related SQLValue functions to foreign server

2022-01-17 Thread Tom Lane
Alexander Pyhalov writes: >> Perhaps in a MACRO? > Changed this check to a macro, also fixed condition in > is_foreign_param() and added test for it. > Also fixed comment in prepare_query_params(). I took a quick look at this. I'm unconvinced that you need the TIME_RELATED_SQLVALUE_FUNCTION

Re: [PATCH] reduce page overlap of GiST indexes built using sorted method

2022-01-17 Thread Björn Harrtell
Hi Aliaksandr, Nice work on this. I've been following it a bit since the regression when it was noted and it sparked renewed interest in R-tree structure and optimization for me. As for ideas. I'm not deep into details of postgresql and gist, but I've learned that the node size for gist indexes

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-17 Thread Peter Geoghegan
On Mon, Jan 17, 2022 at 2:13 PM Robert Haas wrote: > On Mon, Jan 17, 2022 at 4:28 PM Peter Geoghegan wrote: > > Updating relfrozenxid should now be thought of as a continuous thing, > > not a discrete thing. > > I think that's pretty nearly 100% wrong. The most simplistic way of > expressing

Re: Pluggable toaster

2022-01-17 Thread Nikita Malakhov
Hi, >This sounds interesting, but very much like column compression, which >was proposed some time ago. If we haven't made much progrees with that >patch (AFAICS), what's the likelihood we'll succeed here, when it's >combined with yet more complexity? The main concern is that this patch provides

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Alvaro Herrera
On 2022-Jan-17, Tom Lane wrote: > But could we please do it in a way that is designed to keep the > code readable, rather than to minimize the number of lines of diff? > It makes zero sense to have the bits in AFTER_TRIGGER_TUP_BITS not > be adjacent. So what should happen here is to renumber

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 4:28 PM Peter Geoghegan wrote: > Updating relfrozenxid should now be thought of as a continuous thing, > not a discrete thing. I think that's pretty nearly 100% wrong. The most simplistic way of expressing that is to say - clearly it can only happen when VACUUM runs,

Re: \d with triggers: more than one row returned by a subquery used as an expression

2022-01-17 Thread Tom Lane
Justin Pryzby writes: > On Fri, Dec 17, 2021 at 09:43:56AM -0600, Justin Pryzby wrote: >> I want to mention that the 2nd problem I mentioned here is still broken. >> https://www.postgresql.org/message-id/20210717010259.gu20...@telsasoft.com >> It happens if non-inheritted triggers on child and

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 4:34 PM Alvaro Herrera wrote: > On 2022-Jan-14, James Coleman wrote: > > The logical slot can't flush past the > > last commit, so even if there's 100s of megabytes of unflushed WAL on > > the slot there may be zero lag (in terms of what's possible to > > process). > > > >

Re: pg14 psql broke \d datname.nspname.relname

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 1:06 PM Mark Dilger wrote: > > On Jan 15, 2022, at 12:28 AM, Julien Rouhaud wrote: > > Could you send a rebased version? > Yes. Here it is: This is not a full review, but I just noticed that: + * dotcnt: how many separators were parsed from the pattern, by reference. +

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Tom Lane
Alvaro Herrera writes: > So this patch releases one bit from AFTER_TRIGGER_OFFSET and makes it > become AFTER_TRIGGER_CP_UPDATE. As far as I can tell there is no harm > in doing so. I agree that taking a bit away from AFTER_TRIGGER_OFFSET is okay (it could spare even a couple more, if we need

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-17 Thread Alvaro Herrera
On 2022-Jan-14, James Coleman wrote: > The logical slot can't flush past the > last commit, so even if there's 100s of megabytes of unflushed WAL on > the slot there may be zero lag (in terms of what's possible to > process). > > I've attached a simple patch (sans tests and documentation) to get

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-17 Thread Peter Geoghegan
On Mon, Jan 17, 2022 at 7:12 AM Robert Haas wrote: > On Thu, Jan 13, 2022 at 4:27 PM Peter Geoghegan wrote: > > 1. Cases where our inability to get a cleanup lock signifies nothing > > at all about the page in question, or any page in the same table, with > > the same workload. > > > > 2.

Re: a misbehavior of partition row movement (?)

2022-01-17 Thread Alvaro Herrera
> @@ -3398,7 +3432,7 @@ typedef SetConstraintStateData *SetConstraintState; > */ > typedef uint32 TriggerFlags; > > -#define AFTER_TRIGGER_OFFSET 0x0FFF /* must be > low-order bits */ > +#define AFTER_TRIGGER_OFFSET 0x07FF /* must be >

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-17 Thread Robert Haas
On Fri, Jan 14, 2022 at 7:42 PM James Coleman wrote: > I've attached a simple patch (sans tests and documentation) to get > feedback early. After poking around this afternoon it seemed to me > that the simplest approach was to hook into the commit timestamps > infrastructure and store the

Re: Adding CI to our tree

2022-01-17 Thread Andrew Dunstan
On 1/17/22 13:19, Andres Freund wrote: > Hi, > > On 2022-01-17 10:25:12 -0500, Andrew Dunstan wrote: >> The buildfarm is moving in the opposite direction, to disaggregate >> steps. > I'm a bit confused as to where you want changes to vcregress.pl > going. Upthread you argued against adding more

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 9:57 AM Shruthi Gowda wrote: > I have rebased and generated the patches on top of PostgreSQL commit > ID cf925936ecc031355cd56fbd392ec3180517a110. > Kindly apply v8-0001-pg_upgrade-Preserve-relfilenodes-and-tablespace-O.patch > first and then

Re: slowest tap tests - split or accelerate?

2022-01-17 Thread Andres Freund
Hi, On 2022-01-17 15:13:57 -0500, Robert Haas wrote: > I guess there must be something explaining it, but I don't know what > it could be. The client and the server are each running the checksum > algorithm over the same data. If that's not the same speed then I > don't get it. Unless,

Re: slowest tap tests - split or accelerate?

2022-01-17 Thread Tom Lane
Andres Freund writes: > I've occasionally pondered caching initdb results and reusing them across > tests - just the locking around it seems a bit nasty, but perhaps that could > be done as part of the tmp_install step. Of course, it'd need to deal with > different options etc... I'd actually

Re: removing datlastsysoid

2022-01-17 Thread Tom Lane
Robert Haas writes: > Since that doesn't seem like an especially good idea, PFA a patch to > remove it. Note that, even prior to that commit, it wasn't being used > for anything when dumping modern servers, so it would still have been > OK to remove it from the current system catalog structure.

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2022-01-17 Thread Tom Lane
Jelte Fennema writes: > Thanks for all the cleanup and adding of windows support. To me it now looks > good to merge. I was about to commit this when I started to wonder if it actually does anything useful. In particular, I read in the Linux tcp(7) man page TCP_USER_TIMEOUT (since

Re: missing indexes in indexlist with partitioned tables

2022-01-17 Thread Arne Roland
Hi! Afaiac the join pruning where the outer table is a partitioned table is the relevant case. I am not sure whether there are other cases. The join pruning, which works great for plain relations since 9.0, falls short for partitioned tables, since the optimizer fails to prove uniqueness

Re: Adding CI to our tree

2022-01-17 Thread Andres Freund
Hi, On 2022-01-17 14:30:53 -0500, Tom Lane wrote: > Andres Freund writes: > > I think it's not actually that hard, with something like I described in the > > email upthread, with each tests going into a prescribed location, and the > > on-disk status being inspectable in an automated way.

Re: slowest tap tests - split or accelerate?

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 2:57 PM Andres Freund wrote: > I wonder if there's something explaining why pg_verifybackup is greatly slowed > down by sha224 but not crc32c, but the server's runtime only differs by ~20ms? > It seems incongruous that pg_basebackup, with all the complexity of needing to >

removing datlastsysoid

2022-01-17 Thread Robert Haas
Hi, While reviewing another patch, I noticed that it slightly adjusted the treatment of datlastsysoid. That made me wonder what datlastsysoid is used for, so I started poking around and discovered that the answer, at least insofar as I can determine, is "nothing". The documentation claims that

Re: SLRUs in the main buffer pool, redux

2022-01-17 Thread Thomas Munro
On Mon, Jan 17, 2022 at 11:23 PM Heikki Linnakangas wrote: > IIRC one issue with this has been performance. When an SLRU is working > well, a cache hit in the SLRU is very cheap. Linearly scanning the SLRU > array is cheap, compared to computing the hash and looking up a buffer > in the buffer

Re: slowest tap tests - split or accelerate?

2022-01-17 Thread Andres Freund
Hi, On 2022-01-17 14:05:17 -0500, Robert Haas wrote: > On Mon, Jan 17, 2022 at 1:41 PM Andres Freund wrote: > > The reason these in particular are slow is that they do a lot of > > pg_basebackups without either / one-of -cfast / --no-sync. The lack of > > -cfast > > in particularly is

Re: Blank archive_command

2022-01-17 Thread Tom Lane
Robert Haas writes: > It might be nice to do something about the fact that you can't change > archive_mode without a server restart, though. I suspect we had a good > reason for that limitation from an engineering perspective, but from a > user perspective, it sucks pretty hard. Agreed. I don't

Re: Adding CI to our tree

2022-01-17 Thread Tom Lane
Andres Freund writes: > I think it's not actually that hard, with something like I described in the > email upthread, with each tests going into a prescribed location, and the > on-disk status being inspectable in an automated way. check-world could invoke > a command to summarize the tests at

Re: Adding CI to our tree

2022-01-17 Thread Andres Freund
Hi, On 2022-01-17 13:50:04 -0500, Robert Haas wrote: > On Mon, Jan 17, 2022 at 1:19 PM Andres Freund wrote: > > FWIW, to me this shouldn't require a lot of separate manual test > > invocations. And continuing to have lots of granular test invocations from > > the > > buildfarm client is *bad*,

Re: Pluggable toaster

2022-01-17 Thread Tomas Vondra
On 1/14/22 19:41, Teodor Sigaev wrote: In my understanding, we want to be able to 1. Access data from a toasted object one slice at a time, by using knowledge of the structure 2. If toasted data is updated, then update a minimum number of slices(s), without rewriting the existing slices 3.

Re: Adding CI to our tree

2022-01-17 Thread Tom Lane
Robert Haas writes: > I have a lot of sympathy with Andrew here, actually. If you just do > 'make check-world' and assume that will cover everything, you get one > giant output file. That is not great at all. Yeah. I agree with Andrew that we want output that is more modular, not less so. But

Re: ICU for global collation

2022-01-17 Thread Finnerty, Jim
On 10.01.22 12:49, Daniel Verite wrote: > I think some users would want their db-wide ICU collation to be > case/accent-insensitive. ... > IIRC, that was the context for some questions where people were > enquiring about db-wide ICU collations. +1. There is the DEFAULT_COLLATION_OID, which

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-17 Thread Robert Haas
On Tue, Dec 14, 2021 at 1:21 PM Shruthi Gowda wrote: > Thanks, Robert for the updated version. I reviewed the changes and it > looks fine. > I also tested the patch. The patch works as expected. Thanks. > > - I adjusted the function header comment for heap_create. Your > > proposed comment

Re: slowest tap tests - split or accelerate?

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 1:41 PM Andres Freund wrote: > The reason these in particular are slow is that they do a lot of > pg_basebackups without either / one-of -cfast / --no-sync. The lack of -cfast > in particularly is responsible for a significant proportion of the test > time. The only reason

Re: Adding CI to our tree

2022-01-17 Thread Robert Haas
On Mon, Jan 17, 2022 at 1:19 PM Andres Freund wrote: > FWIW, to me this shouldn't require a lot of separate manual test > invocations. And continuing to have lots of granular test invocations from the > buildfarm client is *bad*, because it requires constantly syncing up the set > of test

Re: slowest tap tests - split or accelerate?

2022-01-17 Thread Andres Freund
Hi, On 2021-12-31 11:25:28 -0800, Andres Freund wrote: > cfbot now runs most tests on windows, the windows task is by far the slowest, > and the task limitted most in concurrency [2]. Running tap tests is the > biggest part of that. This is a bigger issue on windows because we don't have >

  1   2   >