Re: Direct I/O

2023-01-24 Thread Bharath Rupireddy
On Thu, Dec 22, 2022 at 7:34 AM Thomas Munro wrote: > > On Wed, Dec 14, 2022 at 5:48 PM Thomas Munro wrote: > > 0001 -- David's palloc_aligned() patch > > https://commitfest.postgresql.org/41/3999/ > > 0002 -- I/O-align almost all buffers used for I/O > > 0003 -- Add the GUCs > > 0004 --

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 14:35:12 -0800, Andres Freund wrote in > > 0002: > > > > + Assert(pgstat_bktype_io_stats_valid(bktype_shstats, MyBackendType)); > > > > This is relatively complex checking. We already asserts-out increments > > of invalid counters. Thus this is checking if some unrelated

Re: drop postmaster symlink

2023-01-24 Thread Peter Eisentraut
On 12.01.23 20:11, Devrim Gündüz wrote: On Thu, 2023-01-12 at 13:35 -0500, Joe Conway wrote: To be clear, I am completely in agreement with you about removing the symlink. I just wanted to be sure Devrim was alerted because I knew he had a strong opinion on this topic ;-) Red Hat's own

Re: Record queryid when auto_explain.log_verbose is on

2023-01-24 Thread Michael Paquier
On Tue, Jan 24, 2023 at 11:01:46PM +0900, torikoshia wrote: > On 2023-01-23 09:35, Michael Paquier wrote: >> ExplainPrintTriggers() is kind of different because there is >> auto_explain_log_triggers. Still, we could add a flag in ExplainState >> deciding if the triggers should be printed, so as

Re: recovery modules

2023-01-24 Thread Michael Paquier
On Mon, Jan 23, 2023 at 01:44:28PM -0800, Nathan Bossart wrote: > On Mon, Jan 23, 2023 at 11:44:57AM +0900, Michael Paquier wrote: > I updated the documentation as you suggested, and I renamed basic_archive > to basic_wal_module. Thanks. The renaming of basic_archive to basic_wal_module looked

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart wrote in > On Tue, Jan 24, 2023 at 01:13:55PM -0500, Tom Lane wrote: > > Either that comment needs to be rewritten or we need to invent some > > more macros. > > Here is a first attempt at a patch. I scanned through all the existing > uses of

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Amit Kapila
On Wed, Jan 25, 2023 at 11:57 AM Kyotaro Horiguchi wrote: > > At Tue, 24 Jan 2023 12:19:04 +, "Takamichi Osumi (Fujitsu)" > wrote in > > Attached the patch v20 that has incorporated all comments so far. > ... > > > + in which case no additional wait is necessary. If the system

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Amit Kapila
On Wed, Jan 25, 2023 at 11:23 AM Takamichi Osumi (Fujitsu) wrote: > > > Thank you for checking the patch ! > On Wednesday, January 25, 2023 10:17 AM Kyotaro Horiguchi > wrote: > > In short, I'd like to propose renaming the parameter in_delayed_apply of > > send_feedback to

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > I have updated the patch accordingly and it looks good to me. I'll > push this first patch early next week (Monday) unless there are more > comments. Thanks for updating. I checked v88-0001 and I have no objection. LGTM. Best Regards, Hayato Kuroda FUJITSU LIMITED

Re: document the need to analyze partitioned tables

2023-01-24 Thread Laurenz Albe
On Wed, 2023-01-25 at 16:26 +1300, David Rowley wrote: > On Wed, 18 Jan 2023 at 22:15, Laurenz Albe wrote: > > Attached is a new version of my patch that tries to improve the wording. > > I had a look at this and agree that we should adjust the paragraph in > question if people are finding it

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Amit Kapila
On Wed, Jan 25, 2023 at 10:05 AM Amit Kapila wrote: > > On Wed, Jan 25, 2023 at 3:15 AM Peter Smith wrote: > > > > 1. > > @@ -210,7 +210,7 @@ int logical_decoding_work_mem; > > static const Size max_changes_in_memory = 4096; /* XXX for restore only */ > > > > /* GUC variable */ > > -int

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 12:19:04 +, "Takamichi Osumi (Fujitsu)" wrote in > Attached the patch v20 that has incorporated all comments so far. Thanks! I looked thourgh the documentation part. + + subminapplydelay int8 + + + Total time spent delaying the

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
Hi, Horiguchi-san Thank you for checking the patch ! On Wednesday, January 25, 2023 10:17 AM Kyotaro Horiguchi wrote: > In short, I'd like to propose renaming the parameter in_delayed_apply of > send_feedback to "has_unprocessed_change". > > At Tue, 24 Jan 2023 12:27:58 +0530, Amit Kapila >

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
Hi, On Wednesday, January 25, 2023 2:02 PM shveta malik wrote: > On Tue, Jan 24, 2023 at 5:49 PM Takamichi Osumi (Fujitsu) > wrote: > > > > > > Attached the patch v20 that has incorporated all comments so far. > > Kindly have a look at the attached patch. > Thank You for patch. My previous

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Michael Paquier
On Wed, Jan 25, 2023 at 10:32:10AM +0900, Michael Paquier wrote: > Thanks, my memory was fuzzy regarding that. I am curious if the error > in the recovery tests will persist with that set up. The next run > will be in a few hours, so let's see.. So it looks like tanaget is able to reproduce the

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread shveta malik
On Tue, Jan 24, 2023 at 5:49 PM Takamichi Osumi (Fujitsu) wrote: > > > Attached the patch v20 that has incorporated all comments so far. > Kindly have a look at the attached patch. > > > Best Regards, > Takamichi Osumi > Thank You for patch. My previous comments are addressed. Tested it

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-24 Thread Andres Freund
Hi, On 2023-01-23 19:22:18 -0800, Peter Geoghegan wrote: > On Mon, Jan 23, 2023 at 6:56 PM Andres Freund wrote: > > Why is there TABLE_ in AUTOVACUUM_TABLE_XID_AGE but not > > AUTOVACUUM_DEAD_TUPLES? Both are on tables. > > Why does vacuum_freeze_table_age contain the word "table", while >

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-01-24 Thread mahendrakar s
Hi All, The "issuer" field has been removed to align with the RFC implementation - https://www.rfc-editor.org/rfc/rfc7628. This patch "v6" is a single patch to support the OAUTH BEARER token through psql connection string. Below flow is supported. Added the documentation in the commit messages.

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Tom Lane
Andres Freund writes: > Patches attached. +1 for 0001. I'm still nervous about 0002. However, maybe the cases that we had trouble with are legacy issues that nobody cares about anymore in 2023. We can always look for another answer if we get complaints, I guess.

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Amit Kapila
On Wed, Jan 25, 2023 at 3:15 AM Peter Smith wrote: > > 1. > @@ -210,7 +210,7 @@ int logical_decoding_work_mem; > static const Size max_changes_in_memory = 4096; /* XXX for restore only */ > > /* GUC variable */ > -int logical_decoding_mode = LOGICAL_DECODING_MODE_BUFFERED; > +int

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 18:32:46 -0800, Andres Freund wrote: > > I ran out of energy to test on aix with xlc, I spent way more time on this > > than I have already. I'll pick it up later. > > Building in the background now. Also passes. Thus I think getting rid of the #undefines is the best plan

Re: Fix to enum hashing for dump and restore

2023-01-24 Thread Tom Lane
Andrew writes: > I have discovered a bug in one usage of enums. If a table with hash > partitions uses an enum as a partitioning key, it can no longer be > backed up and restored correctly. This is because enums are represented > simply as oids, and the hash function for enums hashes that oid to

Fix to enum hashing for dump and restore

2023-01-24 Thread Andrew
Hello, I have discovered a bug in one usage of enums. If a table with hash partitions uses an enum as a partitioning key, it can no longer be backed up and restored correctly. This is because enums are represented simply as oids, and the hash function for enums hashes that oid to determine

Re: document the need to analyze partitioned tables

2023-01-24 Thread David Rowley
On Wed, 18 Jan 2023 at 22:15, Laurenz Albe wrote: > Attached is a new version of my patch that tries to improve the wording. I had a look at this and agree that we should adjust the paragraph in question if people are finding it confusing. For your wording, I found I had a small problem with

Re: GUCs to control abbreviated sort keys

2023-01-24 Thread Robert Haas
On Sat, Jan 21, 2023 at 8:16 PM Jeff Davis wrote: > This is fairly simple, so I plan to commit soon. I find it a bit premature to include this comment in the very first email what if other people don't like the idea? I would like to hear about the cases where abbreviated keys resulted in a

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 17:48:56 -0800, Andres Freund wrote: > Also for autoconf, I needed to link > $prefix/lib/python3.11/config-3.11/libpython3.11.a > to > $prefix/lib/libpython3.11.a > That might be a python version difference or be related to building python > with --enable-shared - but I see saw

Re: BF animal malleefowl reported an failure in 001_password.pl

2023-01-24 Thread Thomas Munro
On Fri, Jan 20, 2023 at 9:16 AM Tom Lane wrote: > Thomas Munro writes: > > So I think we probably need something like the attached, which I was > > originally trying to avoid. > > Yeah, something like that. I also wonder if you don't need to think > a bit harder about the ordering of the flag

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 16:16:06 -0500, Tom Lane wrote: > Andres Freund writes: > > Python's _POSIX_C_SOURCE value is set to a specific value in their configure > > script: > > > if test $define_xopen_source = yes > > then > > ... > > AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
Sorry for making you bothered by this. At Tue, 24 Jan 2023 10:12:40 +, "Hayato Kuroda (Fujitsu)" wrote in > > > Couldn't we maintain an additional static variable "last_applied" > > > along with last_received? > > > > > > > It won't be easy to maintain the meaning of last_applied because

Re: GUCs to control abbreviated sort keys

2023-01-24 Thread Justin Pryzby
On Sat, Jan 21, 2023 at 05:16:01PM -0800, Jeff Davis wrote: > + xreflabel="sort_abbreviated_keys"> > + sort_abbreviated_keys (boolean) > + > + sort_abbreviated_keys configuration > parameter > + > + > + > + > +Enables or disables the use of

bitscan forward/reverse on Windows

2023-01-24 Thread John Naylor
Attached is a quick-and-dirty attempt to add MSVC support for the rightmost/leftmost-one-pos functions. 0001 adds asserts to the existing coding. 0002 adds MSVC support. Tests pass on CI, but it's of course possible that there is some bug that prevents hitting the fastpath. I've mostly used the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-24 Thread Masahiko Sawada
On Mon, Jan 23, 2023 at 8:20 PM John Naylor wrote: > > On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote: > > > > On Mon, Jan 16, 2023 at 2:02 PM John Naylor > > wrote: > > In v21, all of your v20 improvements to the radix tree template and test have > been squashed into 0003, with one

Re: Add LZ4 compression in pg_dump

2023-01-24 Thread Justin Pryzby
On Tue, Jan 24, 2023 at 03:56:20PM +, gkokola...@pm.me wrote: > On Monday, January 23rd, 2023 at 7:00 PM, Justin Pryzby > wrote: > > On Mon, Jan 23, 2023 at 05:31:55PM +, gkokola...@pm.me wrote: > > > > > Please find attached v23 which reintroduces the split. > > > > > > 0001 is

Re: Improve logging when using Huge Pages

2023-01-24 Thread Justin Pryzby
On Mon, Jan 23, 2023 at 05:33:35PM -0800, Andres Freund wrote: > Hi, > > On 2023-01-23 19:21:00 -0600, Justin Pryzby wrote: > > Michael seemed to support this idea and nobody verbalized hatred of it, > > so I implemented it. In v15, we have shared_memory_size_in_huge_pages, > > so this adds

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Michael Paquier
On Tue, Jan 24, 2023 at 07:42:06PM -0500, Tom Lane wrote: > That systemd behavior affects IPC resources regardless of what process > created them. Thanks, my memory was fuzzy regarding that. I am curious if the error in the recovery tests will persist with that set up. The next run will be in a

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 14:22:19 +0530, Amit Kapila wrote in > On Tue, Jan 24, 2023 at 12:44 PM Peter Smith wrote: > > > > On Tue, Jan 24, 2023 at 5:58 PM Amit Kapila wrote: > > > > > > On Tue, Jan 24, 2023 at 8:15 AM Kyotaro Horiguchi > > > wrote: > > > > > > > > > Attached the updated patch

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
In short, I'd like to propose renaming the parameter in_delayed_apply of send_feedback to "has_unprocessed_change". At Tue, 24 Jan 2023 12:27:58 +0530, Amit Kapila wrote in > > send_feedback(): > > +* If the subscriber side apply is delayed (because of time-delayed > > +*

Re: heapgettup refactoring

2023-01-24 Thread Melanie Plageman
On Tue, Jan 24, 2023 at 04:17:23PM -0500, Melanie Plageman wrote: > Thanks for taking a look! > > On Mon, Jan 23, 2023 at 6:08 AM David Rowley wrote: > > > > On Thu, 19 Jan 2023 at 00:04, Peter Eisentraut > > wrote: > > > In your v2 patch, you remove these assertions: > > > > > > - /*

heapgettup() with NoMovementScanDirection unused in core?

2023-01-24 Thread Melanie Plageman
Hi, David Rowley and I were discussing how to test the NoMovementScanDirection case for heapgettup() and heapgettup_pagemode() in [1] (since there is not currently coverage). We are actually wondering if it is dead code (in core). This is a link to the code in question on github in [2] (side

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Tom Lane
Michael Paquier writes: > On Wed, Jan 25, 2023 at 01:20:39PM +1300, Thomas Munro wrote: >> Something to do with >> https://www.postgresql.org/docs/current/kernel-resources.html#SYSTEMD-REMOVEIPC >> ? > Still this is unrelated? This is a buildfarm instance, so the backend > does not run with

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Michael Paquier
On Wed, Jan 25, 2023 at 01:20:39PM +1300, Thomas Munro wrote: > Something to do with > https://www.postgresql.org/docs/current/kernel-resources.html#SYSTEMD-REMOVEIPC > ? Still this is unrelated? This is a buildfarm instance, so the backend does not run with systemd. > The failure I saw looked

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Thomas Munro
On Wed, Jan 25, 2023 at 1:02 PM Michael Paquier wrote: > Well, this host has a problem, for what looks like a kernel issue, I > guess.. This is repeatable across all the branches, randomly, with > various errors with the POSIX DSM implementation: > # [63cf68b7.5e5a:1] ERROR: could not open

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Kyotaro Horiguchi
At Tue, 24 Jan 2023 11:45:36 +0530, Amit Kapila wrote in > Personally, I would prefer the above LOGs to be in reverse order as it > doesn't make much sense to me to first say that we are skipping > changes and then say the transaction is delayed. What do you think? In the first place, I

Re: Generating code for query jumbling through gen_node_support.pl

2023-01-24 Thread Michael Paquier
On Tue, Jan 24, 2023 at 03:57:56PM +0900, Michael Paquier wrote: > Makes sense. That would be my intention if 0004 is the most > acceptable and splitting things makes things a bit easier to review. There was a silly mistake in 0004 where the jumbling code relied on compute_query_id rather than

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Michael Paquier
On Wed, Jan 25, 2023 at 12:40:02PM +1300, Thomas Munro wrote: > I remembered this thread after seeing the failure of Michael's new > build farm animal "tanager". I think we need to solve this somehow... Well, this host has a problem, for what looks like a kernel issue, I guess.. This is

Re: suppressing useless wakeups in logical/worker.c

2023-01-24 Thread Tom Lane
Nathan Bossart writes: > [ v2-0001-suppress-unnecessary-wakeups-in-logical-worker.c.patch ] I took a look through this, and have a number of mostly-cosmetic issues: * It seems wrong that next_sync_start isn't handled as one of the wakeup[NUM_LRW_WAKEUPS] entries. I see that it needs to be

Re: 011_crash_recovery.pl intermittently fails

2023-01-24 Thread Thomas Munro
On Mon, Mar 8, 2021 at 9:32 PM Kyotaro Horiguchi wrote: > At Sun, 07 Mar 2021 20:09:33 -0500, Tom Lane wrote in > > Thomas Munro writes: > > > Thanks! I'm afraid I wouldn't get around to it for a few weeks, so if > > > you have time, please do. (I'm not sure if it's strictly necessary to > >

Getting relations accessed by a query using the raw query string

2023-01-24 Thread Amin
Hi, Having a query string, I am trying to use the postgres parser to find which relations the query accesses. This is what I currently have: const char *query_string="select * from dummytable;"; List *parsetree_list=pg_parse_query(query_string); ListCell *parsetree_item;

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Peter Smith
Here are my review comments for patch v87-0002. == doc/src/sgml/config.sgml 1. -Allows streaming or serializing changes immediately in logical decoding. The allowed values of logical_replication_mode are -buffered and immediate. When set -to

Re: Mutable CHECK constraints?

2023-01-24 Thread Laurenz Albe
On Tue, 2023-01-24 at 01:38 -0500, Tom Lane wrote: > Laurenz Albe writes: > > We throw an error if the expression in a CREATE INDEX statement is not > > IMMUTABLE. > > But while the documentation notes that expressions in CHECK constraints are > > not > > to be immutable, we don't enforce that. 

Re: Update comments in multixact.c

2023-01-24 Thread Peter Geoghegan
On Wed, Jan 18, 2023 at 2:02 AM shiy.f...@fujitsu.com wrote: > Thanks for your reply. > > Attach a patch which fixed them. Pushed something close to that just now. I decided that it was better to not specify when truncation happened in these two places at all, though. The important detail is

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 17:22:03 +0900, Kyotaro Horiguchi wrote: > Hello. > > At Thu, 19 Jan 2023 21:15:34 -0500, Melanie Plageman > wrote in > > Oh dear-- an extra FlushBuffer() snuck in there somehow. > > Removed it in attached v51. > > Also, I fixed an issue in my tablespace.sql updates > > I

Re: Non-superuser subscription owners

2023-01-24 Thread Robert Haas
On Mon, Jan 23, 2023 at 3:50 PM Jeff Davis wrote: > I believe your patch conflates two use cases: > > (A) Tightly-coupled servers that are managed by the administrator. In > this case, there are a finite number of connection strings to make, and > the admin knows about all of them. Validation is

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-24 Thread Peter Smith
On Tue, Jan 24, 2023 at 11:49 PM houzj.f...@fujitsu.com wrote: > ... > > Sorry, the patch set was somehow attached twice. Here is the correct new > version > patch set which addressed all comments so far. > Here are my review comments for patch v87-0001. ==

Re: heapgettup refactoring

2023-01-24 Thread Melanie Plageman
Thanks for taking a look! On Mon, Jan 23, 2023 at 6:08 AM David Rowley wrote: > > On Thu, 19 Jan 2023 at 00:04, Peter Eisentraut > wrote: > > In your v2 patch, you remove these assertions: > > > > - /* check that rs_cindex is in sync */ > > - Assert(scan->rs_cindex <

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Tom Lane
Andres Freund writes: > Python's _POSIX_C_SOURCE value is set to a specific value in their configure > script: > if test $define_xopen_source = yes > then > ... > AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features from IEEE > Stds 1003.1-2008) > fi Hm. I looked into Python

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Andrew Dunstan
On 2023-01-24 Tu 13:42, Jelte Fennema wrote: >> Here's another improvement I think will be useful when the new gadgets >> are used in a git hook: first, look for the excludes file under the >> current directory if we aren't setting $code_base (e.g if we have files >> given on the command line),

Re: Making Vars outer-join aware

2023-01-24 Thread David G. Johnston
On Tue, Jan 24, 2023 at 1:25 PM Tom Lane wrote: > "David G. Johnston" writes: > > On Tue, Jan 24, 2023 at 12:31 PM Tom Lane wrote: > >> select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1); > >> > >> If we turn the generic equivclass.c logic loose on these clauses, > >> it will deduce

Re: Reducing power consumption on idle servers

2023-01-24 Thread Tom Lane
Thomas Munro writes: > Yeah, I definitely want to fix it. I just worry that 60s is so long > that it also needs that analysis work to be done to explain that it's > OK that we're a bit sloppy on noticing when to wake up, at which point > you might as well go to infinity. Yeah. The

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-24 Thread Peter Geoghegan
On Tue, Jan 24, 2023 at 11:21 AM Robert Haas wrote: > > The whole article was about how this DROP TRIGGER pattern worked just > > fine most of the time, because most of the time autovacuum was just > > autocancelled. They say this at one point: > > > > "The normal autovacuum mechanism is skipped

Re: Making Vars outer-join aware

2023-01-24 Thread Tom Lane
"David G. Johnston" writes: > On Tue, Jan 24, 2023 at 12:31 PM Tom Lane wrote: >> select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1); >> >> If we turn the generic equivclass.c logic loose on these clauses, >> it will deduce t2.y = 1, which is good, and then apply t2.y = 1 at >> the

postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-24 Thread Robert Haas
[ Changing subject line to something more appropriate: This is branched from the "Non-superuser subscription owners" thread, but the topic has become connection security more generally for outbound connections from a PostgreSQL instance, the inadequacies of just trying to require that such

Re: Reducing power consumption on idle servers

2023-01-24 Thread Thomas Munro
On Wed, Nov 30, 2022 at 7:40 PM Simon Riggs wrote: > On Wed, 30 Nov 2022 at 03:50, Thomas Munro wrote: > > I'm just curious, and not suggesting that 60s wakeups are a problem > > for the polar ice caps, but why even time out at all? Are the latch > > protocols involved not reliable enough? At

Re: Making Vars outer-join aware

2023-01-24 Thread David G. Johnston
On Tue, Jan 24, 2023 at 12:31 PM Tom Lane wrote: > I wrote: > > Hans Buschmann writes: > >> I just noticed your new efforts in this area. > >> I wanted to recurr to my old thread [1] considering constant > propagation of quals. > >> [1] >

Re: Making Vars outer-join aware

2023-01-24 Thread Tom Lane
I wrote: > Hans Buschmann writes: >> I just noticed your new efforts in this area. >> I wanted to recurr to my old thread [1] considering constant propagation of >> quals. >> [1] https://www.postgresql.org/message-id/1571413123735.26...@nidsa.net > Yeah, this patch series is not yet quite up

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-24 Thread Robert Haas
On Fri, Jan 20, 2023 at 4:24 PM Peter Geoghegan wrote: > > It sounds like they used DROP TRIGGER pretty regularly. So I think this > > sounds like exactly the kind of case I was talking about, where > > autovacuums keep getting cancelled until we decide to stop cancelling > > them. > > I don't

Re: Unicode grapheme clusters

2023-01-24 Thread Bruce Momjian
On Tue, Jan 24, 2023 at 11:40:01AM -0500, Greg Stark wrote: > On Sat, 21 Jan 2023 at 13:17, Tom Lane wrote: > > > > Probably our long-term answer is to avoid depending on wcwidth > > and use wcswidth instead. But it's hard to get excited about > > doing the legwork for that until popular libc

Re: Non-superuser subscription owners

2023-01-24 Thread Jacob Champion
On Tue, Jan 24, 2023 at 5:50 AM Robert Haas wrote: > I think this has some potential, but it's pretty complex, seeming to > require protocol extensions and having backward-compatibility problems > and so on. Yeah. > What do you think about something in the spirit of a > reverse-pg_hba.conf? The

Re: HOT chain validation in verify_heapam()

2023-01-24 Thread Robert Haas
On Sun, Jan 22, 2023 at 10:19 AM Himanshu Upadhyaya wrote: > I was trying to use lp_valid as I need to identify the root of the HOT chain > and we are doing validation on the root of the HOT chain when we loop over > the predecessor array. > Was resetting lp_valid in the last patch because we

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, On 2023-01-24 12:55:15 -0500, Tom Lane wrote: > Andres Freund writes: > > The background for the undefines is that _POSIX_C_SOURCE needs to be defined > > the same for the whole compilation, not change in the middle, and Python.h > > defines it. To protect "our" parts a11cf433413 instituted

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Bruce Momjian
On Tue, Jan 24, 2023 at 09:54:57AM -0500, Tom Lane wrote: > As another example, the mechanisms we use to create the typedefs list > in the first place are pretty squishy/leaky: they depend on which > buildfarm animals are running the typedef-generation step, and on > whether anything's broken

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Jelte Fennema
> I think that would be undesirable, because then reindentation noise > in completely-unrelated files would get baked into feature commits, > complicating review and messing up "git blame" history. With a rebase hook similar to the the pre-commit hook that I shared upthread, your files will be

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Nathan Bossart
On Tue, Jan 24, 2023 at 01:13:55PM -0500, Tom Lane wrote: > Either that comment needs to be rewritten or we need to invent some > more macros. Here is a first attempt at a patch. I scanned through all the existing uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything else

Re: Helper functions for wait_for_catchup() in Cluster.pm

2023-01-24 Thread Alvaro Herrera
Looking again, I have two thoughts for making things easier: 1. I don't think wait_for_write_catchup is necessary, because calling wait_for_catchup() and omitting the 'mode' and 'lsn' arguments would already do the same thing. So what we should do is patch places that currently give those two

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Tom Lane
Nathan Bossart writes: > IMO ideally there should be a DSA_HANDLE_INVALID and DSHASH_HANDLE_INVALID > for use with dsa_handle and dshash_table_handle, respectively. But your > patch does seem like an improvement. Yeah, particularly given that dsa.h says /* * The handle for a dsa_area is

Re: plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Tom Lane
Andres Freund writes: > The background for the undefines is that _POSIX_C_SOURCE needs to be defined > the same for the whole compilation, not change in the middle, and Python.h > defines it. To protect "our" parts a11cf433413 instituted the rule that all > postgres headers have to be included

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-24 Thread Ankit Kumar Pandey
I think more benchmarking is required so we can figure out if this is a corner case or a common case I did some more benchmarks: #1. AIM: Pushdown column whose size is very high create table test(a int, b int, c text); insert into test select a,b,c from generate_series(1,1000)a,

Re: Non-superuser subscription owners

2023-01-24 Thread Andrew Dunstan
On 2023-01-24 Tu 08:50, Robert Haas wrote: > > What do you think about something in the spirit of a > reverse-pg_hba.conf? The idea being that PostgreSQL facilities that > make outbound connections are supposed to ask it whether those > connections are OK to initiate. Then you could have a

Re: psql: Add role's membership options to the \du+ command

2023-01-24 Thread David G. Johnston
On Mon, Jan 9, 2023 at 9:09 AM Pavel Luzanov wrote: > When you include one role in another, you can specify three options: > ADMIN, INHERIT (added in e3ce2de0) and SET (3d14e171). > > For example. > > CREATE ROLE alice LOGIN; > > GRANT pg_read_all_settings TO alice WITH ADMIN TRUE, INHERIT TRUE,

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-24 Thread Nathan Bossart
On Tue, Jan 24, 2023 at 02:55:07AM +, houzj.f...@fujitsu.com wrote: > I noticed one minor thing in this commit. > > - > LogicalRepCtx->last_start_dsh = DSM_HANDLE_INVALID; > - > > The code takes the last_start_dsh as dsm_handle, but it seems it is a > dsa_pointer. > " typedef dsa_pointer

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Andrew Dunstan
On 2023-01-24 Tu 11:43, Tom Lane wrote: > Jelte Fennema writes: >> Sounds like this conflict could be handled fairly easily by >> having a local git hook rerunning pgindent whenever >> you rebase a commit: >> 1. if you changed typedefs.list the hook would format all files >> 2. if you didn't it

plpython vs _POSIX_C_SOURCE

2023-01-24 Thread Andres Freund
Hi, A recent commit of mine [1] broke compilation of plpython on AIX [2]. But my commit turns out to only be very tangentially related - it only causes a failure because it references clock_gettime() in an inline function instead of a macro and, as it turns out, plpython currently breaks

Re: cutting down the TODO list thread

2023-01-24 Thread Bruce Momjian
On Tue, Jan 24, 2023 at 10:46:34AM +0700, John Naylor wrote: > > On Wed, Jan 18, 2023 at 3:13 AM Bruce Momjian wrote: > > > I think we risk overloading people with too many words above, and they > > will not read it fully.  Can it be simplified?  I wonder if some of this > > belows in the

Re: Unicode grapheme clusters

2023-01-24 Thread Isaac Morland
On Tue, 24 Jan 2023 at 11:40, Greg Stark wrote: > > At the end of the day Unicode kind of assumes a variable-width display > where the rendering is handled by something that has access to the > actual font metrics. So anything trying to line things up in columns > in a way that works with any

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Tom Lane
Jelte Fennema writes: > Sounds like this conflict could be handled fairly easily by > having a local git hook rerunning pgindent whenever > you rebase a commit: > 1. if you changed typedefs.list the hook would format all files > 2. if you didn't it only formats the files that you changed I think

Re: Unicode grapheme clusters

2023-01-24 Thread Greg Stark
On Sat, 21 Jan 2023 at 13:17, Tom Lane wrote: > > Probably our long-term answer is to avoid depending on wcwidth > and use wcswidth instead. But it's hard to get excited about > doing the legwork for that until popular libc implementations > get it right. Here's an interesting blog post about

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Jelte Fennema
> As a concrete example, suppose Alice commits some code that uses "foo" > as a variable name, and more or less concurrently, Bob commits something > that defines "foo" as a typedef. Bob's change is likely to have > side-effects on the formatting of Alice's code. If they're working in >

Re: CREATEROLE users vs. role properties

2023-01-24 Thread Robert Haas
On Tue, Jan 24, 2023 at 9:07 AM tushar wrote: > right, Neha/I have tested with different scenarios using > createdb/replication/bypassrls and other > privileges properties on the role. also checked pg_dumpall/pg_basebackup and > everything looks fine. Thanks. I have committed the patch. --

Re: old_snapshot_threshold bottleneck on replica

2023-01-24 Thread Robert Haas
On Mon, Jan 23, 2023 at 9:40 AM Maxim Orlov wrote: > One of our customers stumble onto a significant performance degradation while > running multiple OLAP-like queries on a replica. > After some investigation, it became clear that the problem is in accessing > old_snapshot_threshold parameter.

Re: pgindent vs variable declaration across multiple lines

2023-01-24 Thread Tom Lane
Andrew Dunstan writes: > On 2023-01-22 Su 17:34, Tom Lane wrote: >> I've also attached a diff >> representing the delta between what current pg_bsd_indent wants to do >> to HEAD and what this would do. All the changes it wants to make look >> good, although I can't say whether there are other

Re: Making Vars outer-join aware

2023-01-24 Thread Tom Lane
Hans Buschmann writes: > I just noticed your new efforts in this area. > I wanted to recurr to my old thread [1] considering constant propagation of > quals. > [1] https://www.postgresql.org/message-id/1571413123735.26...@nidsa.net Yeah, this patch series is not yet quite up to the point of

Re: pgindent vs variable declaration across multiple lines

2023-01-24 Thread Andrew Dunstan
On 2023-01-22 Su 17:34, Tom Lane wrote: > I've also attached a diff > representing the delta between what current pg_bsd_indent wants to do > to HEAD and what this would do. All the changes it wants to make look > good, although I can't say whether there are other places it's failing > to

Re: Improve GetConfigOptionValues function

2023-01-24 Thread Tom Lane
Bharath Rupireddy writes: > On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: >> Also, I intentionally dropped the GUC_NO_SHOW_ALL check in >> get_explain_guc_options, because it seems redundant given >> the preceding GUC_EXPLAIN check. It's unlikely we'd ever have >> a variable that's marked

Re: to_hex() for negative inputs

2023-01-24 Thread Dean Rasheed
On Tue, 24 Jan 2023 at 13:43, Aleksander Alekseev wrote: > > Adding extra arguments for something the user can implement > (him/her)self doesn't seem to be a great idea. With this approach we > may end up with hundreds of arguments one day. > I don't see how a couple of extra arguments will

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Vladimir Sitnikov
Andres> Of course, but I somehow feel a change of formatting should be reviewable to Andres> at least some degree One way of reviewing the formatting changes is to compare the compiled binaries. If the binaries before and after formatting are the same, then there's a high chance the behaviour is

Re: Minimal logical decoding on standbys

2023-01-24 Thread Drouvot, Bertrand
Hi, On 1/24/23 12:21 AM, Melanie Plageman wrote: I'm new to this thread and subject, but I had a few basic thoughts about the first patch in the set. Thanks for looking at it! On Mon, Jan 23, 2023 at 12:03:35PM +0100, Drouvot, Bertrand wrote: Please find V42 attached. From

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread Takamichi Osumi (Fujitsu)
On Tuesday, January 24, 2023 8:32 AM Euler Taveira wrote: > Good to know that you keep improving this patch. I have a few suggestions that > were easier to provide a patch on top of your latest patch than to provide an > inline suggestions. Thanks for your review ! We basically adopted your

Re: run pgindent on a regular basis / scripted manner

2023-01-24 Thread Tom Lane
Jelte Fennema writes: >> One issue on requiring patches to have run pgindent previously is >> actually the typedef list. If someone adds a typedef in a commit, they >> will see different pgident output in the committed files, and perhaps >> others, and the new typedefs might only appear after

Re: Improve GetConfigOptionValues function

2023-01-24 Thread Bharath Rupireddy
On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > LGTM. I've marked it RfC. > > After looking at this, it seemed to me that the factorization > wasn't quite right after all: specifically, the new function > could be used in several more places if it confines

Re: Exclusion constraints on partitioned tables

2023-01-24 Thread Ronan Dunklau
Le vendredi 16 décembre 2022, 06:11:49 CET Paul Jungwirth a écrit : > On 12/15/22 16:12, Tom Lane wrote: > >> This patch also requires the matching constraint columns to use equality > >> comparisons (`(foo WITH =)`), so it is really equivalent to the existing > >> b-tree rule. > > > > That's not

  1   2   >