Re: POC: Parallel processing of indexes in autovacuum

2025-07-21 Thread Daniil Davydov
Hi, On Mon, Jul 21, 2025 at 11:40 PM Sami Imseih wrote: > > I have only reviewed the v8-0001-Parallel-index-autovacuum.patch so far and > have a few comments from my initial pass. > > 1/ Please run pgindent. OK, I'll do it. > 2/ Documentation is missing. There may be more, but here are the plac

Re: CAST(... ON DEFAULT) - WIP build on top of Error-Safe User Functions

2025-07-21 Thread Vik Fearing
On 22/07/2025 03:59, jian he wrote: Based on my reading of [4], it seems CAST(EXPRESSION AS TYPE DEFAULT def_expr ON ERROR) is not included in SQL:2023. [4]https://peter.eisentraut.org/blog/2023/04/04/sql-2023-is-finished-here-is-whats-new It was accepted into the standard after 2023 was re

Support getrandom() for pg_strong_random() source

2025-07-21 Thread Masahiko Sawada
Hi all, Currently we have three options for pg_strong_random() sources: 1. OpenSSL's RAND_bytes() 2. Windows' CryptGenRandom() function 3. /dev/urandom The patch supports the getrandom() function as a new source of pg_strong_random(). The getrandom() function uses the same source as the /dev/ura

Re: generic plans and "initial" pruning

2025-07-21 Thread Amit Langote
On Thu, Jul 17, 2025 at 9:11 PM Amit Langote wrote: > The refinements I described in my email above might help mitigate some > of those executor-related issues. However, I'm starting to wonder if > it's worth reconsidering our decision to handle pruning, locking, and > validation entirely at execu

Re: Log prefix missing for subscriber log messages received from publisher

2025-07-21 Thread Fujii Masao
On 2025/07/22 14:29, Fujii Masao wrote: On Mon, Jul 21, 2025 at 5:51 PM Álvaro Herrera wrote: On 2025-Jul-21, Fujii Masao wrote: Thanks for updating the patch! It looks good to me, except for one minor point: static inline PGresult *libpqsrv_get_result(PGconn *conn, uint32 wait_event_

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Bertrand Drouvot
Hi, On Mon, Jul 21, 2025 at 03:20:55PM -0500, Nathan Bossart wrote: > On Fri, Jul 18, 2025 at 01:39:15PM +, Bertrand Drouvot wrote: > > +#define PG_BUILTIN_LWTRANCHE(id, name) [id] = name, > > +#include "storage/lwlocktranchelist.h" > > +#undef PG_BUILTIN_LWTRANCHE > > Why not reuse PG_LWLOCK

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Bertrand Drouvot
Hi, On Mon, Jul 21, 2025 at 03:28:14PM -0500, Nathan Bossart wrote: > On Mon, Jul 21, 2025 at 03:20:55PM -0500, Nathan Bossart wrote: > > On Fri, Jul 18, 2025 at 01:39:15PM +, Bertrand Drouvot wrote: > >> +#define PG_BUILTIN_LWTRANCHE(id, name) [id] = name, > >> +#include "storage/lwlocktranch

Re: redis_fdw failure on crake

2025-07-21 Thread Michael Paquier
On Tue, Jul 22, 2025 at 12:12:25PM +0900, Richard Guo wrote: > After pushing commit e2debb643 I noticed redis_fdw failure on crake. > > +ERROR: failed to connect to Redis: 1 > +CONTEXT: SQL statement "select (select count(*) from > db15) + > +(select count(*) from db15

Re: IndexAmRoutine aminsertcleanup function can be NULL?

2025-07-21 Thread Michael Paquier
On Thu, Jul 17, 2025 at 01:34:42PM +0800, Japin Li wrote: > On Wed, 16 Jul 2025 at 10:08, Peter Smith wrote: >> What's going on there? Is it just an accidentally missing "/* can be >> NULL */" comment? > > It appears commit c1ec02be1d79 is missing the comment. Agreed. That's user-visible, so ba

Re: Log prefix missing for subscriber log messages received from publisher

2025-07-21 Thread Fujii Masao
On Mon, Jul 21, 2025 at 5:51 PM Álvaro Herrera wrote: > > On 2025-Jul-21, Fujii Masao wrote: > > > Thanks for updating the patch! It looks good to me, except for one minor > > point: > > > > static inline PGresult *libpqsrv_get_result(PGconn *conn, uint32 > > wait_event_info); > > +static inlin

Re: Proposal: QUALIFY clause

2025-07-21 Thread Tom Lane
Nico Williams writes: > On Mon, Jul 21, 2025 at 09:43:15PM -0600, Merlin Moncure wrote: >> Hm, HAVING requires to apply 'group by' which windows functions do not >> require (unlike aggregates). > Pavel's point is precisely to allow HAVING w/o a GROUP BY when there are > window functions since win

Re: Proposal: QUALIFY clause

2025-07-21 Thread Merlin Moncure
On Mon, Jul 21, 2025 at 10:08 PM Nico Williams wrote: > On Mon, Jul 21, 2025 at 09:43:15PM -0600, Merlin Moncure wrote: > > On Mon, Jul 21, 2025 at 9:19 PM Pavel Stehule > > wrote: > > > just for curiosity - why the HAVING clause was not used? > > > > > > Any window functions are +/- an "aggrega

Re: Update Examples in Logical Replication Docs

2025-07-21 Thread Amit Kapila
On Mon, Jul 21, 2025 at 10:57 PM Shlok Kyal wrote: > > While going through the example for column lists [1] and row filters > [2] in logical replication, I found that the output of "\dRp+" is not > updated as per PostgreSQL 18. In PG 18, a new column "Generated > columns' was added in "\dRp+" by c

Re: Improve LWLock tranche name visibility across backends

2025-07-21 Thread Sami Imseih
> >> I was imagining putting the array in one big DSA allocation instead of > >> carting around a pointer for each tranche name. (Sorry, I realize I am > >> hand-waving over some of the details.) > > > > I understood it like this. Here is a sketch: > > > > ``` > > dsa_pointer p; > > > > dsa = dsa_

Re: Proposal: QUALIFY clause

2025-07-21 Thread Nico Williams
On Mon, Jul 21, 2025 at 09:43:15PM -0600, Merlin Moncure wrote: > On Mon, Jul 21, 2025 at 9:19 PM Pavel Stehule > wrote: > > just for curiosity - why the HAVING clause was not used? > > > > Any window functions are +/- an "aggregate" function, and then HAVING > > looks more natural to me. > > Hm,

Re: Conflict detection for update_deleted in logical replication

2025-07-21 Thread Amit Kapila
On Mon, Jul 21, 2025 at 11:27 PM Masahiko Sawada wrote: > > On Sun, Jul 20, 2025 at 9:00 PM Amit Kapila wrote: > > > > > If so, I agree > > with you, we don't need XIDs of other databases as logical WALSender > > will anyway won't process transactions in other databases, so we can > > exclude tho

Re: Proposal: QUALIFY clause

2025-07-21 Thread Merlin Moncure
On Mon, Jul 21, 2025 at 9:19 PM Pavel Stehule wrote: > > just for curiosity - why the HAVING clause was not used? > > Any window functions are +/- an "aggregate" function, and then HAVING > looks more natural to me. > Hm, HAVING requires to apply 'group by' which windows functions do not require

Re: Proposal: QUALIFY clause

2025-07-21 Thread Pavel Stehule
Hi út 22. 7. 2025 v 0:12 odesílatel Vik Fearing napsal: > > On 21/07/2025 23:29, Matheus Alcantara wrote: > > On Mon Jul 21, 2025 at 5:23 PM -03, Vik Fearing wrote: > >> On 21/07/2025 14:47, Matheus Alcantara wrote: > >>> Hi all, > >>> > >>> I'm sending a proof-of-concept patch to add support fo

redis_fdw failure on crake

2025-07-21 Thread Richard Guo
After pushing commit e2debb643 I noticed redis_fdw failure on crake. +ERROR: failed to connect to Redis: 1 +CONTEXT: SQL statement "select (select count(*) from db15) + +(select count(*) from db15_hash) + +(select count(*) from db15_set) + +(select count

Re: Reduce "Var IS [NOT] NULL" quals during constant folding

2025-07-21 Thread Richard Guo
On Wed, Jul 16, 2025 at 10:57 AM Richard Guo wrote: > On Wed, Jul 9, 2025 at 3:32 PM Richard Guo wrote: > > Here is a new rebase. I moved the call to preprocess_relation_rtes to > > a later point within convert_EXISTS_sublink_to_join, so we can avoid > > the work if it turns out that the EXISTS

Re: [PATCH] Check for TupleTableSlot nullness before dereferencing

2025-07-21 Thread David Rowley
On Tue, 22 Jul 2025 at 10:20, Jacob Champion wrote: > > On Fri, Dec 13, 2024 at 12:54 AM Alexander Kuznetsov > > 1. slot2 is NULL at line 968, > > 2. The while loop at line 971 executes once, filling slot1 (slot2 still > > remains NULL), > > 3. No changes occur to slot2 thereafter, up to line 100

Re: teach pg_upgrade to handle in-place tablespaces

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 09:06:59PM -0500, Nathan Bossart wrote: > On Tue, Jul 22, 2025 at 10:37:05AM +0900, Michael Paquier wrote: >> This would not choke as long as the old cluster is at least at v10, >> but well why not. > > I'm not sure what you mean. allow_in_place_tablespaces was added in v1

Re: Update Examples in Logical Replication Docs

2025-07-21 Thread Peter Smith
On Tue, Jul 22, 2025 at 3:27 AM Shlok Kyal wrote: > > Hi, > > While going through the example for column lists [1] and row filters > [2] in logical replication, I found that the output of "\dRp+" is not > updated as per PostgreSQL 18. In PG 18, a new column "Generated > columns' was added in "\dRp

Re: teach pg_upgrade to handle in-place tablespaces

2025-07-21 Thread Nathan Bossart
On Tue, Jul 22, 2025 at 10:37:05AM +0900, Michael Paquier wrote: > On Mon, Jul 21, 2025 at 07:57:32PM -0500, Nathan Bossart wrote: > +if (!defined($ENV{oldinstall})) > +{ > +$new->append_conf('postgresql.conf', > +"allow_in_place_tablespaces = true"); > +$old->ap

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 08:34:41PM -0500, Nathan Bossart wrote: > I bet we could maintain a decent level of readability with some extra > commentary. IMHO it's worth it to avoid maintaining duplicate lists. But > that's not something I feel terribly strong about, if others disagree. > FWIW I was

Re: CAST(... ON DEFAULT) - WIP build on top of Error-Safe User Functions

2025-07-21 Thread jian he
hi. more preparation work has been committed. 1. SQL/JSON patch [1] added keyword ERROR 2. CoerceViaIo, CoerceToDomain can be evaluated error safe. see commit [2]. 3. ExprState added ErrorSaveContext point, so before calling ExecInitExprRec set valid ErrorSaveContext for ExprState->escontext we sh

Re: Skipping schema changes in publication

2025-07-21 Thread Peter Smith
Hi Shlok, Some review comments for patch v17-0003. I also checked the TAP test this time. == doc/src/sgml/logical-replication.sgml 1. + publish_generated_columns. Specifying generated + columns in a column list using the EXCEPT clause excludes + the specified generated columns from bei

Re: Non-text mode for pg_dumpall

2025-07-21 Thread Andrew Dunstan
On 2025-07-21 Mo 8:53 PM, Noah Misch wrote: On Mon, Jul 21, 2025 at 04:41:03PM -0400, Andrew Dunstan wrote: On 2025-07-17 Th 6:18 AM, Mahendra Singh Thalor wrote --- a/src/bin/pg_dump/pg_restore.c +++ b/src/bin/pg_dump/pg_restore.c +/* + * read_one_statement + * + * This will start reading fr

Re: teach pg_upgrade to handle in-place tablespaces

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 07:57:32PM -0500, Nathan Bossart wrote: +if (!defined($ENV{oldinstall})) +{ +$new->append_conf('postgresql.conf', +"allow_in_place_tablespaces = true"); +$old->append_conf('postgresql.conf', +"allow_in_place_tablespaces = true"

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Nathan Bossart
On Tue, Jul 22, 2025 at 08:02:52AM +0900, Michael Paquier wrote: > Ah, you mean removing the need to have to maintain BuiltinTrancheIds. > This structure depends on NUM_INDIVIDUAL_LWLOCKS for the start value. > Not really an objection per-se, but trying to automate everything may > impact the reada

Re: Support tid range scan in parallel?

2025-07-21 Thread David Rowley
On Tue, 10 Jun 2025 at 11:04, Cary Huang wrote: > I have addressed your comment in the attached v6 patch. Thank you again for > the review. Here's a review of v6: 1. In cost_tidrangescan() you're dividing the total costs by the number of workers yet the comment is claiming that's CPU cost. I thi

Re: teach pg_upgrade to handle in-place tablespaces

2025-07-21 Thread Nathan Bossart
On Mon, Jul 21, 2025 at 08:16:12PM -0400, Corey Huinker wrote: > Everything here makes sense to me, but I do have one question: Thanks for reviewing. > In src/bin/pg_upgrade/info.c > @@ -616,11 +630,21 @@ process_rel_infos(DbInfo *dbinfo, PGresult *res, void > *arg) > + if (inplace) > + tablespac

Re: Improve error reporting in 027_stream_regress test

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 11:53:00AM +0300, Nazir Bilal Yavuz wrote: > I realized that we actually don't trim the file, we do the opposite; > read the file from both ends. Sorry for not realizing earlier. I will > update the remaining patches according to that but I think trim_file() > is helpful, to

Re: Non-text mode for pg_dumpall

2025-07-21 Thread Noah Misch
On Mon, Jul 21, 2025 at 04:41:03PM -0400, Andrew Dunstan wrote: > On 2025-07-17 Th 6:18 AM, Mahendra Singh Thalor wrote > > > > > > > --- a/src/bin/pg_dump/pg_restore.c > > > > > > > +++ b/src/bin/pg_dump/pg_restore.c > > > > > > > +/* > > > > > > > + * read_one_statement > > > > > > > + * > > > >

Re: teach pg_upgrade to handle in-place tablespaces

2025-07-21 Thread Corey Huinker
On Tue, Jul 1, 2025 at 4:06 PM Nathan Bossart wrote: > rebased > > -- > nathan Everything here makes sense to me, but I do have one question: In src/bin/pg_upgrade/info.c @@ -616,11 +630,21 @@ process_rel_infos(DbInfo *dbinfo, PGresult *res, void *arg) + if (inplace) + tablespace = psprintf("%

Re: Parallel heap vacuum

2025-07-21 Thread Andres Freund
Hi, On 2025-07-21 12:41:49 -0700, Masahiko Sawada wrote: > The reason why I added some callbacks as table AM callbacks in the > patch is that I could not find other better places. Currently, > vacuumparallel.c handles several critical operations for parallel > vacuuming: allocating and initializin

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-21 Thread Masahiko Sawada
On Sun, Jul 20, 2025 at 11:53 PM Amit Kapila wrote: > > On Mon, Jul 21, 2025 at 11:24 AM shveta malik wrote: > > > > On Mon, Jul 21, 2025 at 10:48 AM shveta malik > > wrote: > > > > > > I'm continuing to think it through and will share any further thoughts > > > if something comes to mind. > >

Re: track generic and custom plans in pg_stat_statements

2025-07-21 Thread Sami Imseih
> Note: the size of the change in pg_stat_statements--1.12--1.13.sql > points that we should seriously consider splitting these attributes > into multiple sub-functions. So we don't lose track of this. This should be a follow-up thread. I do agree something has to be done about the exploding list

Re: Non-reproducible AIO failure

2025-07-21 Thread Andres Freund
Hi, On 2025-06-19 10:16:12 -0500, Nico Williams wrote: > On Thu, Jun 19, 2025 at 05:05:25PM +0200, Daniel Gustafsson wrote: > > I also dug out an archeologically old MacBook Pro running macOS High Sierra > > 10.13.6 with an i5 using Apple LLVM version 10.0.0 (clang-1000.10.44.4), > > and it > > t

Re: track generic and custom plans in pg_stat_statements

2025-07-21 Thread Sami Imseih
> Yes, I think that this is a much better idea to isolate the whole > concept and let pgss grab these values. We have lived with such > additions for monitoring in EState a few times already, see for > example de3a2ea3b264 and 1d477a907e63 that are tainted with my > fingerprints. correct, there i

Re: track generic and custom plans in pg_stat_statements

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 04:47:31PM -0500, Sami Imseih wrote: > Last week I published a v11 that adds a field to QueryDesc, but as I thought > about this more, how about we just add 2 bool fields in QueryDesc->estate > ( es_cached_plan and es_is_generic_plan ), a field in CachedPlan ( > is_generic_p

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 03:28:14PM -0500, Nathan Bossart wrote: > On Mon, Jul 21, 2025 at 03:20:55PM -0500, Nathan Bossart wrote: >> Can we add a note to wait_event_names.txt about the required >> ordering/matching of the non-predefined LWLocks? Otherwise, these patches >> look pretty good to me.

Re: Proposal: QUALIFY clause

2025-07-21 Thread Tom Lane
Vik Fearing writes: > That is my preferred grammar, thank you. I have not looked at the C > code by this can be obtained with a syntax transformation. To wit: > SELECT a, b, c > FROM tab > QUALIFY wf() OVER () = ? > can be rewritten as: > SELECT a, b, c > FROM ( >     SELECT a, b, c, wf() OV

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 02:06:03PM +0300, Nikita Malakhov wrote: > While it is very fast on small data - I see several disadvantages: > - first of all, VACUUM should be revised to work with such tables; > - problematic batch insertion due to necessity to store TID chain. > > It is just a POC imple

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-07-21 Thread Michael Paquier
On Mon, Jul 21, 2025 at 02:20:31PM +0300, Nikita Malakhov wrote: > I agree that storing reltoastrelid in each Toast pointer seems to be > a waste of disk space since the current Postgres state does not > allow multiple TOAST tables per relation. va_toastrelid is a central part of the current syste

Re: [PATCH] Check for TupleTableSlot nullness before dereferencing

2025-07-21 Thread Jacob Champion
On Fri, Dec 13, 2024 at 12:54 AM Alexander Kuznetsov wrote: > ping. What do you think about reasoning below? Maybe we should consider > proposing different patch for removing redundant check there? To move this forward a bit, your reasoning: > 1. slot2 is NULL at line 968, > 2. The while loop at

Re: Add estimated hit ratio to Memoize in EXPLAIN to explain cost adjustment

2025-07-21 Thread David Rowley
On Fri, 4 Jul 2025 at 20:30, Ilia Evdokimov wrote: > I attached rebased v10 patch on 5a6c39b. I've gone over this and made some cosmetic adjustments. A few adjustments to the comments and used Cardinality rather than double for some data types. I also moved the MemoizePath.calls field down below

Re: [PATCH] Proposal to Enable/Disable Index using ALTER INDEX

2025-07-21 Thread Robert Treat
On Mon, Jul 21, 2025 at 5:24 PM Sami Imseih wrote: > > > > > it will still be extremely risky in > > > > heavy production workloads. In short, we're both walking a bull > > > > through the china shop, but it would seem mine is much more > > > > temperamental than yours. > > > > > > Robert, Could y

Re: Proposal: QUALIFY clause

2025-07-21 Thread Vik Fearing
On 21/07/2025 23:29, Matheus Alcantara wrote: On Mon Jul 21, 2025 at 5:23 PM -03, Vik Fearing wrote: On 21/07/2025 14:47, Matheus Alcantara wrote: Hi all, I'm sending a proof-of-concept patch to add support for the QUALIFY clause in Postgres. This feature allows filtering rows after window f

Re: track generic and custom plans in pg_stat_statements

2025-07-21 Thread Sami Imseih
> > "plan cache mode" to be accessible in ExecutorEnd somehow. > I agree with this - adding references to CachedPlan into the QueryDesc > looks kludge. > The most boring aspect of pg_stat_statements for me is the multiple > statements case: a single incoming query (the only case in the cache of > a

Re: Proposal: QUALIFY clause

2025-07-21 Thread Matheus Alcantara
On Mon Jul 21, 2025 at 5:23 PM -03, Vik Fearing wrote: > > On 21/07/2025 14:47, Matheus Alcantara wrote: >> Hi all, >> >> I'm sending a proof-of-concept patch to add support for the QUALIFY >> clause in Postgres. This feature allows filtering rows after window >> functions are computed, using a syn

Re: [PATCH] Proposal to Enable/Disable Index using ALTER INDEX

2025-07-21 Thread Sami Imseih
> > > it will still be extremely risky in > > > heavy production workloads. In short, we're both walking a bull > > > through the china shop, but it would seem mine is much more > > > temperamental than yours. > > > > Robert, Could you describe the GUC you would like to see? > > > > Also, I'd like

Re: pgsql: Introduce pg_shmem_allocations_numa view

2025-07-21 Thread Christoph Berg
Re: Tomas Vondra > >>> Submitted: https://marc.info/?l=linux-mm&m=175077821909222&w=2 > >>> > >> > >> Thanks! Now we wait ... > > > > It looks like that the bug is "confirmed" and that it will be fixed: > > https://marc.info/?l=linux-kernel&m=175088392116841&w=2 If I'm reading the Linux git log c

Re: Non-text mode for pg_dumpall

2025-07-21 Thread Andrew Dunstan
On 2025-07-17 Th 6:18 AM, Mahendra Singh Thalor wrote --- a/src/bin/pg_dump/pg_restore.c +++ b/src/bin/pg_dump/pg_restore.c +/* + * read_one_statement + * + * This will start reading from passed file pointer using fgetc and read till + * semicolon(sql statement terminator for global.dat file) +

Re: Proposal: QUALIFY clause

2025-07-21 Thread Nico Williams
On Mon, Jul 21, 2025 at 10:26:51PM +0200, Vik Fearing wrote: > On 21/07/2025 19:30, Tom Lane wrote: > > * I'm not exactly convinced that the committee would standardize > > it just like this. For one thing, QUALIFY is not even the right > > part of speech: it's a verb, and thus more fit to be a pr

Re: [PATCH] Proposal to Enable/Disable Index using ALTER INDEX

2025-07-21 Thread Robert Treat
On Mon, Jul 21, 2025 at 1:17 PM Sami Imseih wrote: > > > it will still be extremely risky in > > heavy production workloads. In short, we're both walking a bull > > through the china shop, but it would seem mine is much more > > temperamental than yours. > > Robert, Could you describe the GUC you

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Nathan Bossart
On Mon, Jul 21, 2025 at 03:20:55PM -0500, Nathan Bossart wrote: > On Fri, Jul 18, 2025 at 01:39:15PM +, Bertrand Drouvot wrote: >> +#define PG_BUILTIN_LWTRANCHE(id, name) [id] = name, >> +#include "storage/lwlocktranchelist.h" >> +#undef PG_BUILTIN_LWTRANCHE > > Why not reuse PG_LWLOCK for thi

Re: Proposal: QUALIFY clause

2025-07-21 Thread Vik Fearing
On 21/07/2025 19:30, Tom Lane wrote: "Matheus Alcantara" writes: You're right — semantically, using QUALIFY is equivalent to wrapping the query in a subquery and applying a WHERE clause to the result. The main motivation here is to provide a more ergonomic and readable syntax. While I underst

Re: Proposal: QUALIFY clause

2025-07-21 Thread Vik Fearing
On 21/07/2025 14:47, Matheus Alcantara wrote: Hi all, I'm sending a proof-of-concept patch to add support for the QUALIFY clause in Postgres. This feature allows filtering rows after window functions are computed, using a syntax similar to the WHERE or HAVING clauses. I took a very brief lo

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

2025-07-21 Thread Nathan Bossart
On Fri, Jul 18, 2025 at 01:39:15PM +, Bertrand Drouvot wrote: > +#define PG_BUILTIN_LWTRANCHE(id, name) [id] = name, > +#include "storage/lwlocktranchelist.h" > +#undef PG_BUILTIN_LWTRANCHE Why not reuse PG_LWLOCK for this? > + # Stop recording if we reach another section. > + last if

Re: Proposal: QUALIFY clause

2025-07-21 Thread Vik Fearing
On 21/07/2025 16:41, Tom Lane wrote: Isaac Morland writes: I'll be honest, I'm skeptical that we need another keyword that basically means “WHERE, but applied at a different point in the query processing”. That was my reaction too. I'm especially skeptical that getting out front of the SQL

Re: Parallel heap vacuum

2025-07-21 Thread Masahiko Sawada
On Mon, Jul 21, 2025 at 7:28 AM Melanie Plageman wrote: > > On Fri, Jul 18, 2025 at 10:00 PM Masahiko Sawada > wrote: > > > > On Thu, Jul 17, 2025 at 1:39 PM Melanie Plageman > > wrote: > > > > > I think there is a fundamental tension here related to whether or not > > > vacuumparallel.c should

Re: Skipping schema changes in publication

2025-07-21 Thread Shlok Kyal
On Mon, 21 Jul 2025 at 16:22, shveta malik wrote: > > On Sat, Jul 19, 2025 at 4:17 PM Shlok Kyal wrote: > > > > On Mon, 30 Jun 2025 at 16:25, shveta malik wrote: > > > > > > Few more comments on 002: > > > > > > 5) > > > +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) > > > { > >

Re: display hot standby state in psql prompt

2025-07-21 Thread Greg Sabino Mullane
On Thu, Jun 26, 2025 at 3:22 AM Jim Jones wrote: > What do you think? > Seems good enough for me. I think as long as we document it well, it's only going to be a net positive, even with some edge cases. Cheers, Greg -- Crunchy Data - https://www.crunchydata.com Enterprise Postgres Software Pro

Re: Exposing some hidden serializable transaction costs

2025-07-21 Thread Greg Sabino Mullane
I would imagine you'd need a pretty edge case query with a ton of rows before you would really be able to have enough difference to change plans. And what would it change to? Cheers, Greg -- Crunchy Data - https://www.crunchydata.com Enterprise Postgres Software Products & Tech Support

Re: [PATCH] Generate random dates/times in a specified range

2025-07-21 Thread Greg Sabino Mullane
Damien, maybe we can let the time ones go? Tom and I are not big fans of those, and nobody else has stepped up to defend them. Cheers, Greg -- Crunchy Data - https://www.crunchydata.com Enterprise Postgres Software Products & Tech Support

Re: Proposal: QUALIFY clause

2025-07-21 Thread Marko Tiikkaja
On Mon, Jul 21, 2025 at 9:32 PM Thom Brown wrote: >> * I'm not exactly convinced that the committee would standardize >> it just like this. For one thing, QUALIFY is not even the right >> part of speech: it's a verb, and thus more fit to be a primary >> statement keyword. What you need here is a

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-21 Thread Andres Freund
Hi, On 2025-07-21 13:37:04 -0400, Greg Burd wrote: > On 7/18/25 13:03, Andres Freund wrote: > Hello.  Thanks again for taking the time to review the email and patch, > I think we're onto something good here. > > > > > I'd be curious if anybody wants to argue for keeping the clock sweep. Except >

Re: Proposal: QUALIFY clause

2025-07-21 Thread Thom Brown
On Mon, 21 Jul 2025, 18:31 Tom Lane, wrote: > "Matheus Alcantara" writes: > > You're right — semantically, using QUALIFY is equivalent to wrapping the > > query in a subquery and applying a WHERE clause to the result. The main > > motivation here is to provide a more ergonomic and readable synta

Question on any plans to use the User Server/User Mapping to provide Logical Replication Subscriptions the user/password in an encrypted manner

2025-07-21 Thread Vitale, Anthony, Sony Music
Hello All I am not sure if I am posting this to the correct PG list, please let me know if there are other lists better suited to answer this question. Postgresql dblinks and dblink_fdw allow for the use of Server and user mapping to be able to store the user/password of a connection and save i

Re: Conflict detection for update_deleted in logical replication

2025-07-21 Thread Masahiko Sawada
On Sun, Jul 20, 2025 at 9:00 PM Amit Kapila wrote: > > On Sat, Jul 19, 2025 at 10:32 AM Amit Kapila wrote: > > > > On Sat, Jul 19, 2025 at 3:01 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Jul 18, 2025 at 5:03 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > > > Here are some review c

Re: [PATCH] Let's get rid of the freelist and the buffer_strategy_lock

2025-07-21 Thread Greg Burd
On 7/18/25 13:03, Andres Freund wrote: > Hi, Hello.  Thanks again for taking the time to review the email and patch, I think we're onto something good here. > > I'd be curious if anybody wants to argue for keeping the clock sweep. Except > for the have_free_buffer() use in autoprewarm, it's a rat

Re: Proposal: QUALIFY clause

2025-07-21 Thread Tom Lane
"Matheus Alcantara" writes: > You're right — semantically, using QUALIFY is equivalent to wrapping the > query in a subquery and applying a WHERE clause to the result. The main > motivation here is to provide a more ergonomic and readable syntax. > While I understand the hesitation around introdu

Update Examples in Logical Replication Docs

2025-07-21 Thread Shlok Kyal
Hi, While going through the example for column lists [1] and row filters [2] in logical replication, I found that the output of "\dRp+" is not updated as per PostgreSQL 18. In PG 18, a new column "Generated columns' was added in "\dRp+" by commit [3], so I think we should update the docs to reflec

Re: [PATCH] Proposal to Enable/Disable Index using ALTER INDEX

2025-07-21 Thread Sami Imseih
> it will still be extremely risky in > heavy production workloads. In short, we're both walking a bull > through the china shop, but it would seem mine is much more > temperamental than yours. Robert, Could you describe the GUC you would like to see? Also, I'd like to ask. what would be the argu

Re: POC: Parallel processing of indexes in autovacuum

2025-07-21 Thread Sami Imseih
Thanks for the patches! I have only reviewed the v8-0001-Parallel-index-autovacuum.patch so far and have a few comments from my initial pass. 1/ Please run pgindent. 2/ Documentation is missing. There may be more, but here are the places I found that likely need updates for the new behavior, rel

Re: Optimize shared LWLock acquisition for high-core-count systems

2025-07-21 Thread Zhou, Zhiguo
On 7/11/2025 4:35 PM, Yura Sokolov wrote: 10.07.2025 18:57, Zhou, Zhiguo пишет: On 7/9/2025 3:56 PM, Yura Sokolov wrote: 30.05.2025 14:30, Zhou, Zhiguo пишет: Hi Hackers, I am reaching out to solicit your insights and comments on this patch addressing a significant performance bottleneck

Re: Issues with hash and GiST LP_DEAD setting for kill_prior_tuple

2025-07-21 Thread Peter Geoghegan
On Thu, Jul 17, 2025 at 7:27 PM Mihail Nikalayeu wrote: > > FWIW _hash_readpage has a comment about a stashed LSN, so it seems as > > if this was barely missed by the work on hash indexes around 2017: > > I think commit 22c5e735 [0] (Remove lsn from HashScanPosData) is the > thing you are looking

Re: Fwd: [PATCH] Support for basic ALTER TABLE progress reporting.

2025-07-21 Thread Álvaro Herrera
On 2025-Jul-21, Jiří Kavalík wrote: > I noticed that but not sure if it is targeting v19? It is -- I'm working on that patch. > I hoped to make the change as small as possible, but if it would > collide with the refactoring then it makes sense to separate the > functionality. I think from the u

Fwd: [PATCH] Support for basic ALTER TABLE progress reporting.

2025-07-21 Thread Jiří Kavalík
I did not add CC to the list to my reply so forwarding.. -- Forwarded message - From: Jiří Kavalík Date: Sun, Jul 20, 2025 at 8:22 PM Subject: Re: [PATCH] Support for basic ALTER TABLE progress reporting. To: jian he Hello, On Tue, Jul 8, 2025 at 3:42 PM jian he wrote: > hi.

Re: IPC/MultixactCreation on the Standby server

2025-07-21 Thread Andrey Borodin
> On 18 Jul 2025, at 18:53, Andrey Borodin wrote: > > Please find attached dirty test and a sketch of the fix. It is done against > PG 16, I wanted to ensure that problem is reproducible before 17. Here'v v7 with improved comments and cross-check for correctness. Also, MultiXact wraparound is

Re: track generic and custom plans in pg_stat_statements

2025-07-21 Thread Andrei Lepikhov
On 18/7/2025 21:37, Sami Imseih wrote: Thanks for clearing up my understanding. Essentially, override the current cost-based method of determining custom vs. generic by using something like execution time, which is somehow tracked by the extension. That is how I understand this. Now, I wonder if

Re: Proposal: QUALIFY clause

2025-07-21 Thread Matheus Alcantara
On Mon Jul 21, 2025 at 11:29 AM -03, Isaac Morland wrote: > Is this different from using the window functions in a subquery and then > applying a WHERE clause on the outer query? > > SELECT … FROM (SELECT … [including window functions] FROM …) WHERE [stuff > that would be in QUALIFY] > > I'll be ho

Re: Proposal: QUALIFY clause

2025-07-21 Thread Tom Lane
Isaac Morland writes: > I'll be honest, I'm skeptical that we need another keyword that basically > means “WHERE, but applied at a different point in the query processing”. That was my reaction too. I'm especially skeptical that getting out front of the SQL standards committee is a good thing to

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-07-21 Thread Nathan Bossart
On Mon, Jul 21, 2025 at 02:03:45AM +0200, Hannu Krosing wrote: > Do you think the current patch could be backported to at least some > latest versions ? I think that's pretty unlikely. It'd be a pretty big departure from our versioning policy. In the past, we have back-patched "critical" perform

Re: Proposal: QUALIFY clause

2025-07-21 Thread Isaac Morland
On Mon, 21 Jul 2025 at 10:19, Mike Artz wrote: > Many times I have thought it would be nice if there was a QUALIFY clause > in Postgres! > > Just would like to add that including your list, Teradata, Redshift, SAP > HANA, HP Vertica, and Trino all support the QUALIFY clause. > > Also it seems Pos

Re: Parallel heap vacuum

2025-07-21 Thread Melanie Plageman
On Fri, Jul 18, 2025 at 10:00 PM Masahiko Sawada wrote: > > On Thu, Jul 17, 2025 at 1:39 PM Melanie Plageman > wrote: > > > I think there is a fundamental tension here related to whether or not > > vacuumparallel.c should be table-AM agnostic. All of its code is > > invoked below heap_vacuum_rel(

Re: Proposal: QUALIFY clause

2025-07-21 Thread Mike Artz
Many times I have thought it would be nice if there was a QUALIFY clause in Postgres! Just would like to add that including your list, Teradata, Redshift, SAP HANA, HP Vertica, and Trino all support the QUALIFY clause. Also it seems Postgres would be the first leading RDBMS - meaning like traditi

Re: Test instability when pg_dump orders by OID

2025-07-21 Thread Robert Haas
On Fri, Jul 18, 2025 at 3:17 PM Noah Misch wrote: > > This comment is useful, but if I were to be critical, it does a better > > job saying what this field isn't than what it is. > > True. I've changed it to this: That looks great. > - /* To have a stable sort order, break ties for some o

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-21 Thread Ajin Cherian
On Thu, Jul 17, 2025 at 4:21 PM Amit Kapila wrote: > > It seems the patch assumes that the above lock is sufficient because > AlterSubscription will take an AcessExclusiveLock on the same > subscription. So, with this proposal, if both of those become > serialized then the other locking order may

Proposal: QUALIFY clause

2025-07-21 Thread Matheus Alcantara
Hi all, I'm sending a proof-of-concept patch to add support for the QUALIFY clause in Postgres. This feature allows filtering rows after window functions are computed, using a syntax similar to the WHERE or HAVING clauses. The idea for this came from a discussion and suggestion by Peter Eisentrau

Re: index prefetching

2025-07-21 Thread Thomas Munro
On Sun, Jul 20, 2025 at 1:07 AM Thomas Munro wrote: > On Sat, Jul 19, 2025 at 11:23 PM Tomas Vondra wrote: > > Thanks for the link. It seems I came up with an almost the same patch, > > with three minor differences: > > > > 1) There's another place that sets "distance = 0" in > > read_stream_next

Re: Improve error reporting in 027_stream_regress test

2025-07-21 Thread Nazir Bilal Yavuz
Hi, On Sat, 19 Jul 2025 at 09:06, Michael Paquier wrote: > > The structure is strange, it seems to me that we should target things > so as we have only one PG_TEST_FILE_TRIM_LINES defined in the tree, > not two with one local to 027. I see your point. Then the problem is regression_log_helper()

Re: Document slot's restart_lsn can go backward

2025-07-21 Thread Alexander Korotkov
On Mon, Jul 21, 2025 at 11:40 AM vignesh C wrote: > > On Mon, 21 Jul 2025 at 11:04, Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 5:11 PM Alexander Korotkov > > wrote: > > > > > > While working on the patch fixing the situation when slot's > > > restart_lsn ends up pointing to a removed WA

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-07-21 Thread Nikita Malakhov
Hi! I agree that storing reltoastrelid in each Toast pointer seems to be a waste of disk space since the current Postgres state does not allow multiple TOAST tables per relation. But if we consider this as a viable option it could bring additional advantages. I've successfully tried to use multipl

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-07-21 Thread Nikita Malakhov
Hi! Michael and Hannu, here's a POC patch with direct TIDs TOAST. The simplest implementation where we store a chain of TIDs, each chunk stores the next TID to be fetched. Patch applies on top of commit 998b0b51d5ea763be081804434f177082ba6772b (origin/toast_64bit_v2) Author: Michael Paquier Date:

Re: Skipping schema changes in publication

2025-07-21 Thread shveta malik
On Sat, Jul 19, 2025 at 4:17 PM Shlok Kyal wrote: > > On Mon, 30 Jun 2025 at 16:25, shveta malik wrote: > > > > Few more comments on 002: > > > > 5) > > +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) > > { > > > > + List*exceptlist; > > + > > + exceptlist = GetPublicationRelat

Re: Logical Replication of sequences

2025-07-21 Thread shveta malik
On Mon, Jul 21, 2025 at 2:55 PM Amit Kapila wrote: > > On Mon, Jul 21, 2025 at 2:36 PM vignesh C wrote: > > > > On Mon, 21 Jul 2025 at 11:15, Dilip Kumar wrote: > > > > > > > > 3. Some of the syntaxes works for sequence which doesn't make sense to > > > me, as listed below, I think there are mor

Re: Add 64-bit XIDs into PostgreSQL 15

2025-07-21 Thread Yura Sokolov
21.07.2025 12:28, Daniil Davydov пишет: > Hi, > > On Thu, Jul 17, 2025 at 8:36 PM Maksim.Melnikov > wrote: >> >> On 07.07.2025 11:17, Evgeny Voropaev wrote: >>> Do-side: >>> 1. Having page ABC with several tuples. >>> 2. Starting to perform insertion of new tuple >>>2.1. In the case of an ina

Re: Logical Replication of sequences

2025-07-21 Thread Dilip Kumar
On Mon, Jul 21, 2025 at 2:36 PM vignesh C wrote: > > On Mon, 21 Jul 2025 at 11:15, Dilip Kumar wrote: > > > > On Mon, Jul 21, 2025 at 10:36 AM Dilip Kumar wrote: > > > > > > I was just trying a different test, so I realized that ALTER > > > PUBLICATION ADD SEQUENCE is not supported, any reason f

  1   2   >