Re: pg_rewind WAL segments deletion pitfall

On 2022-09-29 17:18, Polina Bungina wrote: I agree with your suggestions, so here is the updated version of patch. Hope I haven't missed anything. Thanks for the patch, I've marked this as ready-for-committer. BTW, this issue can be considered a bug, right? I think it would be appropriate to p

Re: dubious warning: FORMAT JSON has no effect for json and jsonb types

On 16.08.23 16:59, Merlin Moncure wrote: On Wed, Aug 16, 2023 at 8:55 AM Peter Eisentraut > wrote: This warning comes from parse_expr.c transformJsonValueExpr() and is triggered for example by the following test case: SELECT JSON_OBJECT('foo': NULL::json

Re: Rethink the wait event names for postgres_fdw, dblink and etc

On Fri, Aug 18, 2023 at 12:27:02PM +0900, Masahiro Ikeda wrote: > I expect that no one will object to changing the names to appropriate > ones. But, we need to discuss that naming convention, the names themselves, > document descriptions and so on. > > I made the v1 patch > * CamelCase naming conv

Re: logical decoding and replication of sequences, take 2

On Thu, Aug 17, 2023 at 7:13 PM Ashutosh Bapat wrote: > > On Wed, Aug 16, 2023 at 7:56 PM Tomas Vondra > wrote: > > > > > > > > But whether or not that's the case, downstream should not request (and > > > hence receive) any changes that have been already applied (and > > > committed) downstream a

Re: Extract numeric filed in JSONB more effectively

On Fri, Aug 18, 2023 at 10:55 AM Chapman Flack wrote: > > > Again, all of that complication stems from the choice to use the > anyelement return type and rely on polymorphic type resolution > to figure the oid out, when we already have the oid to begin with > and the oid is all we want. > you wan

Re: New WAL record to detect the checkpoint redo location

On Fri, Aug 18, 2023 at 5:24 AM Michael Paquier wrote: > > On Thu, Aug 17, 2023 at 01:11:50PM +0530, Dilip Kumar wrote: > > Yeah right, actually I was confused, I assumed it will return the > > start LSN of the record. And I do not see any easy way to identify > > the Start LSN of this record so

Re: [PoC] pg_upgrade: allow to upgrade publisher node

Here are some review comments for the patch v21-0003 == Commit message 1. pg_upgrade fails if the old node has slots which status is 'lost' or they do not consume all WAL records. These are needed for prevent the data loss. ~ Maybe some minor brush-up like: SUGGESTION In order to prevent d

Re: Adding a LogicalRepWorker type field

On Fri, Aug 18, 2023 at 8:50 AM Amit Kapila wrote: > > On Mon, Aug 14, 2023 at 12:08 PM Peter Smith wrote: > > > > The main patch for adding the worker type enum has been pushed [1]. > > > > Here is the remaining (rebased) patch for changing some previous > > cascading if/else to switch on the Lo

Rethink the wait event names for postgres_fdw, dblink and etc

Hi, Recently, the API to define custom wait events for extension is supported. * Change custom wait events to use dynamic shared hash tables(af720b4c5) So, I'd like to rethink the wait event names for modules which use "WAIT_EVENT_EXTENSION" wait events. * postgres_fdw * dblink * pg_prewarm *

Re: Adding a LogicalRepWorker type field

On Mon, Aug 14, 2023 at 12:08 PM Peter Smith wrote: > > The main patch for adding the worker type enum has been pushed [1]. > > Here is the remaining (rebased) patch for changing some previous > cascading if/else to switch on the LogicalRepWorkerType enum instead. > I see this as being useful if

RE: [PoC] pg_upgrade: allow to upgrade publisher node

Dear Amit, > > I have not used other plugins, but forcibly renamed the shared object file. > > I would test by plugins like wal2json[1] if more cases are needed. > > > > 1. created logical replication slots on old node > > SELECT * FROM pg_create_logical_replication_slot('test', 'test_decoding')

Re: Extract numeric filed in JSONB more effectively

On 2023-08-17 21:14, Andy Fan wrote: The idea of an 'internal' return type with no 'internal' parameter was quickly and rightly shot down. Yes, it mainly breaks the type-safety system. Parser need to know the result type, so PG defines the rule like this: Well, the reason "internal return ty

Re: [PoC] pg_upgrade: allow to upgrade publisher node

On Thu, Aug 17, 2023 at 2:10 PM Peter Smith wrote: > > Here are some review comments for the first 2 patches. > > > 3. > > + > + Before you start upgrading the publisher node, ensure that the > + subscription is temporarily disabled. After the upgrade is complete, > + execute the >

Re: Remove distprep

On 2023-08-18 10:11:12 +0900, Michael Paquier wrote: > On Wed, Aug 09, 2023 at 07:38:40AM -0700, Andres Freund wrote: > > On 2023-08-09 16:25:28 +0200, Christoph Berg wrote: > >> Understood, I was just pointing out there are more types of generated > >> files in there. > > > > The situation for co

Re: pg_upgrade - typo in verbose log

On Fri, Aug 18, 2023 at 10:47 AM Michael Paquier wrote: > > On Thu, Aug 17, 2023 at 06:09:31PM +1000, Peter Smith wrote: > > Ping. > > Funnily enough, I was looking at this entry yesterday, before you > replied, and was wondering what's happening here. > > > I thought v2 was ready to be pushed, bu

Re: Extract numeric filed in JSONB more effectively

Hi Chapman, Thanks for the review! The idea of an 'internal' return type with no 'internal' parameter > was quickly and rightly shot down. Yes, it mainly breaks the type-safety system. Parser need to know the result type, so PG defines the rule like this: anyelement fn(anyment in); if the ex

Re: Remove distprep

On Wed, Aug 09, 2023 at 07:38:40AM -0700, Andres Freund wrote: > On 2023-08-09 16:25:28 +0200, Christoph Berg wrote: >> Understood, I was just pointing out there are more types of generated >> files in there. > > The situation for configure is somewhat different, due to being maintained in > the r

Re: Query regarding sharing data directory

On Thu, Aug 17, 2023 at 10:51:45AM +0530, Harsh N Bhatt wrote: > I have a doubt regarding the storage of the data directory path of > PostgreSQL. I'm using PostgreSQL server 15 in which I want to give a path > to an external drive RAID Memory Storage. Which is on the LAN Network? Is > it compatible

Re: pg_upgrade - typo in verbose log

On Thu, Aug 17, 2023 at 06:09:31PM +1000, Peter Smith wrote: > Ping. Funnily enough, I was looking at this entry yesterday, before you replied, and was wondering what's happening here. > I thought v2 was ready to be pushed, but then this thread went silent > for 3 months Change looks fine, so ap

Re: Ignore 2PC transaction GIDs in query jumbling

On Tue, Aug 15, 2023 at 08:49:37AM +0900, Michael Paquier wrote: > Hmm. One issue with the patch is that we finish by considering > DEALLOCATE ALL and DEALLOCATE $1 as the same things, compiling the > same query IDs. The difference is made in the Nodes by assigning NULL > to the name but we would

Re: New WAL record to detect the checkpoint redo location

On Thu, Aug 17, 2023 at 01:11:50PM +0530, Dilip Kumar wrote: > Yeah right, actually I was confused, I assumed it will return the > start LSN of the record. And I do not see any easy way to identify > the Start LSN of this record so maybe this solution will not work. I > will have to think of some

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

On Thu, Aug 17, 2023 at 03:29:28PM -0400, Stephen Frost wrote: > On Thu, Aug 17, 2023 at 15:23 Robert Haas wrote: >> For what it's worth, my vote would be for "connection authenticated: >> ... method=trust". > > I don’t have any particular objection to this language and agree that it’s > actually

Re: Normalization of utility queries in pg_stat_statements

On Wed, Aug 16, 2023 at 05:11:47PM +0800, jian he wrote: > SELECT calls, toplevel, rows, query FROM pg_stat_statements ORDER BY > query COLLATE "C"; > returns: > calls | toplevel | rows | query > ---+--+--+ > 1 | t|

Re: Use of additional index columns in rows filtering

On Wed, 2023-08-09 at 17:14 -0700, Peter Geoghegan wrote: > + Index quals are better than equivalent index filters because bitmap > index scans can only use index quals It seems there's consensus that: * Index Filters (Tomas's patch and the topic of this thread) are more general, because they c

Re: Fix an entry in wait_event_names.txt

On Thu, Aug 17, 2023 at 03:25:27PM +0900, Masahiro Ikeda wrote: > +1. Thanks! Applied. -- Michael signature.asc Description: PGP signature

Re: WIP: new system catalog pg_wait_event

On Thu, Aug 17, 2023 at 04:37:22PM +0900, Masahiro Ikeda wrote: > On 2023-08-17 14:53, Drouvot, Bertrand wrote: > BTW, is it better to start with "Waiting" like any other lines? > For example, "Waiting for custom event \"worker_spi_main\" defined by an > extension". Using "Waiting for custom event

Re: [PATCH] Add native windows on arm64 support

On Thu, Aug 17, 2023 at 09:41:44AM +0100, Anthony Roberts wrote: > Just following up on this, has there been any movement? > > I did see another message go into the thread here with no reply: > https://www.postgresql.org/message-id/ZKPLjLjIjwN2lxkg%40paquier.xyz I don't have an environment to tes

Re: meson: pgxs Makefile.global differences

On Thu Aug 17, 2023 at 3:51 PM CDT, Andres Freund wrote: PS: I don't have and...@anarazel.dev , just .de :) Fat fingered a "v" somehow. -- Tristan Partin Neon (https://neon.tech)

Re: meson: pgxs Makefile.global differences

> On Aug 17, 2023, at 4:51 PM, Andres Freund wrote: > > Hi, > >> On 2023-08-17 14:45:54 -0500, Tristan Partin wrote: >>> On Thu Aug 17, 2023 at 2:32 PM CDT, Andrew Dunstan wrote: >>> I started digging into a warning I noticed on my FDW builds where >>> Postgres is built with meson, e.g. >>

Re: meson: pgxs Makefile.global differences

Hi, On 2023-08-17 14:45:54 -0500, Tristan Partin wrote: > On Thu Aug 17, 2023 at 2:32 PM CDT, Andrew Dunstan wrote: > > I started digging into a warning I noticed on my FDW builds where > > Postgres is built with meson, e.g. > >

Re: Extract numeric filed in JSONB more effectively

On 2023-08-17 05:07, Andy Fan wrote: Thanks for the review, v9 attached! From the earliest iterations of this patch, I seem to recall a couple of designs being considered: In one, the type-specific cast function would only be internally usable, would take a type oid as an extra parameter (sup

Re: meson: pgxs Makefile.global differences

On Thu Aug 17, 2023 at 2:32 PM CDT, Andrew Dunstan wrote: I started digging into a warning I noticed on my FDW builds where Postgres is built with meson, e.g. which has

meson: pgxs Makefile.global differences

I started digging into a warning I noticed on my FDW builds where Postgres is built with meson, e.g. which has this: cc1: warning: ‘-Wformat-security’ ignored without ‘

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

Greetings, On Thu, Aug 17, 2023 at 15:23 Robert Haas wrote: > On Thu, Aug 17, 2023 at 12:54 PM Jacob Champion > wrote: > > On Thu, Aug 17, 2023 at 9:46 AM Stephen Frost > wrote: > > > Don't like 'skipped' but that feels closer. > > > > > > How about 'connection bypassed authentication'? > > >

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

On Thu, Aug 17, 2023 at 12:54 PM Jacob Champion wrote: > On Thu, Aug 17, 2023 at 9:46 AM Stephen Frost wrote: > > Don't like 'skipped' but that feels closer. > > > > How about 'connection bypassed authentication'? > > Works for me; see v2. For what it's worth, my vote would be for "connection au

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

On Thu, Aug 17, 2023 at 9:46 AM Stephen Frost wrote: > Don't like 'skipped' but that feels closer. > > How about 'connection bypassed authentication'? Works for me; see v2. Thanks! --Jacob From 5404c28391d2faae8a306e4e21c2ddfe1b70b53e Mon Sep 17 00:00:00 2001 From: Jacob Champion Date: Wed, 16

Re: Using defines for protocol characters

On Thu, Aug 17, 2023 at 12:22:24PM -0400, Robert Haas wrote: > I think this is going to be a major improvement in terms of readability! > > I'm pretty happy with this version, too. We may be running out of > things to argue about -- and no, I don't want to argue about > underscores more! Awesome.

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

Greetings, * Jacob Champion (jchamp...@timescale.com) wrote: > On Thu, Aug 17, 2023 at 9:01 AM Stephen Frost wrote: > > Maybe 'connection allowed' instead..? > > Hm. It hasn't really been allowed yet, either. To illustrate what I mean: > > LOG: connection received: host=[local] > LOG:

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

On Thu, Aug 17, 2023 at 9:01 AM Stephen Frost wrote: > That doesn't seem quite right ... admittedly, 'trust' isn't performing > authentication but there can certainly be an argument made that the > basic 'matched a line in pg_hba.conf' is a form of authentication I'm not personally on board with

Re: Using defines for protocol characters

On Thu, Aug 17, 2023 at 9:22 AM Robert Haas wrote: > On Thu, Aug 17, 2023 at 12:13 PM Nathan Bossart > wrote: > > On Thu, Aug 17, 2023 at 09:31:55AM +0900, Michael Paquier wrote: > > > Looks sensible seen from here. > > > > Thanks for taking a look. > > I think this is going to be a major improv

Re: Using defines for protocol characters

On Thu, Aug 17, 2023 at 12:13 PM Nathan Bossart wrote: > On Thu, Aug 17, 2023 at 09:31:55AM +0900, Michael Paquier wrote: > > Looks sensible seen from here. > > Thanks for taking a look. I think this is going to be a major improvement in terms of readability! I'm pretty happy with this version,

Re: Using defines for protocol characters

On Thu, Aug 17, 2023 at 09:31:55AM +0900, Michael Paquier wrote: > Looks sensible seen from here. Thanks for taking a look. > This patch is missing the installation of protocol.h in > src/tools/msvc/Install.pm for MSVC. For pqcomm.h, we are doing that: > lcopy('src/include/libpq/pqcomm.h', $targ

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

Greetings, * Jacob Champion (jchamp...@timescale.com) wrote: > Maybe something like the attached? > - I used the phrasing "connection not authenticated" in the hopes that > it's a bit more greppable than just "connection", especially in > combination with the existing "connection authenticated" l

Re: [PATCH] Add function to_oct

On Thu, Aug 17, 2023 at 12:35:54PM +0700, John Naylor wrote: > That makes it a lexically-scoped global variable, which we don't need > either. Can we have the internal function allocate on the stack, then > call cstring_to_text() on that, returning the text result? That does its > own palloc. > >

Re: PG 16 draft release notes ready

On Wed, Aug 16, 2023 at 10:36:05PM -0400, Bruce Momjian wrote: > You can view the Postgres 16 release notes, with markup and links to our > docs, here: > > https://momjian.us/pgsql_docs/release-16.html FYI, thanks to this patch: commit 78ee60ed84 Author: Tom Lane D

Re: run pgindent on a regular basis / scripted manner

On Wed, Aug 16, 2023 at 01:15:55PM -0700, Peter Geoghegan wrote: > On Tue, Aug 15, 2023 at 1:31 PM Nathan Bossart > wrote: >> Should we add those? Patch attached. > > I think that that makes sense. Committed. > I just don't want to normalize updating > .git-blame-ignore-revs very frequently.

Re: Extending SMgrRelation lifetimes

On Thu, Aug 17, 2023 at 1:11 AM Thomas Munro wrote: > Still WIP while I think about edge cases, but so far I think this is > the better option. I think this direction makes a lot of sense. The lack of a defined lifetime for SMgrRelation objects makes correct programming difficult, makes efficient

Allow Index AM scan API to access information about LIMIT clause

Hi hackers, I'm developing an index AM for a specific user defined type. Details about LIMIT clause can lead to different strategy during scan. So "SELECT * FROM tbl ORDER BY col LIMIT 5" will have a different code path to "SELECT * FROM tbl ORDER BY col LIMIT 500". It's still the same IndexScan.

Re: RFC: Logging plan of the running query

On 2023-06-16 01:34, James Coleman wrote: Attached is v28 which sets ProcessLogQueryPlanInterruptActive to false in errfinish when necessary. Once built with those two patches I'm simply running `make check`. With v28-0001 and v28-0002 patch, I confirmed backend processes consume huge amount

Re: Allow parallel plan for referential integrity checks?

On 8/17/23 14:00, Frédéric Yhuel wrote: On 8/17/23 09:32, Frédéric Yhuel wrote: On 8/10/23 17:06, Juan José Santamaría Flecha wrote: Recently I restored a database from a directory format backup and having this feature would have been quite useful Hi, Thanks for resuming work on this

Re: logical decoding and replication of sequences, take 2

On Wed, Aug 16, 2023 at 7:56 PM Tomas Vondra wrote: > > > > > But whether or not that's the case, downstream should not request (and > > hence receive) any changes that have been already applied (and > > committed) downstream as a principle. I think a way to achieve this is > > to update the replo

Re: [PoC] pg_upgrade: allow to upgrade publisher node

On Thu, Aug 17, 2023 at 3:48 PM Hayato Kuroda (Fujitsu) wrote: > > > > * Added checks for output plugin libraries. pg_upgrade ensures that > > > plugins > > > referred by old slots were installed to the new executable directory. > > > > > > > I think this is a good idea but did you test it with

Re: [PoC] pg_upgrade: allow to upgrade publisher node

On Thu, Aug 17, 2023 at 6:07 PM Masahiko Sawada wrote: > > On Tue, Aug 15, 2023 at 12:06 PM Amit Kapila wrote: > > > > On Tue, Aug 15, 2023 at 7:51 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Aug 14, 2023 at 2:07 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, Aug 14, 2023 at 7:57 A

Re: [PoC] pg_upgrade: allow to upgrade publisher node

On Tue, Aug 15, 2023 at 12:06 PM Amit Kapila wrote: > > On Tue, Aug 15, 2023 at 7:51 AM Masahiko Sawada wrote: > > > > On Mon, Aug 14, 2023 at 2:07 PM Amit Kapila wrote: > > > > > > On Mon, Aug 14, 2023 at 7:57 AM Masahiko Sawada > > > wrote: > > > > Another idea is (which might have already d

Re: walsender "wakeup storm" on PG16, likely because of bc971f4025c (Optimize walsender wake up logic using condition variables)

Thomas Munro wrote: > On Wed, Aug 16, 2023 at 11:18 PM Antonin Houska wrote: > > I try to understand this patch (commit 5ffb7c7750) because I use condition > > variable in an extension. One particular problem occured to me, please > > consider: > > > > ConditionVariableSleep() gets interrupted,

Re: Allow parallel plan for referential integrity checks?

On 8/17/23 09:32, Frédéric Yhuel wrote: On 8/10/23 17:06, Juan José Santamaría Flecha wrote: Recently I restored a database from a directory format backup and having this feature would have been quite useful Hi, Thanks for resuming work on this patch. I forgot to mention this in my origi

Re: meson: Non-feature feature options

Hi, I was looking at older threads and found that. There were failures when rebasing v1-uuid patch to master so I updated it and also added documentation. I attached the v2 of the patch. I am sending v2 patch to this thread but should I create a new thread for uuid patch? Regards, Nazir Bilal Yav

Re: Synchronizing slots from primary to standby

On Thu, Aug 17, 2023 at 11:55 AM shveta malik wrote: > > On Thu, Aug 17, 2023 at 11:44 AM Drouvot, Bertrand > wrote: > > > > Hi, > > > > On 8/14/23 11:52 AM, shveta malik wrote: > > > > > > > > We (myself and Ajin) performed the tests to compute the lag in standby > > > slots as compared to prima

Re: POC, WIP: OR-clause support for indexes

Sorry, I didn't write correctly enough, about the second second place in the code where the conversion works well enough is the removal of duplicate OR expressions. I attached patch to learn it in more detail. On 17.08.2023 13:08, a.rybakina wrote: Hi, all! The optimizer will itself do a lim

Re: POC, WIP: OR-clause support for indexes

Hi, all! The optimizer will itself do a limited form of "normalizing to CNF". Are you familiar with extract_restriction_or_clauses(), from orclauses.c? Comments above the function have an example of how this can work:   * Although a join clause must reference multiple relations overall,   * an O

Re: [PATCH] Add native windows on arm64 support

Hi All, Just following up on this, has there been any movement? I did see another message go into the thread here with no reply: https://www.postgresql.org/message-id/ZKPLjLjIjwN2lxkg%40paquier.xyz Thanks, Anthony On 14/06/2023 13:25, Niyas Sait wrote: Hello, Just wanted to give a heads up

Query regarding sharing data directory

Respected Sir/Ma'am, I have a doubt regarding the storage of the data directory path of PostgreSQL. I'm using PostgreSQL server 15 in which I want to give a path to an external drive RAID Memory Storage. Which is on the LAN Network? Is it compatible or not? If it is then which file system is suitab

Re: Extract numeric filed in JSONB more effectively

Hi jian: On Thu, Aug 17, 2023 at 12:32 AM jian he wrote: > On Wed, Aug 16, 2023 at 2:28 PM Andy Fan wrote: > > > > update with the correct patch.. > > regression=# select proname, pg_catalog.pg_get_function_arguments(oid) > from pg_proc > where proname = 'jsonb_extract_path_type'; > p

Re: [PoC] pg_upgrade: allow to upgrade publisher node

Here are some review comments for the first 2 patches. (There are a couple of overlaps with what Hou-san already wrote review comments about) For patch v21-0001... == 1. SaveSlotToPath - /* and don't do anything if there's nothing to write */ - if (!was_dirty) + /* + * and don't do anything

Re: pg_upgrade - typo in verbose log

On Thu, May 11, 2023 at 7:09 PM Peter Smith wrote: > > On Thu, May 11, 2023 at 6:30 PM Daniel Gustafsson wrote: > > > > > On 11 May 2023, at 07:41, Peter Smith wrote: > > > > > While reviewing another patch for the file info.c, I noticed some > > > misplaced colon (':') in the verbose logs for p

Re: New WAL record to detect the checkpoint redo location

On Thu, Aug 17, 2023 at 10:52 AM Michael Paquier wrote: > > On Tue, Aug 15, 2023 at 02:23:43PM +0530, Dilip Kumar wrote: > > Yeah, good idea, actually we can do this insert outside of the > > exclusive insert lock and set the LSN of this insert as the > > checkpoint. redo location. So now we do n

Re: WIP: new system catalog pg_wait_event

Hi, On 2023-08-17 14:53, Drouvot, Bertrand wrote: On 8/17/23 3:53 AM, Masahiro Ikeda wrote: 1) The regular expression needs to be changed in generate-wait_event_types.pl. I have compared the documentation with the output of the pg_wait_event view and found the following differences. * For p

Re: Allow parallel plan for referential integrity checks?

On 8/10/23 17:06, Juan José Santamaría Flecha wrote: Recently I restored a database from a directory format backup and having this feature would have been quite useful Hi, Thanks for resuming work on this patch. I forgot to mention this in my original email, but the motivation was also to