Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Ashutosh Bapat
On Wed, Jul 19, 2023 at 1:20 AM Tomas Vondra wrote: > >> > > > > This behaviour doesn't need any on-disk changes or has nothing in it > > which prohibits us from changing it in future. So I think it's good as > > a v0. If required we can add the protocol option to provide more > > flexible

Re: doc: clarify the limitation for logical replication when REPILICA IDENTITY is FULL

2023-07-18 Thread Amit Kapila
On Mon, Jul 17, 2023 at 11:51 AM Önder Kalacı wrote: > >> > >> > The last line seems repetitive to me. So, I have removed it. Apart >> > from that patch looks good to me. Sergie, Peter, and others, any >> > thoughts? >> >> The v5 patch LGTM. >> > > Overall looks good to me as well. Please

Re: Allow pg_archivecleanup to remove backup history files

2023-07-18 Thread Michael Paquier
On Fri, Jun 30, 2023 at 03:48:43PM +0900, Michael Paquier wrote: > I have begun cleaning up my board, and applied 0001 for the moment. And a few weeks later.. I have come around this thread and applied 0002 and 0003. The flow of 0002 was straight-forward. My main issue was in 0003, actually,

Re: logicalrep_message_type throws an error

2023-07-18 Thread Ashutosh Bapat
On Wed, Jul 19, 2023 at 9:01 AM Amit Kapila wrote: > > On Tue, Jul 18, 2023 at 10:27 AM Masahiko Sawada > wrote: > > > > On Tue, Jul 18, 2023 at 12:15 PM Amit Kapila > > wrote: > > > > > > On Mon, Jul 17, 2023 at 7:54 PM Alvaro Herrera > > > wrote: > > > > > > > > > > I have tried to check

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread Amit Kapila
On Wed, Jul 19, 2023 at 8:38 AM Peter Smith wrote: > > Some review comments for v19-0001 > ... > == > src/backend/replication/logical/worker.c > > 3. set_stream_options > > +/* > + * Sets streaming options including replication slot name and origin start > + * position. Workers need these

Re: Support to define custom wait events for extensions

2023-07-18 Thread Masahiro Ikeda
Hi, I updated the patches. * v6-0001-Support-custom-wait-events-for-extensions.patch The main diffs are * rebase it atop current HEAD * update docs to show users how to use the APIs * rename of functions and variables * fix typos * define a new spinlock in shared memory for this purpose *

Re: harmonize password reuse in vacuumdb, clusterdb, and reindexdb

2023-07-18 Thread Zhang Mingli
HI, On Jun 29, 2023 at 13:24 +0800, Nathan Bossart , wrote: > > Connecting to different databases with the same > host/port/user information seems okay. Have a look, yeah, cluster_all_databases/vacuum_all_databases/reindex_all_databases will get there. LGTM. Regards, Zhang Mingli

Re: logicalrep_message_type throws an error

2023-07-18 Thread Amit Kapila
On Tue, Jul 18, 2023 at 10:27 AM Masahiko Sawada wrote: > > On Tue, Jul 18, 2023 at 12:15 PM Amit Kapila wrote: > > > > On Mon, Jul 17, 2023 at 7:54 PM Alvaro Herrera > > wrote: > > > > > > > I have tried to check whether we have such usage in any other error > > callbacks. Though I haven't

Re: Generating code for query jumbling through gen_node_support.pl

2023-07-18 Thread Michael Paquier
On Tue, Jul 11, 2023 at 07:35:43AM +0900, Michael Paquier wrote: > I still don't think that we need both methods based on these numbers, > but there may be more opinions about that? Are people OK if this open > item is discarded? Hearing nothing about this point, removed from the open item list,

Re: FATAL: operator class "xxxx" does not exist for access method "btree"

2023-07-18 Thread Tom Lane
mao zhang writes: > running bootstrap script ... 2023-07-19 09:40:47.083 CST [2808392] FATAL: > operator class "key_ops" does not exist for access method "btree" I'm not sure what you find so mysterious about that error message. > Oid

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread Peter Smith
Some review comments for v19-0001 == src/backend/replication/logical/tablesync.c 1. run_tablesync_worker +run_tablesync_worker(WalRcvStreamOptions *options, + char *slotname, + char *originname, + int originname_size, + XLogRecPtr *origin_startpos) +{ + /* Start table synchronization. */ +

FATAL: operator class "xxxx" does not exist for access method "btree"

2023-07-18 Thread mao zhang
Dear pgsql: When we adding a custom system table and defining an index for it at the same time, the code compilation is possible, but the following errors may occur when starting the database:

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2023-07-18 Thread Michael Paquier
On Sun, Jul 16, 2023 at 09:37:28PM -0500, Justin Pryzby wrote: > I understand that it's possible for it to be conditional, but I don't > undertand why it's desirable/important ? Because it's cheaper on repeated commands, like no CCI necessary. > It's not conditional in the tablespace code that

FATAL: operator class "xxxx" does not exist for access method "btree"

2023-07-18 Thread mao zhang
Dear pgsql: When we adding a custom system table and defining an index for it at the same time, the code compilation is possible, but the following errors may occur when starting the database:

Re: pg_recvlogical prints bogus error when interrupted

2023-07-18 Thread Michael Paquier
On Mon, Jul 10, 2023 at 01:44:45PM +0900, Michael Paquier wrote: > As StreamLogicalLog() states once it leaves its main loop because > time_to_abort has been switched to true, we want a clean exit. I > think that this patch is just a more complicated way to avoid doing > twice the operations done

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-18 Thread Thomas Munro
On Wed, Jul 19, 2023 at 12:45 PM Andres Freund wrote: > I don't think "invalidating" is the right terminology. Note that we already > have InvalidateBuffer() - but it's something we can't allow users to do, as it > throws away dirty buffer contents (it's used for things like dropping a > table).

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-18 Thread Andres Freund
Hi, I wanted this feature a couple times before... On 2023-07-03 13:56:29 +0530, Palak Chaturvedi wrote: > +PG_FUNCTION_INFO_V1(pg_buffercache_invalidate); > +Datum > +pg_buffercache_invalidate(PG_FUNCTION_ARGS) I don't think "invalidating" is the right terminology. Note that we already have

Re: Use of additional index columns in rows filtering

2023-07-18 Thread Jeff Davis
On Wed, 2023-07-19 at 00:36 +0200, Tomas Vondra wrote: > > * I'm confused about the relationship of an IOS to an index filter. > > It > > seems like the index filter only works for an ordinary index scan? > > Why > > is that? > > What would it do for IOS? The way it's presented is slightly

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-07-18 Thread Thomas Munro
On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion wrote: > We're not setting EV_RECEIPT for these -- is that because none of the > filters we're using are EV_CLEAR, and so it doesn't matter if we > accidentally pull pending events off the queue during the kevent() call? +1 for EV_RECEIPT ("just

Re: Use of additional index columns in rows filtering

2023-07-18 Thread Tomas Vondra
On 7/18/23 22:21, Jeff Davis wrote: > Hi, > > > On Sun, 2023-07-16 at 22:36 +0200, Tomas Vondra wrote: >> This kept bothering me, so I looked at it today, and reworked it to >> use >> the IOS approach. > > Initial comments on patch 20230716: > > * check_index_filter() alredy looks at

Re: [feature]COPY FROM enable FORCE_NULL/FORCE_NOT_NULL on all columns

2023-07-18 Thread Zhang Mingli
Hi, On Jul 9, 2023 at 11:51 +0800, Zhang Mingli , wrote: HI, Regards, Zhang Mingli On Jul 7, 2023, 18:00 +0800, Damir Belyalov , wrote: The patch does not work for the current version of postgres, it needs to be updated. I tested your patch. Everything looks simple and works well.

Re: Use of additional index columns in rows filtering

2023-07-18 Thread Jeff Davis
Hi, On Sun, 2023-07-16 at 22:36 +0200, Tomas Vondra wrote: > This kept bothering me, so I looked at it today, and reworked it to > use > the IOS approach. Initial comments on patch 20230716: * check_index_filter() alredy looks at "canreturn", which should mean that you don't need to later

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Tomas Vondra
On 7/18/23 15:52, Ashutosh Bapat wrote: > On Fri, Jul 14, 2023 at 7:33 PM Tomas Vondra > wrote: > >> >> Thanks for testing / confirming this! So, do we agree this behavior is >> reasonable? >> > > This behaviour doesn't need any on-disk changes or has nothing in it > which prohibits us from

Re: harmonize password reuse in vacuumdb, clusterdb, and reindexdb

2023-07-18 Thread Gurjeet Singh
On Mon, Jul 17, 2023 at 1:47 PM Nathan Bossart wrote: > > Here is a new version of the patch in which I've updated this comment as > proposed. Gurjeet, do you have any other concerns about this patch? With the updated comment, the patch looks good to me. Best regards, Gurjeet http://Gurje.et

Re: Regarding Installation of PostgreSQL

2023-07-18 Thread David G. Johnston
You are still in the wrong place - this is a developers list, which is only slightly less bad than sending it to a security list. We have a "general" list if you really cannot find a better place to send stuff. But in this case your complaint has to do with the pgAdmin program so its support

Re: Inefficiency in parallel pg_restore with many tables

2023-07-18 Thread Nathan Bossart
On Tue, Jul 18, 2023 at 06:05:11PM +0200, Alvaro Herrera wrote: > On 2023-Jul-17, Nathan Bossart wrote: > >> @@ -35,7 +42,11 @@ binaryheap_allocate(int capacity, binaryheap_comparator >> compare, void *arg) >> binaryheap *heap; >> >> sz = offsetof(binaryheap, bh_nodes) +

Re: Inefficiency in parallel pg_restore with many tables

2023-07-18 Thread Alvaro Herrera
On 2023-Jul-17, Nathan Bossart wrote: > @@ -35,7 +42,11 @@ binaryheap_allocate(int capacity, binaryheap_comparator > compare, void *arg) > binaryheap *heap; > > sz = offsetof(binaryheap, bh_nodes) + sizeof(Datum) * capacity; > +#ifdef FRONTEND > + heap = (binaryheap *)

Re: Giving more detail in pg_upgrade errormessage

2023-07-18 Thread Tom Lane
Daniel Gustafsson writes: > Looking at the upgrade question in [0] made me realize that we discard > potentially useful information for troubleshooting. When we check if the > cluster is properly shut down we might as well include the status from > pg_controldata in the errormessage as per the

Re: remaining sql/json patches

2023-07-18 Thread Alvaro Herrera
On 2023-Jul-18, Amit Langote wrote: > Attached updated patches. In 0002, I removed the mention of the > RETURNING clause in the JSON(), JSON_SCALAR() documentation, which I > had forgotten to do in the last version which removed its support in > code. > I think 0001 looks ready to go. Alvaro?

Re: Giving more detail in pg_upgrade errormessage

2023-07-18 Thread Mingli Zhang
Hi, > Is there a reason not to be verbose here as users might copy/paste this output > when asking for help? Seems better than nothing. > [0] CACoPQdbQTysF=ekckyfngtdpodxxmesf_2acno+bcnqqcb5...@mail.gmail.com Full link for convenience.

Giving more detail in pg_upgrade errormessage

2023-07-18 Thread Daniel Gustafsson
Looking at the upgrade question in [0] made me realize that we discard potentially useful information for troubleshooting. When we check if the cluster is properly shut down we might as well include the status from pg_controldata in the errormessage as per the trivial (but yet untested) proposed

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread Tom Lane
Vik Fearing writes: > On 7/18/23 11:30, mahendrakar s wrote: >> We have encountered an issue (invalid message length) when the >> password length is > 1000 in pg 11,12,13 versions. > The third option is to upgrade. Yeah. I don't see any good reason to consider this behavior change as something

Re: POC, WIP: OR-clause support for indexes

2023-07-18 Thread Alena Rybakina
Hi, all! I sent a patch to commitfest and noticed that the authors and the reviewer were incorrectly marked. Sorry about that. I fixed it and sent the current version of the patch. -- Regards, Alena Rybakina Postgres Professional From 087125cc413429bda05f22ebbd51115c23819285 Mon Sep 17

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread vignesh C
On Tue, 11 Jul 2023 at 08:30, Peter Smith wrote: > > On Tue, Jul 11, 2023 at 12:31 AM Melih Mutlu wrote: > > > > Hi, > > > > Hayato Kuroda (Fujitsu) , 6 Tem 2023 Per, > > 12:47 tarihinde şunu yazdı: > > > > > > Dear Melih, > > > > > > > Thanks for the 0003 patch. But it did not work for me. Can

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Ashutosh Bapat
On Fri, Jul 14, 2023 at 7:33 PM Tomas Vondra wrote: > > Thanks for testing / confirming this! So, do we agree this behavior is > reasonable? > This behaviour doesn't need any on-disk changes or has nothing in it which prohibits us from changing it in future. So I think it's good as a v0. If

Re: [RFC] Add jit deform_counter

2023-07-18 Thread Daniel Gustafsson
> On 15 Apr 2023, at 16:40, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Fri, Mar 31, 2023 at 07:39:27PM +0200, Dmitry Dolgov wrote: >>> On Wed, Mar 29, 2023 at 01:50:37PM +1300, David Rowley wrote: I had a look at this patch today and I agree that it would be good to give the user an

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2023-07-18 Thread stephane tachoires
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Only documentation patch applied on 4e465aac36ce9a9533c68dbdc83e67579880e628

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread Amit Kapila
On Tue, Jul 18, 2023 at 2:33 PM Melih Mutlu wrote: > > Attached the fixed patchset. > Few comments on 0001 1. + logicalrep_worker_attach(worker_slot); + + /* Setup signal handling */ + pqsignal(SIGHUP, SignalHandlerForConfigReload); + pqsignal(SIGTERM, die); +

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread Vik Fearing
On 7/18/23 11:30, mahendrakar s wrote: Hi hackers, We have encountered an issue (invalid message length) when the password length is > 1000 in pg 11,12,13 versions. This is due to the limit(1000) on the max length of the password. In this case the password is an access token(JWT) which can have

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread mahendrakar s
Access token length with bare minimal claims is more than 1000 in this case. Workarounds are not possible in production. On Tue, 18 Jul 2023 at 15:10, Daniel Gustafsson wrote: > > > On 18 Jul 2023, at 11:30, mahendrakar s wrote: > > > So we have two options: > > 1. Backport patch[1] to 11,12,13

Re: Protect extension' internal tables - how?

2023-07-18 Thread Nikita Malakhov
Hi, Aleksander, thank you very much. Tables are already placed into special schema, but there are some dynamically created tables and the goal is to protect all these tables from direct insert, update and delete operations from users. I've read about the SECURITY DEFINER, it will do the trick.

Re: Add TOAST support for more system tables

2023-07-18 Thread Sofia Kopikova
On Mon, Jul 17, 2023 at 06:31:04PM -0400, Tom Lane wrote: Sofia Kopikova writes: This patch adds TOAST support for system tables pg_class, pg_attribute and pg_largeobject_metadata, as they include ACL columns, which may be potentially large in size. We have been around on this topic before,

Re: In Postgres 16 BETA, should the ParseNamespaceItem have the same index as it's RangeTableEntry?

2023-07-18 Thread Amit Langote
Hello, On Sat, Jul 15, 2023 at 4:43 AM Farias de Oliveira wrote: > I believe I have found something interesting that might be the root of the > problem with RTEPermissionInfo. But I do not know how to fix it exactly. In > AGE's code, the execution of it goes through a function called >

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread Daniel Gustafsson
> On 18 Jul 2023, at 11:30, mahendrakar s wrote: > So we have two options: > 1. Backport patch[1] to 11,12,13 > 2. Change ONLY the limit on the max length of the password(my patch attached). We typically only backpatch bugfixes and not functional changes, and this seems to fall in the latter

Re: doc: improve the restriction description of using indexes on REPLICA IDENTITY FULL table.

2023-07-18 Thread Amit Kapila
On Tue, Jul 18, 2023 at 12:10 PM Masahiko Sawada wrote: > > BTW, IsIndexOnlyExpression() is not necessary but the current code > still works fine. So do we need to backpatch it to PG16? I'm thinking > we can apply it to only HEAD. > Either way is fine but I think if we backpatch it then the code

Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread mahendrakar s
Hi hackers, We have encountered an issue (invalid message length) when the password length is > 1000 in pg 11,12,13 versions. This is due to the limit(1000) on the max length of the password. In this case the password is an access token(JWT) which can have varied lengths > 1000. I see that this

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-07-18 Thread Amit Kapila
On Mon, Jul 17, 2023 at 6:19 PM Amit Kapila wrote: > > On Fri, Jun 30, 2023 at 7:29 PM Hayato Kuroda (Fujitsu) > wrote: > > > > I have analyzed more, and concluded that there are no difference between > > manual > > and shutdown checkpoint. > > > > The difference was whether the CHECKPOINT

Re: generic plans and "initial" pruning

2023-07-18 Thread Thom Brown
On Tue, 18 Jul 2023, 08:26 Amit Langote, wrote: > Hi Thom, > > On Tue, Jul 18, 2023 at 1:33 AM Thom Brown wrote: > > On Thu, 13 Jul 2023 at 13:59, Amit Langote > wrote: > > > In an absolutely brown-paper-bag moment, I realized that I had not > > > updated src/backend/executor/README to reflect

There should be a way to use the force flag when restoring databases

2023-07-18 Thread Joan
Since posgres 13 there's the option to do a FORCE when dropping a database (so it disconnects current users) Documentation here: https://www.postgresql.org/docs/current/sql-dropdatabase.html I am currently using dir format for the output pg_dump -d "bdname" -F d -j 4 -v -f /tmp/dir And

Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-07-18 Thread Richard Guo
On Fri, Jun 2, 2023 at 1:33 AM James Coleman wrote: > On Wed, May 31, 2023 at 10:30 PM Richard Guo > wrote: > > Thanks for the review! > > Sure thing! I've updated the patch according to the reviews as attached. But I did not add ORDER BY clause in the test, as we don't need it for

Re: doc: improve the restriction description of using indexes on REPLICA IDENTITY FULL table.

2023-07-18 Thread Masahiko Sawada
On Sat, Jul 15, 2023 at 2:11 PM Amit Kapila wrote: > > On Thu, Jul 13, 2023 at 10:55 AM Masahiko Sawada > wrote: > > > > On Wed, Jul 12, 2023 at 11:15 PM Masahiko Sawada > > wrote: > > > > > > > I think this is a valid concern. Can't we move all the checks > > > > (including the remote attrs