Re: Synchronizing slots from primary to standby

2024-03-31 Thread Amit Kapila
On Mon, Apr 1, 2024 at 10:01 AM Bharath Rupireddy wrote: > > On Thu, Mar 28, 2024 at 10:08 AM Zhijie Hou (Fujitsu) > wrote: > > > > [2] The steps to reproduce the data miss issue on a primary->standby setup: > > I'm trying to reproduce the problem with [1], but I can see the > changes after the

Re: Synchronizing slots from primary to standby

2024-03-31 Thread Bharath Rupireddy
On Thu, Mar 28, 2024 at 10:08 AM Zhijie Hou (Fujitsu) wrote: > > [2] The steps to reproduce the data miss issue on a primary->standby setup: I'm trying to reproduce the problem with [1], but I can see the changes after the standby is promoted. Am I missing anything here?

Re: remaining sql/json patches

2024-03-31 Thread jian he
typedef struct JsonTableExecContext { int magic; JsonTablePlanState *rootplanstate; JsonTablePlanState **colexprplans; } JsonTableExecContext; imho, this kind of naming is kind of inconsistent. "state" and "plan" are mixed together. maybe typedef struct JsonTableExecContext { int magic;

Re: Improve eviction algorithm in ReorderBuffer

2024-03-31 Thread Masahiko Sawada
On Mon, Apr 1, 2024 at 11:26 AM Masahiko Sawada wrote: > > On Fri, Mar 29, 2024 at 7:37 PM Amit Kapila wrote: > > > > On Fri, Mar 29, 2024 at 12:13 PM Masahiko Sawada > > wrote: > > > > > > On Fri, Mar 29, 2024 at 2:09 PM vignesh C wrote: > > > > > > > > On Tue, 26 Mar 2024 at 10:05, Masahiko

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-31 Thread Amit Kapila
On Fri, Mar 29, 2024 at 6:17 PM Bertrand Drouvot wrote: > > On Fri, Mar 29, 2024 at 03:03:01PM +0530, Amit Kapila wrote: > > On Fri, Mar 29, 2024 at 11:49 AM Bertrand Drouvot > > wrote: > > > > > > On Fri, Mar 29, 2024 at 09:39:31AM +0530, Amit Kapila wrote: > > > > > > > > Commit message

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-31 Thread Bharath Rupireddy
On Fri, Mar 29, 2024 at 9:39 AM Amit Kapila wrote: > > Commit message states: "why we can't just update inactive_since for > synced slots on the standby with the value received from remote slot > on the primary. This is consistent with any other slot parameter i.e. > all of them are synced from

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-31 Thread Masahiko Sawada
On Fri, Mar 29, 2024 at 4:21 PM John Naylor wrote: > > On Thu, Mar 28, 2024 at 12:55 PM Masahiko Sawada > wrote: > > I think the patch is in good shape. Do you have other comments or > > suggestions, John? > > --- a/doc/src/sgml/config.sgml > +++ b/doc/src/sgml/config.sgml > @@ -1918,11 +1918,6

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-03-31 Thread Masahiko Sawada
On Fri, Mar 29, 2024 at 11:54 AM torikoshia wrote: > > On 2024-03-28 21:54, Masahiko Sawada wrote: > > On Thu, Mar 28, 2024 at 9:38 PM torikoshia > > wrote: > >> > >> On 2024-03-28 10:20, Masahiko Sawada wrote: > >> > Hi, > >> > > >> > On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada > >> >

Re: Improve eviction algorithm in ReorderBuffer

2024-03-31 Thread Masahiko Sawada
On Fri, Mar 29, 2024 at 8:48 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > > Agreed. > > > > I think the patch is in good shape. I'll push the patch with the > > suggestion next week, barring any objections. > > Thanks for working on this. Agreed it is committable. > Few minor

Re: Improve eviction algorithm in ReorderBuffer

2024-03-31 Thread Masahiko Sawada
On Fri, Mar 29, 2024 at 7:37 PM Amit Kapila wrote: > > On Fri, Mar 29, 2024 at 12:13 PM Masahiko Sawada > wrote: > > > > On Fri, Mar 29, 2024 at 2:09 PM vignesh C wrote: > > > > > > On Tue, 26 Mar 2024 at 10:05, Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Mar 14, 2024 at 12:02 PM

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-31 Thread Bharath Rupireddy
On Mon, Apr 1, 2024 at 5:54 AM Alexander Korotkov wrote: > > > 9. To me the following query blocks even though I didn't mention timeout. > > CALL pg_wal_replay_wait('0/fff'); > > If your primary server is freshly initialized, you need to do quite > data modifications to reach this LSN.

Re: Extract numeric filed in JSONB more effectively

2024-03-31 Thread Andy Fan
Here is latest version, nothing changed besides the rebase to the latest master. The most recent 3 questions should be addressed. - The error message compatible issue [1] and the Peter's answer at [2]. - Peter's new question at [2] and my answer at [3]. Any effrot to move this patch ahead is

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > IIRC, "variadic any" requires having at least one variadic parameter. > But that seems fine --- what would be the point, or even the > semantics, of calling pg_set_attribute_stats with no data fields? > If my pg_dump run emitted a bunch of stats that could never be imported, I'd want to know.

Re: Popcount optimization using AVX512

2024-03-31 Thread Nathan Bossart
On Sat, Mar 30, 2024 at 03:03:29PM -0500, Nathan Bossart wrote: > My current plan is to add some new tests for > pg_popcount() with many bytes, and then I'll give it a few more days for > any additional feedback before committing. Here is a v18 with a couple of new tests. Otherwise, it is the

Re: Add new error_action COPY ON_ERROR "log"

2024-03-31 Thread Masahiko Sawada
On Sat, Mar 30, 2024 at 11:05 PM Bharath Rupireddy wrote: > > On Thu, Mar 28, 2024 at 6:31 PM Masahiko Sawada wrote: > > > > That is, > > since the LOG_VERBOSITY option is an enum parameter, it might make > > more sense to require the value, instead of making the value optional. > > For example,

RE: Synchronizing slots from primary to standby

2024-03-31 Thread Zhijie Hou (Fujitsu)
On Friday, March 29, 2024 2:50 PM Amit Kapila wrote: > > On Fri, Mar 29, 2024 at 6:36 AM Zhijie Hou (Fujitsu) > wrote: > > > > > > Attach a new version patch which fixed an un-initialized variable > > issue and added some comments. > > > > The other approach to fix this issue could be that the

Cant link libpq with a pg extension using cmake

2024-03-31 Thread Tony Wayne
i am getting linking issues when trying to link libpq with my pg extension and i am using pg's libpq ,so libpq is built along with pg,so i did this in my extension's cmakelists.txt file (GLOB storage_SRC CONFIGURE_DEPENDS "*.cpp" ) add_library(storage OBJECT ${storage_SRC})

Re: Statistics Import and Export

2024-03-31 Thread Tom Lane
Corey Huinker writes: >> and I really think that we need to provide >> the source server's major version number --- maybe we will never >> need that, but if we do and we don't have it we will be sad. > The JSON had it, and I never did use it. Not against having it again. Well, you don't need it

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-31 Thread Alexander Korotkov
Hi Bharath, Thank you for your feedback. On Sun, Mar 31, 2024 at 8:44 AM Bharath Rupireddy wrote: > On Sun, Mar 31, 2024 at 7:41 AM Alexander Korotkov > wrote: > Thanks for the patch. I have a few comments on the v16 patch. > > 1. Is there any specific reason for pg_wal_replay_wait() being a

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > > I concur with the plan of extracting data from pg_stats not > pg_statistics, and with emitting a single "set statistics" > call per attribute. (I think at one point I'd suggested a call > per stakind slot, but that would lead to a bunch of UPDATEs on > existing pg_attribute tuples and hence

Re: remaining sql/json patches

2024-03-31 Thread jian he
FAILED: src/interfaces/ecpg/test/sql/sqljson_jsontable.c /home/jian/postgres/buildtest6/src/interfaces/ecpg/preproc/ecpg --regression -I../../Desktop/pg_src/src6/postgres/src/interfaces/ecpg/test/sql -I../../Desktop/pg_src/src6/postgres/src/interfaces/ecpg/include/ -o

Re: Catalog domain not-null constraints

2024-03-31 Thread jian he
On Tue, Mar 26, 2024 at 2:28 AM Dean Rasheed wrote: > > On Fri, 22 Mar 2024 at 08:28, jian he wrote: > > > > On Thu, Mar 21, 2024 at 7:23 PM Peter Eisentraut > > wrote: > > > > > > Hmm. CREATE DOMAIN uses column constraint syntax, but ALTER DOMAIN uses > > > table constraint syntax. Attached

Re: Statistics Import and Export

2024-03-31 Thread Tom Lane
Corey Huinker writes: >> I can't quibble with that view of what has priority. I'm just >> suggesting that redesigning what pg_upgrade does in this area >> should come later than doing something about extended stats. > I mostly agree, with the caveat that pg_upgrade's existing message saying >

Re: Psql meta-command conninfo+

2024-03-31 Thread Imseih (AWS), Sami
>. However, > I didn't complete item 4. I'm not sure, but I believe that linking it to the > documentation > could confuse the user a bit. I chose to keep the descriptions as they were. > However, if > you have any ideas on how we could outline it, let me know and perhaps we can > implement it.

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > I wonder if the right answer to that is "let's enhance the I/O > functions for those types". But whether that helps or not, it's > v18-or-later material for sure. > That was Stephen's take as well, and I agreed given that I had to throw the kitchen-sink of source-side oid mappings (attname,

Re: Statistics Import and Export

2024-03-31 Thread Tom Lane
Corey Huinker writes: > On Sun, Mar 31, 2024 at 2:41 PM Tom Lane wrote: >> There's a bigger issue though: AFAICS this patch set does nothing >> about dumping extended statistics. I surely don't want to hold up >> the patch insisting that that has to happen before we can commit the >>

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
On Sun, Mar 31, 2024 at 2:41 PM Tom Lane wrote: > Corey Huinker writes: > > Having given this some thought, I'd be inclined to create a view, > > pg_stats_missing, with the same security barrier as pg_stats, but looking > > for tables that lack stats on at least one column, or lack stats on an

RE: Psql meta-command conninfo+

2024-03-31 Thread Maiquel Grassi
Note that, in the patch as posted, the column names are not translatable. In order to be translatable, the code needs to do something like appendPQExpBuffer(, " NULL AS \"%s\",\n" " NULL AS \"%s\",\n" " NULL AS \"%s\",\n" " NULL

Re: Security lessons from liblzma

2024-03-31 Thread Andres Freund
Hi, On 2024-03-31 12:18:29 +0200, Michael Banck wrote: > If you ask where they are maintained, the answer is here: > > https://salsa.debian.org/postgresql/postgresql/-/tree/17/debian/patches?ref_type=heads > > the other major versions have their own branch. Luckily these are all quite small,

Re: Security lessons from liblzma

2024-03-31 Thread Tom Lane
Michael Banck writes: > On Sun, Mar 31, 2024 at 01:05:40PM -0400, Joe Conway wrote: >> But it has always bothered me how many patches get applied to the upstream >> tarballs by the OS package builders. > I think this more an artifact of how RHEL development works, i.e. trying > to ship the same

Re: Add column name to error description

2024-03-31 Thread Tom Lane
Erik Wienhold writes: > On 2024-03-31 15:22 +0200, Marcos Pegoraro wrote: >> This is my first patch, so sorry if I miss something. > Please make sure that tests are passing by running make check: check-world, in fact. > The format "%d-%s" is not ideal. I suggesst "%d (%s)". I didn't like

Re: Security lessons from liblzma

2024-03-31 Thread Michael Banck
Hi, On Sun, Mar 31, 2024 at 01:05:40PM -0400, Joe Conway wrote: > But it has always bothered me how many patches get applied to the upstream > tarballs by the OS package builders. Some of them, e.g. glibc on RHEL 7, > include more than 1000 patches that you would have to manually vet if you >

Re: Add column name to error description

2024-03-31 Thread Erik Wienhold
On 2024-03-31 15:22 +0200, Marcos Pegoraro wrote: > This is my first patch, so sorry if I miss something. Please make sure that tests are passing by running make check: https://www.postgresql.org/docs/current/regress-run.html#REGRESS-RUN-TEMP-INST The patch breaks

Re: Statistics Import and Export

2024-03-31 Thread Tom Lane
My apologies for having paid so little attention to this thread for months. I got around to reading the v15 patches today, and while I think they're going in more or less the right direction, there's a long way to go IMO. I concur with the plan of extracting data from pg_stats not pg_statistics,

Re: Statistics Import and Export

2024-03-31 Thread Tom Lane
Corey Huinker writes: > Having given this some thought, I'd be inclined to create a view, > pg_stats_missing, with the same security barrier as pg_stats, but looking > for tables that lack stats on at least one column, or lack stats on an > extended statistics object. The week before feature

Re: Security lessons from liblzma

2024-03-31 Thread Joe Conway
On 3/31/24 11:49, Tom Lane wrote: Joe Conway writes: I am saying maybe those patches should be eliminated in favor of our tree including build options that would produce the same result. I don't really see how that can be expected to work sanely. It turns the responsibility for

Re: Security lessons from liblzma

2024-03-31 Thread Devrim Gündüz
Hi, On Sat, 2024-03-30 at 21:52 -0400, Bruce Momjian wrote: > How would someone access the out-of-tree patches?  Here are the v17 patches: https://git.postgresql.org/gitweb/?p=pgrpms.git;a=tree;f=rpm/redhat/main/non-common/postgresql-17/main You can replace -17 with -16 (and etc) for the other

Re: Security lessons from liblzma

2024-03-31 Thread Tom Lane
Joe Conway writes: > I am saying maybe those patches should be eliminated in favor of our > tree including build options that would produce the same result. I don't really see how that can be expected to work sanely. It turns the responsibility for platform-specific build issues on its head,

Re: New Table Access Methods for Multi and Single Inserts

2024-03-31 Thread Bharath Rupireddy
On Wed, Mar 27, 2024 at 1:42 PM Jeff Davis wrote: > > On Wed, 2024-03-27 at 01:19 +0530, Bharath Rupireddy wrote: > > > > Similarly, with this new AM, the onus lies on the table AM > > implementers to provide an implementation for these new AMs even if > > they just do single inserts. > > Why not

Re: Security lessons from liblzma

2024-03-31 Thread Devrim Gündüz
Hi, On Sun, 2024-03-31 at 08:15 -0400, Joe Conway wrote: > > I am saying maybe those patches should be eliminated in favor of our > tree including build options that would produce the same result. Works for me, as a long as I can commit them and upcoming potential patches to PostgreSQL git

Re: Building with musl in CI and the build farm

2024-03-31 Thread walther
About building one of the CI tasks with musl: Andres Freund: I'd rather adapt one of the existing tasks, to avoid increasing CI costs unduly. I looked into this and I think the only task that could be changed is the SanityCheck. This is because this builds without any additional features

Re: pgbnech: allow to cancel queries during benchmark

2024-03-31 Thread Yugo NAGATA
On Sat, 30 Mar 2024 14:35:37 +0100 Alvaro Herrera wrote: > Due to commit 61461a300c1c, this patch needs to be reworked. Thank you for pointing out this. Although cfbot doesn't report any failure, but PQcancel is now deprecated and insecure. I'll consider it too while fixing a problem I found

Add column name to error description

2024-03-31 Thread Marcos Pegoraro
This is my first patch, so sorry if I miss something. If I have a function which returns lots of columns and any of these columns returns a wrong type it's not easy to see which one is that column because it points me only to its position, not its name. So, this patch only adds that column name,

Re: Security lessons from liblzma

2024-03-31 Thread Joe Conway
On 3/30/24 21:52, Bruce Momjian wrote: On Sat, Mar 30, 2024 at 07:54:00PM -0400, Joe Conway wrote: Virtually every RPM source, including ours, contains out of tree patches that get applied on top of the release tarball. At least for the PGDG packages, it would be nice to integrate them into our

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > That will make it *really* hard for any form of automation or drivers of > this. The information needs to go somewhere where such tools can easily > consume it, and an informational message during runtime (which is also > likely to be translated in many environments) is the exact opposite of

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-31 Thread Justin Pryzby
On Sun, Mar 31, 2024 at 12:00:00PM +0300, Alexander Lakhin wrote: > Hello Alvaro, > > 28.03.2024 18:58, Alvaro Herrera wrote: > > Grumble. I don't like initialization at declare time, so far from the > > code that depends on the value. But the alternative would have been to > > assign right

Re: Security lessons from liblzma

2024-03-31 Thread Michael Banck
On Sat, Mar 30, 2024 at 09:52:47PM -0400, Bruce Momjian wrote: > On Sat, Mar 30, 2024 at 07:54:00PM -0400, Joe Conway wrote: > > Virtually every RPM source, including ours, contains out of tree patches > > that get applied on top of the release tarball. At least for the PGDG > > packages, it would

Re: [PATCH] kNN for btree

2024-03-31 Thread Andrey M. Borodin
> On 15 Jan 2024, at 13:11, Anton A. Melnikov wrote: > > If there are any ideas pro and contra would be glad to discuss them. Hi, Anton! This is kind of ancient thread. I've marked CF entry [0] as "Needs review" and you as an author (seems like you are going to be a point of correspondence

Re: logicalrep_worker_launch -- counting/checking the worker limits

2024-03-31 Thread Andrey M. Borodin
> On 15 Aug 2023, at 07:38, Peter Smith wrote: > > A rebase was needed due to a recent push [1]. > > PSA v3. > On 14 Jan 2024, at 10:43, vignesh C wrote: > > I have changed the status of the patch to "Waiting on Author" as > Amit's queries at [1] have not been verified and concluded.

Re: Wrong results with grouping sets

2024-03-31 Thread Andrey M. Borodin
> On 11 Jan 2024, at 20:10, vignesh C wrote: > > I have changed the status of the patch to "Waiting on Author" as Tom > Lane's comments have not yet been addressed, feel free to address them > and update the commitfest entry accordingly. This CF entry seems to be a fix for actually

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-31 Thread Alexander Lakhin
Hello Alvaro, 28.03.2024 18:58, Alvaro Herrera wrote: Grumble. I don't like initialization at declare time, so far from the code that depends on the value. But the alternative would have been to assign right where this blocks starts, an additional line. I pushed it like you had it. I've