Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

Hi, On 2023-03-31 16:57:41 +0300, Alexander Korotkov wrote: > On Wed, Mar 29, 2023 at 8:34 PM Alexander Korotkov > wrote: > > On Mon, Mar 27, 2023 at 1:49 PM Alexander Korotkov > > wrote: > > > On Fri, Mar 24, 2023 at 3:39 AM Andres Freund wrote: > > > > On 2023-03-23 23:24:19 +0300,

Re: Minimal logical decoding on standbys

On Fri, Mar 31, 2023 at 7:14 PM Drouvot, Bertrand wrote: > > On 3/31/23 1:58 PM, Amit Kapila wrote: > > On Fri, Mar 31, 2023 at 4:17 PM Drouvot, Bertrand > > wrote: > >> > > > > + * This is needed for logical decoding on standby. Indeed the "problem" is > > that > > + * WalSndWaitForWal() waits

Re: regression coverage gaps for gist and hash indexes

Hi, On 4/1/23 1:13 AM, Andres Freund wrote: Hi, On 2023-03-31 17:00:00 +0300, Alexander Lakhin wrote: 31.03.2023 15:55, Tom Lane wrote: See also the thread about bug #16329 [1]. Alexander promised to look into improving the test coverage in this area, maybe he can keep an eye on the WAL

Re: WL_SOCKET_ACCEPT fairness on Windows

On Sat, Apr 1, 2023 at 2:35 PM Andres Freund wrote: > I wonder if we ought to bite the bullet and replace the use of > WaitForMultipleObjects() with RegisterWaitForSingleObject() and then use > GetQueuedCompletionStatus() to wait. The fairness issue here is a motivation, > but the bigger one is

Re: WL_SOCKET_ACCEPT fairness on Windows

Hi, On 2023-04-01 13:42:21 +1300, Thomas Munro wrote: > Commit 7389aad6 started using WaitEventSetWait() to wait for incoming > connections. Before that, we used select(), for which we have our own > implementation for Windows. > > While hacking on patches to rip a bunch of unused Win32 socket

RE: Non-superuser subscription owners

On Saturday, April 1, 2023 4:00 AM Robert Haas Hi, > > On Thu, Mar 30, 2023 at 9:49 PM houzj.f...@fujitsu.com > wrote: > > It looks like the super user check is out of a transaction, I haven't > > checked why it only failed on one BF animal, but it seems we can put > > the check into the

WL_SOCKET_ACCEPT fairness on Windows

Hi, Commit 7389aad6 started using WaitEventSetWait() to wait for incoming connections. Before that, we used select(), for which we have our own implementation for Windows. While hacking on patches to rip a bunch of unused Win32 socket wrapper code out, I twigged that the old pgwin32_select()

Re: zstd compression for pg_dump

On Sat, Apr 01, 2023 at 02:11:12AM +0200, Tomas Vondra wrote: > On 4/1/23 01:16, Justin Pryzby wrote: > > On Tue, Mar 28, 2023 at 06:23:26PM +0200, Tomas Vondra wrote: > >> On 3/27/23 19:28, Justin Pryzby wrote: > >>> On Fri, Mar 17, 2023 at 03:43:31AM +0100, Tomas Vondra wrote: > On 3/16/23

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Fri, Mar 31, 2023 at 8:05 PM David Rowley wrote: > > On Sat, 1 Apr 2023 at 12:57, Melanie Plageman > wrote: > > I've attached v7 with that commit dropped and with support for parallel > > vacuum workers to use the same number of buffers in their own Buffer > > Access Strategy ring as the

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Sat, Apr 01, 2023 at 01:05:19PM +1300, David Rowley wrote: > Attached is what I propose in .diff form so that the CFbot can run on > your v7 patches without picking this up. But it processes .diff, too https://wiki.postgresql.org/wiki/Cfbot#Which_attachments_are_considered_to_be_patches.3F

Re: zstd compression for pg_dump

On 4/1/23 01:16, Justin Pryzby wrote: > On Tue, Mar 28, 2023 at 06:23:26PM +0200, Tomas Vondra wrote: >> On 3/27/23 19:28, Justin Pryzby wrote: >>> On Fri, Mar 17, 2023 at 03:43:31AM +0100, Tomas Vondra wrote: On 3/16/23 05:50, Justin Pryzby wrote: > On Fri, Mar 10, 2023 at 12:48:13PM

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Sat, 1 Apr 2023 at 12:57, Melanie Plageman wrote: > I've attached v7 with that commit dropped and with support for parallel > vacuum workers to use the same number of buffers in their own Buffer > Access Strategy ring as the main vacuum phase did. I also updated the > docs to indicate that

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Fri, Mar 31, 2023 at 5:47 PM David Rowley wrote: > > On Sat, 1 Apr 2023 at 02:52, Melanie Plageman > wrote: > > There was one small typo keeping this from compiling. Also a repeated > > word. I've fixed these. I also edited a bit of indentation and tweaked > > some wording. Diff attached (to

Re: RFC: logical publication via inheritance root?

On Fri, Mar 31, 2023 at 3:17 PM Peter Smith wrote: > OK, my understanding is that TimescaleDB uses some kind of > quasi-partitioned/inherited tables (aka hypertables? [1]) internally, > and your proposed WIP patch provides a set_logical_root() function > which combines with the logical

Re: Add LZ4 compression in pg_dump

On 3/31/23 11:19, gkokola...@pm.me wrote: > >> ... >> >> >> I think the patch is fine, but I'm wondering if the renames shouldn't go >> a bit further. It removes references to LZ4File struct, but there's a >> bunch of functions with LZ4File_ prefix. Why not to simply use LZ4_ >> prefix? We don't

Re: zstd compression for pg_dump

On Tue, Mar 28, 2023 at 06:23:26PM +0200, Tomas Vondra wrote: > On 3/27/23 19:28, Justin Pryzby wrote: > > On Fri, Mar 17, 2023 at 03:43:31AM +0100, Tomas Vondra wrote: > >> On 3/16/23 05:50, Justin Pryzby wrote: > >>> On Fri, Mar 10, 2023 at 12:48:13PM -0800, Jacob Champion wrote: > On Wed,

Re: regression coverage gaps for gist and hash indexes

Hi, On 2023-03-31 17:00:00 +0300, Alexander Lakhin wrote: > 31.03.2023 15:55, Tom Lane wrote: > > See also the thread about bug #16329 [1]. Alexander promised to look > > into improving the test coverage in this area, maybe he can keep an > > eye on the WAL logic coverage too. > > Yes, I'm

Re: Data is copied twice when specifying both child and parent table in publication

On 3/31/23 03:04, shiy.f...@fujitsu.com wrote: > I noticed that a similar problem has been discussed in this thread, see [1] > [2] > [3] [4]. Ah, thank you. I didn't go far back enough in the thread... > It seems complicated to fix it if we want to automatically skip tables > that have been

Re: Data is copied twice when specifying both child and parent table in publication

On 3/30/23 20:01, Peter Smith wrote: > For example, Just imagine if logic could be made smarter to recognize > that since there was already the 'part_def' being subscribed so it > should NOT use the default 'copy_data=true' when the REFRESH launches > the ancestor table 'part'... > > Even if that

Re: running logical replication as the subscription owner

On Fri, 2023-03-31 at 15:17 -0400, Robert Haas wrote: > I think that's too Boolean. The special case in 0001 is a better > solution for the cases where it works. It's both more granular and > more convenient. I guess the "more convenient" is where I'm confused, because the "grant

Re: RFC: logical publication via inheritance root?

On Wed, Mar 1, 2023 at 9:47 AM Jacob Champion wrote: > > Hi, > > I'm going to register this in CF for feedback. > > Summary for potential reviewers: we don't use declarative partitions in > the Timescale partitioning scheme, but it'd be really nice to be able to > replicate between our tables and

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Sat, 1 Apr 2023 at 02:52, Melanie Plageman wrote: > There was one small typo keeping this from compiling. Also a repeated > word. I've fixed these. I also edited a bit of indentation and tweaked > some wording. Diff attached (to be applied on top of your diff). Thanks for fixing that mistake.

Re: Add pg_walinspect function with block info columns

On Thu, Mar 30, 2023 at 2:41 PM Peter Geoghegan wrote: > There is still an outstanding question around the overhead of > outputting FPIs and even block data from pg_get_wal_block_info(). At > one point Melanie suggested that we'd need to do something about that, > and I tend to agree. Attached

Re: Making background psql nicer to use in tap tests

The attached v4 fixes some incorrect documentation (added by me in v3), and fixes that background_psql() didn't honor on_error_stop and extraparams passed by the user. I've also added a commit which implements the \password test from the SCRAM iteration count patchset as well as cleaned up a few

Re: Non-superuser subscription owners

On Thu, Mar 30, 2023 at 9:49 PM houzj.f...@fujitsu.com wrote: > It looks like the super user check is out of a transaction, I haven't checked > why > it only failed on one BF animal, but it seems we can put the check into the > transaction like the following: That looks like a reasonable fix

Re: Minimal logical decoding on standbys

Hi, On 2023-03-30 21:33:00 -0700, Andres Freund wrote: > > diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h > > index 2ce9366277..93fb9d438a 100644 > > --- a/src/include/access/gistxlog.h > > +++ b/src/include/access/gistxlog.h > > @@ -51,11 +51,14 @@ typedef struct

Re: Track IO times in pg_stat_io

Attached is a rebased version in light of 8aaa04b32d - Melanie From 789d4bf1fb749a26523dbcd2c69795916b711c68 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Tue, 21 Mar 2023 16:00:55 -0400 Subject: [PATCH v8 1/4] Count IO time for temp relation writes Both pgstat_database and

Re: Schema variables - new implementation for Postgres 15

> On Tue, Mar 28, 2023 at 09:34:20PM +0200, Pavel Stehule wrote: > Hi > > > Talking about documentation I've noticed that the implementation > > contains few limitations, that are not mentioned in the docs. Examples > > are WITH queries: > > > > WITH x AS (LET public.svar = 100) SELECT * FROM

Re: BUG #17877: Referencing a system column in a foreign key leads to incorrect memory access

Ranier Vilela writes: > I think that commit f0d65c0 > > has an oversight. > Attnum == 0, is system column too, right? No, it's not valid in pg_attribute rows. > All other places at tablecmds.c, has this test:

Re: running logical replication as the subscription owner

On Fri, Mar 31, 2023 at 3:36 AM Jeff Davis wrote: > That's moving the goalposts a little, though: > > "Some concern was expressed ... might break things that are currently > working..." > > https://www.postgresql.org/message-id/CA+TgmoaE35kKS3-zSvGiZszXP9Tb9rNfYzT=+fo8ehk5edk...@mail.gmail.com >

Re: Should vacuum process config file reload more often

On Fri, Mar 31, 2023 at 10:31 AM Melanie Plageman wrote: > > On Thu, Mar 30, 2023 at 3:26 PM Daniel Gustafsson wrote: > > > > > On 30 Mar 2023, at 04:57, Masahiko Sawada wrote: > > > > > As another idea, why don't we use macros for that? For example, > > > suppose VacuumCostStatus is like: > >

Re: [BUG] pg_stat_statements and extended query protocol

> So... The idea here is to set a custom fetch size so as the number of > calls can be deterministic in the tests, still more than 1 for the > tests we'd have. And your point is that libpq enforces always 0 when > sending the EXECUTE message causing it to always return all the rows > for any

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

On 3/31/23 02:14, Daniel Gustafsson wrote: >> On 14 Mar 2023, at 20:20, Jacob Champion wrote: > >> Rebased over yesterday's Meson changes in v8. > > I had a look at this and agree that it's something we should do. Great, thanks for the review! > + # Let tests differentiate between vanilla

Re: [RFC] Add jit deform_counter

> On Wed, Mar 29, 2023 at 01:50:37PM +1300, David Rowley wrote: > On Sun, 12 Jun 2022 at 21:14, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > I've noticed that JIT performance counter generation_counter seems to > > include > > actions, relevant for both jit_expressions and jit_tuple_deforming

Re: SELECT INTO without columns or star

On Fri, Mar 31, 2023 at 11:26 AM Tom Lane wrote: > "David G. Johnston" writes: > > On Fri, Mar 31, 2023 at 8:10 AM Zhang Mingli > wrote: > >> When I exec a sql SELECT INTO without columns or * by mistake, it > succeeds: > > > Yes, a table may have zero columns by design. > > Yup, we've allowed

Re: SQL/JSON revisited

Here's v14 of the IS JSON predicate. I find no further problems here and intend to get it pushed later today. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Siempre hay que alimentar a los dioses, aunque la tierra esté seca" (Orual) >From

Re: BUG #17877: Referencing a system column in a foreign key leads to incorrect memory access

Hi, I think that commit f0d65c0 has an oversight. Attnum == 0, is system column too, right? All other places at tablecmds.c, has this test: if (attnum <= 0) ereport(ERROR, regards, Ranier Vilela

Re: regression coverage gaps for gist and hash indexes

Hi, On 2023-03-31 10:45:51 +0300, Andrey Borodin wrote: > On Fri, Mar 31, 2023 at 8:07 AM Andres Freund wrote: > > > > I was working on committing patch 0001 from [1] and was a bit confused about > > some of the changes to the WAL format for gist and hash index vacuum. It > > looked to me that

Re: SELECT INTO without columns or star

"David G. Johnston" writes: > On Fri, Mar 31, 2023 at 8:10 AM Zhang Mingli wrote: >> When I exec a sql SELECT INTO without columns or * by mistake, it succeeds: > Yes, a table may have zero columns by design. Yup, we've allowed that for some time now; see the compatibility comments at the

Re: SELECT INTO without columns or star

On Fri, Mar 31, 2023 at 8:10 AM Zhang Mingli wrote: > When I exec a sql SELECT INTO without columns or * by mistake, it succeeds: > > Yes, a table may have zero columns by design. David J.

SELECT INTO without columns or star

Hi, hackers When I exec a sql SELECT INTO without columns or * by mistake, it succeeds: select * from t1;  a | b ---+---  1 | 2  2 | 3  3 | 4 (3 rows) select into t2 from t1; SELECT 3  \pset null '(null)' Null display is "(null)". select * from t2; -- (3 rows) It seems that t2 has empty rows

Re: SQL JSON path enhanced numeric literals

Hi! Sorry to bother, but there is a question on JsonPath - how many bits in the JsonPath header could be used for the version? The JsonPath header is 4 bytes, and currently the Version part is defined as #define JSONPATH_VERSION (0x01) Thanks! On Sun, Mar 5, 2023 at 6:55 PM Peter Eisentraut <

Re: Should vacuum process config file reload more often

On Thu, Mar 30, 2023 at 3:26 PM Daniel Gustafsson wrote: > > > On 30 Mar 2023, at 04:57, Masahiko Sawada wrote: > > > As another idea, why don't we use macros for that? For example, > > suppose VacuumCostStatus is like: > > > > typedef enum VacuumCostStatus > > { > >

Re: regression coverage gaps for gist and hash indexes

31.03.2023 15:55, Tom Lane wrote: See also the thread about bug #16329 [1]. Alexander promised to look into improving the test coverage in this area, maybe he can keep an eye on the WAL logic coverage too. Yes, I'm going to analyze that area too. Maybe it'll take more time (a week or two) if

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

On Wed, Mar 29, 2023 at 8:34 PM Alexander Korotkov wrote: > On Mon, Mar 27, 2023 at 1:49 PM Alexander Korotkov > wrote: > > On Fri, Mar 24, 2023 at 3:39 AM Andres Freund wrote: > > > On 2023-03-23 23:24:19 +0300, Alexander Korotkov wrote: > > > > On Thu, Mar 23, 2023 at 8:06 PM Andres Freund

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Thu, Mar 30, 2023 at 11:54 PM David Rowley wrote: > > On Mon, 20 Mar 2023 at 11:50, Melanie Plageman > wrote: > > Attached is an updated v6. > > I had a look over the v6-0001 patch. There are a few things I think > could be done better: > > "Some operations will access a large number of

Re: Minimal logical decoding on standbys

Hi, On 3/31/23 1:58 PM, Amit Kapila wrote: On Fri, Mar 31, 2023 at 4:17 PM Drouvot, Bertrand wrote: + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new

Re: FW: Add the ability to limit the amount of memory that can be allocated to backends.

On Thu, 2023-03-30 at 16:11 +, John Morris wrote: >  Hi Reid! > Some thoughts > I was looking at lmgr/proc.c, and I see a potential integer overflow > - bothmax_total_bkend_mem and result are declared as “int”, so the > expression “max_total_bkend_mem * 1024 * 1024 - result * 1024 * 1024” >

Re: regression coverage gaps for gist and hash indexes

Andrey Borodin writes: > On Fri, Mar 31, 2023 at 8:07 AM Andres Freund wrote: >> Turns out the problem is that we don't reach deletion for hash and gist >> vacuum: > GiST logs deletions in gistXLogUpdate(), which is covered. > gistXLogDelete() is only used for cleaning during page splits. I'd >

Re: Infinite Interval

I hurried too much on the previous patch. It introduced other problems. Attached is a better patch and also fixes problem below #select 'infinity'::interval * 0; ?column? -- infinity (1 row) with the patch we see #select 'infinity'::interval * 0; 2023-03-31 18:00:43.131 IST [240892]

Re: Memory leak from ExecutorState context?

On Tue, 28 Mar 2023 17:25:49 +0200 Tomas Vondra wrote: ... > * Note that BufFile structs are allocated with palloc(), and therefore > * will go away automatically at query/transaction end. Since the > underlying > * virtual Files are made with OpenTemporaryFile, all resources for > * the

Re: Minimal logical decoding on standbys

On Fri, Mar 31, 2023 at 4:17 PM Drouvot, Bertrand wrote: > + * This is needed for logical decoding on standby. Indeed the "problem" is that + * WalSndWaitForWal() waits for the *replay* LSN to increase, but gets woken up + * by walreceiver when new WAL has been flushed. Which means that

Re: TAP output format in pg_regress

> On 29 Mar 2023, at 22:08, Daniel Gustafsson wrote: > >> On 29 Mar 2023, at 21:14, Daniel Gustafsson wrote: > >> Ugh, I clearly should've stayed on the couch yesterday. > > Maybe today as well, just as I had sent this I realized there is mention of > the > output format in the docs that I

Re: Infinite Interval

On Tue, Mar 28, 2023 at 7:17 PM Ashutosh Bapat wrote: > make sure that every > operator that interval as one of its operands or the result has been > covered in the code. time_mi_time - do we want to add an Assert to make sure that this function does not produce an Interval structure which

Re: ICU locale validation / canonicalization

On Thu, 2023-03-30 at 08:59 +0200, Peter Eisentraut wrote: > I don't think we should show the notice when the canonicalization > doesn't change anything.  This is not useful: > > +NOTICE:  using language tag "und-u-kf-upper" for locale "und-u-kf- > upper" Done. > Also, the message should be

RE: Data is copied twice when specifying both child and parent table in publication

On Fri, Mar 31, 2023 2:16 AM Jacob Champion wrote: > > On Wed, Mar 29, 2023 at 2:00 AM Amit Kapila > wrote: > > Pushed. > > While rebasing my logical-roots patch over the top of this, I ran into > another situation where mixed viaroot settings can duplicate data. The > key idea is to subscribe

Re: Minimal logical decoding on standbys

On Fri, 2023-03-31 at 01:31 +0900, Masahiko Sawada wrote: > I think that we don't need to change for the latter case as > WalSndWait() perfectly works. As for the former cases, since we need > to wait for CV, timeout, or socket writable we can use > ConditionVariableEventSleep(). For this patch

Re: Minimal logical decoding on standbys

On Wed, 2023-03-08 at 11:25 +0100, Drouvot, Bertrand wrote: > - Maybe ConditionVariableEventSleep() should take care of the > “WaitEventSetWait returns 1 and cvEvent.event == WL_POSTMASTER_DEATH” > case? Thank you, done. I think the nearby line was also wrong, returning true when there was no

Re: PGdoc: add ID attribute to create_publication.sgml

On Thu, Mar 30, 2023 at 3:48 PM Amit Kapila wrote: > > On Thu, Mar 30, 2023 at 9:22 AM Peter Smith wrote: > > > > I have marked the CF entry for this patch as "ready for committer" > > > > LGTM. I'll push this tomorrow unless there are more comments for it. > Pushed. -- With Regards, Amit

Re: Add LZ4 compression in pg_dump

--- Original Message --- On Wednesday, March 29th, 2023 at 12:02 AM, Tomas Vondra wrote: > > > On 3/28/23 18:07, gkokola...@pm.me wrote: > > > --- Original Message --- > > On Friday, March 24th, 2023 at 10:30 AM, gkokola...@pm.me gkokola...@pm.me > > wrote: > > > > >

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

> On 14 Mar 2023, at 20:20, Jacob Champion wrote: > Rebased over yesterday's Meson changes in v8. I had a look at this and agree that it's something we should do. The patch seems quite close to committable, I just have a few comments on it: + # Let tests differentiate between vanilla OpenSSL

Re: pg_basebackup: Correct type of WalSegSz

> On 30 Mar 2023, at 08:24, Peter Eisentraut > wrote: > > The pg_basebackup code has WalSegSz as uint32, whereas the rest of the code > has it as int. This seems confusing, and using the extra range wouldn't > actually work. This was in the original commit (fc49e24fa6), but I suppose > it

Re: Experiments with Postgres and SSL

And the cfbot wants a small tweak From 3d0a502c25504da32b7a362831c700b4e891f79b Mon Sep 17 00:00:00 2001 From: Greg Stark Date: Fri, 31 Mar 2023 02:31:31 -0400 Subject: [PATCH v6 5/6] Allow pipelining data after ssl request --- src/backend/postmaster/postmaster.c | 54

Re: regression coverage gaps for gist and hash indexes

Hi! On Fri, Mar 31, 2023 at 8:07 AM Andres Freund wrote: > > I was working on committing patch 0001 from [1] and was a bit confused about > some of the changes to the WAL format for gist and hash index vacuum. It > looked to me that the changed code just flat out would not work. > > Turns out

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

On Tue, Mar 21, 2023 at 6:44 PM Aleksander Alekseev < aleksan...@timescale.com> wrote: Okay, the changes look good. To go further, I think we need to combine into two patches, one with 0001-0003 and one with 0004: 1. Correct false statements about "shutdown" etc. This should contain changes that

Re: running logical replication as the subscription owner

On Thu, 2023-03-30 at 16:08 -0400, Robert Haas wrote: > But the run_as_owner option applies to the entire subscription. > If A activates that option, then B's hypothetical triggers that run > afoul of the SECURITY_RESTRICTED_OPERATION restrictions start working > again (woohoo!) but they're now

Re: Experiments with Postgres and SSL

On Mon, 20 Mar 2023 at 16:31, Greg Stark wrote: > > Here's a first cut at ALPN support. > > Currently it's using a hard coded "Postgres/3.0" protocol Apparently that is explicitly disrecommended by the IETF folk. They want something like "TBD" so people don't start using a string until it's been