Re: Logical WAL sender unresponsive during decoding commit

2022-08-15 Thread Masahiko Sawada
On Tue, Aug 16, 2022 at 2:31 PM Amit Kapila wrote: > > On Tue, Aug 16, 2022 at 10:56 AM Masahiko Sawada > wrote: > > > > On Tue, Aug 16, 2022 at 2:08 PM Amit Kapila wrote: > > > > > > On Tue, Aug 16, 2022 at 9:28 AM Andrey Borodin > > > wrote: > > > > > > > > Hi hackers! > > > > > > > > Some

Re: Logical WAL sender unresponsive during decoding commit

2022-08-15 Thread Amit Kapila
On Tue, Aug 16, 2022 at 10:56 AM Masahiko Sawada wrote: > > On Tue, Aug 16, 2022 at 2:08 PM Amit Kapila wrote: > > > > On Tue, Aug 16, 2022 at 9:28 AM Andrey Borodin wrote: > > > > > > Hi hackers! > > > > > > Some time ago I've seen a hanging logical replication that was trying to > > > send

[PG15 Doc] remove "tty" connect string from manual

2022-08-15 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hello, hackers. As of PostgreSQL 14, "tty" in the libpq connection string has already been removed with the commit below. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=14d9b37607ad30c3848ea0f2955a78436eff1268 But

Re: Propose a new function - list_is_empty

2022-08-15 Thread Peter Smith
On Tue, Aug 16, 2022 at 11:27 AM Tom Lane wrote: > > Peter Smith writes: > > During a recent code review I was going to suggest that some new code > > would be more readable if the following: > > if (list_length(alist) == 0) ... > > > was replaced with: > > if (list_is_empty(alist)) ... > > >

Re: Logical WAL sender unresponsive during decoding commit

2022-08-15 Thread Masahiko Sawada
On Tue, Aug 16, 2022 at 2:08 PM Amit Kapila wrote: > > On Tue, Aug 16, 2022 at 9:28 AM Andrey Borodin wrote: > > > > Hi hackers! > > > > Some time ago I've seen a hanging logical replication that was trying to > > send transaction commit after doing table pg_repack. > > I understand that those

Re: pg_upgrade test writes to source directory

2022-08-15 Thread Andres Freund
Hi, On 2022-08-15 20:20:51 -0700, Andres Freund wrote: > On 2022-08-11 11:26:39 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2022-06-01 10:55:28 -0400, Tom Lane wrote: > > >> [...] I'm definitely not happy with the proposed changes to > > >> 010_tab_completion.pl. My recollection

Re: Logical WAL sender unresponsive during decoding commit

2022-08-15 Thread Amit Kapila
On Tue, Aug 16, 2022 at 9:28 AM Andrey Borodin wrote: > > Hi hackers! > > Some time ago I've seen a hanging logical replication that was trying to send > transaction commit after doing table pg_repack. > I understand that those things do not mix well. Yet walsender was ignoring >

Re: Support logical replication of global object commands

2022-08-15 Thread Amit Kapila
On Fri, Aug 12, 2022 at 5:41 PM Amit Kapila wrote: > > On Tue, Aug 9, 2022 at 1:31 AM Zheng Li wrote: > > > > Hello, > > > > Logical replication of DDL commands support is being worked on in [1]. > > However, global object commands are quite different from other > > non-global object DDL

Re: Cleaning up historical portability baggage

2022-08-15 Thread Thomas Munro
On Tue, Aug 16, 2022 at 1:16 PM Andres Freund wrote: > > But let's suppose we want to play by a timid interpretation of that page's > > "do not issue low-level or STDIO.H I/O routines". It also says that SIGINT > > is special and runs the handler in a new thread (in a big warning box > > because

Logical WAL sender unresponsive during decoding commit

2022-08-15 Thread Andrey Borodin
Hi hackers! Some time ago I've seen a hanging logical replication that was trying to send transaction commit after doing table pg_repack. I understand that those things do not mix well. Yet walsender was ignoring pg_terminate_backend() and I think this worth fixing. Can we add

Re: pg_upgrade test writes to source directory

2022-08-15 Thread Andres Freund
Hi, On 2022-08-11 11:26:39 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2022-06-01 10:55:28 -0400, Tom Lane wrote: > >> [...] I'm definitely not happy with the proposed changes to > >> 010_tab_completion.pl. My recollection is that those tests > >> were intentionally written to test

Re: SELECT documentation

2022-08-15 Thread Bruce Momjian
On Sat, Aug 13, 2022 at 10:21:26PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > Hi, I agree we should show the more modern JOIN sytax. However, this is > > just an example, so one example should be sufficient. I went with the > > first one in the attached patch. > > You should not remove

Re: Wrong comment in statscmds.c/CreateStatistics?

2022-08-15 Thread Junwang Zhao
Yeah, the comments are kind of confusing, see some comments inline. On Tue, Aug 16, 2022 at 8:47 AM Peter Smith wrote: > > I happened to notice the following code in > src/backend/commands/statscmds.c, CreateStatistics: > > == > /* > * Parse the statistics kinds. > * > * First check that if

Add find_in_log() and advance_wal() perl functions to core test framework (?)

2022-08-15 Thread Bharath Rupireddy
Hi, It seems like find_in_log() and advance_wal() functions (which are now being used in at least 2 places). find_in_log() is defined and being used in 2 places 019_replslot_limit.pl and 033_replay_tsp_drops.pl. The functionality of advancing WAL is implemented in 019_replslot_limit.pl with

Re: SQL/JSON features for v15

2022-08-15 Thread Andres Freund
Hi, On 2022-08-16 04:02:17 +0300, Nikita Glukhov wrote: > Hi, > > > On 16.08.2022 01:38, Andres Freund wrote: > > Continuation from the thread at > > https://postgr.es/m/20220811171740.m5b4h7x63g4lzgrk%40awork3.anarazel.de > > > > > > I started hacking on this Friday. I think there's some

Re: fix typos

2022-08-15 Thread John Naylor
On Fri, Aug 12, 2022 at 8:55 PM Tom Lane wrote: > > John Naylor writes: > > This is really a straw-man proposal, since I'm not volunteering to do > > the work, or suggest anybody else should do the same. That being the > > case, it seems we should just go ahead with Justin's patch for > >

Re: Propose a new function - list_is_empty

2022-08-15 Thread Peter Smith
On Tue, Aug 16, 2022 at 11:27 AM Tom Lane wrote: > > Peter Smith writes: > > During a recent code review I was going to suggest that some new code > > would be more readable if the following: > > if (list_length(alist) == 0) ... > > > was replaced with: > > if (list_is_empty(alist)) ... > > >

Re: Propose a new function - list_is_empty

2022-08-15 Thread Tom Lane
Peter Smith writes: > During a recent code review I was going to suggest that some new code > would be more readable if the following: > if (list_length(alist) == 0) ... > was replaced with: > if (list_is_empty(alist)) ... > but then I found that actually no such function exists. That's

Propose a new function - list_is_empty

2022-08-15 Thread Peter Smith
During a recent code review I was going to suggest that some new code would be more readable if the following: if (list_length(alist) == 0) ... was replaced with: if (list_is_empty(alist)) ... but then I found that actually no such function exists. ~~~ Searching the PG source found many cases

Re: Cleaning up historical portability baggage

2022-08-15 Thread Andres Freund
Hi, On 2022-08-16 13:02:55 +1200, Thomas Munro wrote: > On Fri, Aug 12, 2022 at 7:42 PM Thomas Munro wrote: > > On Fri, Aug 12, 2022 at 5:14 AM Andres Freund wrote: > > > I don't really know what to do about the warnings around remove_temp() and > > > trapsig(). I think we actually may be

Re: SQL/JSON features for v15

2022-08-15 Thread Andres Freund
Hi, On 2022-08-15 15:38:53 -0700, Andres Freund wrote: > Next question: > > /* >* We should catch exceptions of category ERRCODE_DATA_EXCEPTION and >* execute the corresponding ON ERROR behavior then. >*/ > oldcontext = CurrentMemoryContext; > oldowner =

Re: Cleaning up historical portability baggage

2022-08-15 Thread Thomas Munro
On Fri, Aug 12, 2022 at 7:42 PM Thomas Munro wrote: > On Fri, Aug 12, 2022 at 5:14 AM Andres Freund wrote: > > I don't really know what to do about the warnings around remove_temp() and > > trapsig(). I think we actually may be overreading the restrictions. To me > > the > > documented

Wrong comment in statscmds.c/CreateStatistics?

2022-08-15 Thread Peter Smith
I happened to notice the following code in src/backend/commands/statscmds.c, CreateStatistics: == /* * Parse the statistics kinds. * * First check that if this is the case with a single expression, there * are no statistics kinds specified (we don't allow that for the simple * CREATE

Re: SQL/JSON features for v15

2022-08-15 Thread Andres Freund
Hi, Continuation from the thread at https://postgr.es/m/20220811171740.m5b4h7x63g4lzgrk%40awork3.anarazel.de On 2022-08-11 10:17:40 -0700, Andres Freund wrote: > On 2022-08-11 13:08:27 -0400, Jonathan S. Katz wrote: > > With RMT hat on, Andres do you have any thoughts on this? > > I think I

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-15 Thread Nathan Bossart
On Mon, Aug 15, 2022 at 08:33:21PM +0700, John Naylor wrote: > The attached implements the above, more or less, using new pg_lfind8() > and pg_lfind8_le(), which in turn are based on helper functions that > act on a single vector. The pg_lfind* functions have regression tests, > but I haven't done

identifying the backend that owns a temporary schema

2022-08-15 Thread Nathan Bossart
Hi hackers, As Greg Stark noted elsewhere [0], it is presently very difficult to identify the PID of the session using a temporary schema, which is particularly unfortunate when a temporary table is putting a cluster in danger of transaction ID wraparound. I noted [1] that the following query

Re: Cleaning up historical portability baggage

2022-08-15 Thread Thomas Munro
On Tue, Aug 16, 2022 at 7:51 AM Thomas Munro wrote: > [1] https://cirrus-ci.com/task/4643322672185344?logs=main#L16 Derp, I noticed that that particular horrendous quick and dirty test code was invalidated by a closesocket() call, but in another version I commented that out and it didn't help.

Re: Expose Parallelism counters planned/execute in pg_stat_statements

2022-08-15 Thread Daymel Bonne Solís
Hi: We have rewritten the patch and added the necessary columns to have the > number of times a parallel query plan was not executed using parallelism. > > This version includes comments on the source code and documentation. Regards v3-0001-Add-parallel-counters-to-pg_stat_statements.patch

Re: Cleaning up historical portability baggage

2022-08-15 Thread Thomas Munro
On Tue, Aug 16, 2022 at 7:25 AM Andres Freund wrote: > On 2022-08-15 13:48:22 +1200, Thomas Munro wrote: > > 2022-08-13 20:44:35.174 GMT [4760][postmaster] LOG: listening on Unix > > socket "@c:/cirrus/.s.PGSQL.61696" > > What I find odd is that you said your naive program rejected this... No,

Re: Cleaning up historical portability baggage

2022-08-15 Thread Andres Freund
Hi, On 2022-08-15 13:48:22 +1200, Thomas Munro wrote: > On Sun, Aug 14, 2022 at 10:36 AM Andres Freund wrote: > > On 2022-08-14 10:03:19 +1200, Thomas Munro wrote: > > > I hadn't paid attention to our existing abstract Unix socket support > > > before and now I'm curious: do we have a confirmed

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-15 Thread Ranier Vilela
Em seg., 15 de ago. de 2022 às 15:34, Ranier Vilela escreveu: > Hi, > > I ran this test. > > DROP TABLE IF EXISTS long_json_as_text; > CREATE TABLE long_json_as_text AS > with long as ( > select repeat(description, 11) > from pg_description > ) > select (select json_agg(row_to_json(long))::text

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-15 Thread Ranier Vilela
Hi, I ran this test. DROP TABLE IF EXISTS long_json_as_text; CREATE TABLE long_json_as_text AS with long as ( select repeat(description, 11) from pg_description ) select (select json_agg(row_to_json(long))::text as t from long) from generate_series(1, 100); VACUUM FREEZE long_json_as_text;

Re: build remaining Flex files standalone

2022-08-15 Thread Andres Freund
Hi, Thanks for your work on this! On 2022-08-13 15:39:06 +0700, John Naylor wrote: > Here are the rest. Most of it was pretty straightforward, with the > main exception of jsonpath_scan.c, which is not quite finished. That > one passes tests but still has one compiler warning. I'm unsure how >

Re: Allow logical replication to copy tables in binary format

2022-08-15 Thread Melih Mutlu
Euler Taveira , 11 Ağu 2022 Per, 20:16 tarihinde şunu yazdı: > My main concern is to break a scenario that was previously working (14 -> > 15) but after a subscriber upgrade > it won't (14 -> 16). > Fair concern. Some cases that might break the logical replication with version upgrade would be:

Re: Include the dependent extension information in describe command.

2022-08-15 Thread vignesh C
On Sun, Aug 14, 2022 at 10:24 PM vignesh C wrote: > > On Sun, Aug 14, 2022 at 11:07 AM Tom Lane wrote: > > > > vignesh C writes: > > > Currently we do not include the dependent extension information for > > > index and materialized view in the describe command. I felt it would > > > be useful

Re: Tab completion for "ALTER TYPE typename SET" and rearranged "Alter TYPE typename RENAME"

2022-08-15 Thread vignesh C
On Mon, Aug 15, 2022 at 10:42 AM Michael Paquier wrote: > > On Sun, Aug 14, 2022 at 07:56:00PM +0530, vignesh C wrote: > > Modified the patch to list all the properties in case of "ALTER TYPE > > typename SET (". I have included the properties in alphabetical order > > as I notice that the

Re: Header checker scripts should fail on failure

2022-08-15 Thread Andres Freund
Hi, On 2022-08-15 17:38:21 +1200, Thomas Munro wrote: > I thought commit 81b9f23c9c8 had my back, but nope, we still need to > make CI turn red if "headerscheck" and "cpluspluscheck" don't like our > patches (crake in the build farm should be a secondary defence...). > See attached. +1

Re: shared-memory based stats collector - v70

2022-08-15 Thread Greg Stark
On Thu, 11 Aug 2022 at 02:11, Drouvot, Bertrand wrote: > > As Andres was not -1 about that idea (as it should be low cost to add > and maintain) as long as somebody cares enough to write something: then > I'll give it a try and submit a patch for it. I agree it would be a useful feature. I think

Re: Header checker scripts should fail on failure

2022-08-15 Thread Andrew Dunstan
On 2022-08-15 Mo 01:38, Thomas Munro wrote: > Hi, > > I thought commit 81b9f23c9c8 had my back, but nope, we still need to > make CI turn red if "headerscheck" and "cpluspluscheck" don't like our > patches (crake in the build farm should be a secondary defence...). > See attached. Yeah, the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-08-15 Thread John Naylor
On Mon, Aug 15, 2022 at 12:39 PM Masahiko Sawada wrote: > > On Fri, Jul 22, 2022 at 10:43 AM Masahiko Sawada > wrote: > > > > On Tue, Jul 19, 2022 at 1:30 PM John Naylor > > wrote: > > > > > > > > > > > > On Tue, Jul 19, 2022 at 9:11 AM Masahiko Sawada > > > wrote: > > > > > > > I’d like to

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-15 Thread John Naylor
I wrote > On Mon, Jul 11, 2022 at 11:07 PM Andres Freund wrote: > > > I wonder if we can add a somewhat more general function for scanning until > > some characters are found using SIMD? There's plenty other places that could > > be useful. > > In simple cases, we could possibly abstract the

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2022-08-15 Thread Damir Belyalov
> > > Thank you for feedback. I improved my patch recently and tested it on different sizes of MAX_BUFFERED_TUPLES and REPLAY_BUFFER_SIZE. > I loaded 1 rows which contained 1 wrong row. > I expected I could see rows after COPY, but just saw 999 rows. Also I implemented your case and it

pg_receivewal and SIGTERM

2022-08-15 Thread Christoph Berg
There's a smallish backup tool called pg_backupcluster in Debian's postgresql-common which also ships a systemd service that runs pg_receivewal for wal archiving, and supplies a pg_getwal script for reading the files back on restore, including support for .partial files. So far the machinery was

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2022-08-15 Thread torikoshia
On 2022-07-19 21:40, Damir Belyalov wrote: Hi! Improved my patch by adding block subtransactions. The block size is determined by the REPLAY_BUFFER_SIZE parameter. I used the idea of a buffer for accumulating tuples in it. If we read REPLAY_BUFFER_SIZE rows without errors, the subtransaction

Re: ICU for global collation

2022-08-15 Thread Marina Polyakova
Hello everyone in this thread! While reading and testing the patch that adds ICU for global collations [1] I noticed on master (1c5818b9c68e5c2ac8f19d372f24cce409de1a26) and REL_15_STABLE (63b64d8270691894a9a8f2d4e929e7780020edb8) that: 1) pg_upgrade from REL_14_STABLE

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2022-08-15 Thread Melih Mutlu
Hi Amit, Amit Kapila , 6 Ağu 2022 Cmt, 16:01 tarihinde şunu yazdı: > I think there is some basic flaw in slot reuse design. Currently, we > copy the table by starting a repeatable read transaction (BEGIN READ > ONLY ISOLATION LEVEL REPEATABLE READ) and create a slot that > establishes a snapshot

Fwd: Merging statistics from children instead of re-sampling everything

2022-08-15 Thread Damir Belyalov
> > 3) stadistinct - This is quite problematic. We only have the per-child > estimates, and it's not clear if there's any overlap. For now I've just > summed it up, because that's safer / similar to what we do for gather > merge paths etc. Maybe we could improve this by estimating the overlap >

Re: Cleaning up historical portability baggage

2022-08-15 Thread Thomas Munro
On Mon, Aug 15, 2022 at 8:36 PM Peter Eisentraut wrote: > On 15.08.22 03:48, Thomas Munro wrote: > >> I vaguely remember successfully trying it in the past. But I just tried it > >> unsuccessfully in a VM and there's a bunch of other places saying it's not > >> working... > >>

Re: Making Vars outer-join aware

2022-08-15 Thread Richard Guo
On Tue, Aug 2, 2022 at 3:51 AM Tom Lane wrote: > Here's a rebase up to HEAD, mostly to placate the cfbot. > I accounted for d8e34fa7a (s/all_baserels/all_query_rels/ > in those places) and made one tiny bug-fix change. > Nothing substantive as yet. When we add required PlaceHolderVars to a

Remove remaining mentions of UNSAFE_STAT_OK

2022-08-15 Thread Peter Eisentraut
The last use of UNSAFE_STAT_OK was removed in bed90759fcbcd72d4d06969eebab81e47326f9a2, but the build system(s) still mentions it. Is it safe to remove, or does it interact with system header files in some way that isn't obvious here?From 594268d54fd344348aa547bc0d3fa6393255a52b Mon Sep 17

Re: Cleaning up historical portability baggage

2022-08-15 Thread Peter Eisentraut
On 15.08.22 03:48, Thomas Munro wrote: I vaguely remember successfully trying it in the past. But I just tried it unsuccessfully in a VM and there's a bunch of other places saying it's not working... https://github.com/microsoft/WSL/issues/4240 I think we'd better remove our claim that it works

Re: latest patches not updated in repos - [External Email]

2022-08-15 Thread Graaff, Selwyn
Thank you. regards, Selwyn From: Devrim Gündüz Sent: Sunday, 14 August 2022 16:25 To: Tom Lane ; Graaff, Selwyn Cc: pgsql-hackers@lists.postgresql.org Subject: Re: latest patches not updated in repos - [External Email] Hi, On Sun, 2022-08-14 at 10:10 -0400,

Re: Use SetInstallXLogFileSegmentActive() for setting XLogCtl->InstallXLogFileSegmentActive

2022-08-15 Thread Bharath Rupireddy
On Fri, Aug 12, 2022 at 4:17 AM Nathan Bossart wrote: > > On Thu, Aug 11, 2022 at 09:42:18PM +0530, Bharath Rupireddy wrote: > > Here's a small patch replacing the explicit setting of > > XLogCtl->InstallXLogFileSegmentActive with the existing function > > SetInstallXLogFileSegmentActive(),

Re: Assertion failure in WaitForWALToBecomeAvailable state machine

2022-08-15 Thread Bharath Rupireddy
On Thu, Aug 11, 2022 at 10:06 PM Bharath Rupireddy wrote: > > Today I encountered the assertion failure [2] twice while working on > another patch [1]. The pattern seems to be that the walreceiver got > killed or crashed and set it's state to WALRCV_STOPPING or > WALRCV_STOPPED by the team the