xmlserialize bug - extra empty row at the end

2023-04-22 Thread Pavel Stehule
Hi maybe I found a bug in xmlserialize SELECT xmlserialize(DOCUMENT '42' AS varchar INDENT); (2023-04-23 07:27:53) postgres=# SELECT xmlserialize(DOCUMENT '42' AS varchar INDENT); ┌─┐ │ xmlserialize │ ╞═╡ │ ↵│ │

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2023-04-22 Thread Tatsuo Ishii
> Vik Fearing writes: >> On 4/22/23 14:14, Tatsuo Ishii wrote: >>> Note that RESPECT/IGNORE are not registered as reserved keywords in >>> this patch (but registered as unreserved keywords). I am not sure if >>> this is acceptable or not. > >> For me, this is perfectly okay. Keep them at the

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2023-04-22 Thread Tatsuo Ishii
> Excellent. I was thinking about picking my version of this patch up > again, but I think this might be better than mine. Thanks. > I am curious why set_mark is false in the IGNORE version instead of > also being const_offset. Surely the nth non-null in the frame will > never go backwards.

Bufferless buffered files

2023-04-22 Thread Thomas Munro
"$SUBJECT" may sound like a Zen kōan, but buffile.c does a few useful things other than buffering (I/O time tracking, inter-process file exchange though file sets, segmentation, etc). A couple of places such as logtape.c, sharedtuplestore.c and gistbuildbuffers.c do block-oriented I/O in

Re: Mark a transaction uncommittable

2023-04-22 Thread Julien Rouhaud
Hi, On Sat, Apr 22, 2023 at 12:53:23PM -0400, Isaac Morland wrote: > > I have an application for this: creating various dev/test versions of data > from production. > > Start by restoring a copy of production from backup. Then successively > create several altered versions of the data and save

Re: Should we remove vacuum_defer_cleanup_age?

2023-04-22 Thread Andres Freund
Hi, On 2023-04-13 13:18:38 +0200, Alvaro Herrera wrote: > On 2023-Apr-11, Andres Freund wrote: > > > Updated patch attached. I think we should either apply something like that > > patch, or at least add a to the docs. > > I gave this patch a look. The only code change is that >

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Noah Misch
On Sat, Apr 22, 2023 at 04:15:23PM -0400, Tom Lane wrote: > Noah Misch writes: > > - skip if the break-glass "pgindent: no" appears in a commit message > > There are two things that bother me about putting this functionality > into a server hook, beyond the possible speed issue: > > * The risk

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Tom Lane
Noah Misch writes: > - skip if the break-glass "pgindent: no" appears in a commit message There are two things that bother me about putting this functionality into a server hook, beyond the possible speed issue: * The risk of failure. I don't have a terribly difficult time imagining situations

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Noah Misch
(Given that another commentator is "absolutely against" a hook, this message is mostly for readers considering this for other projects.) On Sat, Apr 22, 2023 at 03:23:59PM +0200, Magnus Hagander wrote: > On Tue, Feb 7, 2023 at 5:43 AM Noah Misch wrote: > > On Mon, Feb 06, 2023 at 06:17:02PM

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Tom Lane
Andrew Dunstan writes: > On 2023-04-22 Sa 11:37, Tom Lane wrote: >> * I see that there's now a 20230309 release, should we consider that >> instead? > A test I just ran gave identical results to those from 20221112 Cool, let's use perltidy 20230309 then. > The great advantage of not doing this

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Andrew Dunstan
On 2023-04-22 Sa 11:37, Tom Lane wrote: Andrew Dunstan writes: On 2023-04-22 Sa 10:39, Tom Lane wrote: Another obstacle in the way of (1) is that there was some discussion of changing perltidy version and/or options. But I don't believe we have a final proposal on that, much less committed

Re: New committers: Nathan Bossart, Amit Langote, Masahiko Sawada

2023-04-22 Thread Jonathan S. Katz
On 4/20/23 1:40 PM, Tom Lane wrote: The Core Team would like to extend our congratulations to Nathan Bossart, Amit Langote, and Masahiko Sawada, who have accepted invitations to become our newest Postgres committers. Please join me in wishing them much success and few reverts. Congratulations

Re: pg_stat_io not tracking smgrwriteback() is confusing

2023-04-22 Thread Jonathan S. Katz
On 4/19/23 1:23 PM, Andres Freund wrote: Hi, I noticed that the numbers in pg_stat_io dont't quite add up to what I expected in write heavy workloads. Particularly for checkpointer, the numbers for "write" in log_checkpoints output are larger than what is visible in pg_stat_io. That partially

Re: check_strxfrm_bug()

2023-04-22 Thread Jonathan S. Katz
On 4/19/23 9:34 PM, Thomas Munro wrote: On Wed, Apr 19, 2023 at 2:31 PM Jonathan S. Katz wrote: To be clear, is the proposal to remove both "check_strxfrm_bug" and "TRUST_STRXFRM"? Given a bunch of folks who have expertise in this area of code all agree with removing the above as part of the

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2023-04-22 Thread Tom Lane
Vik Fearing writes: > On 4/22/23 14:14, Tatsuo Ishii wrote: >> Note that RESPECT/IGNORE are not registered as reserved keywords in >> this patch (but registered as unreserved keywords). I am not sure if >> this is acceptable or not. > For me, this is perfectly okay. Keep them at the lowest

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2023-04-22 Thread Vik Fearing
On 4/22/23 14:14, Tatsuo Ishii wrote: I revisited the thread: https://www.postgresql.org/message-id/flat/CAGMVOdsbtRwE_4%2Bv8zjH1d9xfovDeQAGLkP_B6k69_VoFEgX-A%40mail.gmail.com and came up with attached POC patch (I used some varibale names appearing in the Krasiyan Andreev's patch). I really

Re: pg_collation.collversion for C.UTF-8

2023-04-22 Thread Daniel Verite
Thomas Munro wrote: > It looks like for technical reasons > inside glibc, that couldn't be done before 2.35: > > https://sourceware.org/bugzilla/show_bug.cgi?id=17318 > > That strengthens my opinion that C.UTF-8 (the real C.UTF-8 supplied > by the glibc project) isn't supposed to be

Re: Mark a transaction uncommittable

2023-04-22 Thread Isaac Morland
On Sat, 22 Apr 2023 at 11:01, Gurjeet Singh wrote: > This is a proposal for a new transaction characteristic. I haven't > written any code, yet, and am interested in hearing if others may find > this feature useful. > > Many a times we start a transaction that we never intend to commit; > for

Re: Doc limitation update proposal: include out-of-line OID usage per TOAST-ed columns

2023-04-22 Thread Gurjeet Singh
On Fri, Apr 21, 2023 at 12:14 AM Nikita Malakhov wrote: > This limitation applies not only to wide tables - it also applies to tables > where TOASTed values > are updated very often. You would soon be out of available TOAST value ID > because in case of > high frequency updates autovacuum

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Tom Lane
Andrew Dunstan writes: > On 2023-04-22 Sa 10:39, Tom Lane wrote: >> Another obstacle in the way of (1) is that there was some discussion >> of changing perltidy version and/or options. But I don't believe >> we have a final proposal on that, much less committed code. > Well, I posted a fairly

Re: Improving worst-case merge join performance with often-null foreign key

2023-04-22 Thread Tom Lane
Steinar Kaldager writes: > First-time potential contributor here. We recently had an incident due > to a sudden 1000x slowdown of a Postgres query (from ~10ms to ~10s) > due to a join with a foreign key that was often null. We found that it > was caused by a merge join with an index scan on one

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2023-04-22 Thread Gurjeet Singh
On Thu, Dec 22, 2022 at 10:07 AM Nikita Malakhov wrote: > Any suggestions on the previous message (64-bit toast value ID with > individual counters)? Was this patch ever added to CommitFest? I don't see it in the current Open Commitfest. https://commitfest.postgresql.org/43/ Best regards,

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Andrew Dunstan
On 2023-04-22 Sa 10:39, Tom Lane wrote: Another obstacle in the way of (1) is that there was some discussion of changing perltidy version and/or options. But I don't believe we have a final proposal on that, much less committed code. Well, I posted a fairly concrete suggestion with an

Mark a transaction uncommittable

2023-04-22 Thread Gurjeet Singh
This is a proposal for a new transaction characteristic. I haven't written any code, yet, and am interested in hearing if others may find this feature useful. Many a times we start a transaction that we never intend to commit; for example, for testing, or for EXPLAIN ANALYZE, or after detecting

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Tom Lane
Jelte Fennema writes: > I think there's two things needed to actually start doing this: > 1. We need to reindent the tree to create an indented baseline As far as (1) goes, I've been holding off on that because there are some large patches that still seem in danger of getting reverted, notably

Re: Move un-parenthesized syntax docs to "compatibility" for few SQL commands

2023-04-22 Thread Melanie Plageman
On Fri, Apr 21, 2023 at 09:44:51PM -0700, Nathan Bossart wrote: > I've attached two patches. 0001 adds a parenthesized CLUSTER syntax that > doesn't require a table name. 0002 is your patch with a couple of small > adjustments. > > On Fri, Apr 21, 2023 at 07:29:59PM -0400, Melanie Plageman

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Tom Lane
Magnus Hagander writes: > On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan wrote: >> For 2 the upstream thread listed two approaches: >> a. Install a pre-receive git hook on the git server that rejects >> pushes to master that are not indented >> b. Add a test suite that checks if the code is

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Andrew Dunstan
On 2023-04-22 Sa 08:47, Magnus Hagander wrote: On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan wrote: On 2023-04-22 Sa 04:50, Michael Paquier wrote: On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote: For 2 the upstream thread listed two approaches: a. Install a pre-receive git

Re: Autogenerate some wait events code and documentation

2023-04-22 Thread Drouvot, Bertrand
Hi, On 4/20/23 3:09 AM, Michael Paquier wrote: On Wed, Mar 29, 2023 at 02:51:27PM +0200, Drouvot, Bertrand wrote: Just realized that more polishing was needed. Done in V2 attached. That would be pretty cool to get that done in an automated way, I've wanted that for a few years now. And I

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Magnus Hagander
On Tue, Feb 7, 2023 at 5:43 AM Noah Misch wrote: > > On Mon, Feb 06, 2023 at 06:17:02PM +0100, Peter Eisentraut wrote: > > Also, pgindent takes tens of seconds to run, so hooking that into the git > > push process would slow this down quite a bit. > > The pre-receive hook would do a full pgindent

Improving worst-case merge join performance with often-null foreign key

2023-04-22 Thread Steinar Kaldager
Hi, First-time potential contributor here. We recently had an incident due to a sudden 1000x slowdown of a Postgres query (from ~10ms to ~10s) due to a join with a foreign key that was often null. We found that it was caused by a merge join with an index scan on one join path -- whenever the

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Magnus Hagander
On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan wrote: > > > On 2023-04-22 Sa 04:50, Michael Paquier wrote: > > On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote: > > For 2 the upstream thread listed two approaches: > a. Install a pre-receive git hook on the git server that rejects >

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2023-04-22 Thread Oliver Ford
On Sat, 22 Apr 2023, 13:14 Tatsuo Ishii, wrote: > I revisited the thread: > > https://www.postgresql.org/message-id/flat/CAGMVOdsbtRwE_4%2Bv8zjH1d9xfovDeQAGLkP_B6k69_VoFEgX-A%40mail.gmail.com > > and came up with attached POC patch (I used some varibale names > appearing in the Krasiyan

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2023-04-22 Thread Tatsuo Ishii
I revisited the thread: https://www.postgresql.org/message-id/flat/CAGMVOdsbtRwE_4%2Bv8zjH1d9xfovDeQAGLkP_B6k69_VoFEgX-A%40mail.gmail.com and came up with attached POC patch (I used some varibale names appearing in the Krasiyan Andreev's patch). I really love to have RESPECT/IGNORE NULLS because

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Michael Paquier
On Sat, Apr 22, 2023 at 07:42:36AM -0400, Andrew Dunstan wrote: > Perhaps we should start with a buildfarm module, which would run pg_indent > --show-diff. Nice, I didn't know this one and it has been mentioned a bit on this thread. Indeed, it is possible to just rely on that. -- Michael

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Andrew Dunstan
On 2023-04-22 Sa 04:50, Michael Paquier wrote: On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote: For 2 the upstream thread listed two approaches: a. Install a pre-receive git hook on the git server that rejects pushes to master that are not indented b. Add a test suite that checks

Re: Logging parallel worker draught

2023-04-22 Thread Amit Kapila
On Fri, Apr 21, 2023 at 6:34 PM Benoit Lobréau wrote: > > Following my previous mail about adding stats on parallelism[1], this > patch introduces the log_parallel_worker_draught parameter, which > controls whether a log message is produced when a backend attempts to > spawn a parallel worker but

Re: The order of queues in row lock is changed (not FIFO)

2023-04-22 Thread Amit Kapila
On Tue, Mar 7, 2023 at 4:49 PM Ryo Yamaji (Fujitsu) wrote: > > From: Tom Lane > > I don't see a bug here, or at least I'm not willing to move the goalposts > > to where you want them to be. > > I believe that we do guarantee arrival-order locking of individual tuple > > versions. However, in

Re: Mistake in freespace/README?

2023-04-22 Thread Aleksander Alekseev
Hi, > Hopefully I didn't miss or misunderstood anything. And for sure I did. Particularly the fact that the tree inside the FSM page is not a perfect binary tree: """ 0 1 2 3 4 5 6 7 8 9 A B """ So there are 13 levels and approximately 4K slots per FSM page after all:

Mistake in freespace/README?

2023-04-22 Thread Aleksander Alekseev
Hi, I think there could be a mistake in freespace/README. There are several places where it says about ~4000 slots per FSM page for the default BLKSZ: """ For example, assuming each FSM page can hold information about 4 pages (in reality, it holds (BLCKSZ - headers) / 2, or ~4000 with default

Re: Fix documentation for max_wal_size and min_wal_size

2023-04-22 Thread Fujii Masao
On 2023/04/22 17:39, Michael Paquier wrote: On Tue, Apr 18, 2023 at 01:46:21AM -0700, sirisha chamarthi wrote: "The default size is 80MB. However, if you have changed the WAL segment size from the default of 16MB with the initdb option --wal-segsize, it will be five times the segment

Re: run pgindent on a regular basis / scripted manner

2023-04-22 Thread Michael Paquier
On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote: > For 2 the upstream thread listed two approaches: > a. Install a pre-receive git hook on the git server that rejects > pushes to master that are not indented > b. Add a test suite that checks if the code is correctly indented, so >

Re: A Question about InvokeObjectPostAlterHook

2023-04-22 Thread Michael Paquier
On Fri, Apr 21, 2023 at 04:16:10PM +0800, Legs Mansion wrote: > actually, some location can be tricky to add. > it looks like CREATE, but it’s actually ALTER, should call > InvokeObjectPostAlterHook instead > ofInvokeObjectPostCreateHook? eg.,CREATE OR REPLACE, CREATE > TYPE(perfecting shell

Re: Fix documentation for max_wal_size and min_wal_size

2023-04-22 Thread Michael Paquier
On Tue, Apr 18, 2023 at 01:46:21AM -0700, sirisha chamarthi wrote: > "The default size is 80MB. However, if you have changed the WAL segment size > from the default of 16MB with the initdb option --wal-segsize, it will be > five times the segment size." Yes, I think that something close to that