Emit fewer vacuum records by reaping removable tuples during pruning

%40mail.gmail.com From 15e826c114f2df220c8e08ea0ad045bd462e43da Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Mon, 13 Nov 2023 14:10:05 -0500 Subject: [PATCH v1 3/3] Set would-be dead items LP_UNUSED while pruning If there are no indexes on a relation, items can be marked LP_UNUSED instead

lazy_scan_heap() should release lock on buffer before vacuuming FSM

. ISTM the fix (attached) is just to move down the call to FreeSpaceMapVacuumRange() to after we've released the lock and recorded the space we just freed. - Melanie From 27e459b357085993a35126fe55ec73eb73ae2e87 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Mon, 13 Nov 2023 16:39:25 -0500

Re: Eager page freeze criteria clarification

;aggressiveness" knob. Vacuum has the average error rate over the past vacuums, the average page age calculated over the past vacuums, and the target freeze duration (from the user set guc). We need some kind of aggressiveness knob, but I'm not sure if it is a page age threshold or something else. - Melanie From

Re: Eager page freeze criteria clarification

On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote: > > Robert, Melanie and I spent an evening discussing this topic around > pgconf.nyc. Here are, mildly revised, notes from that: Thanks for taking notes! > The main thing we are worried about is repeated freezing / unfreezing of > pages

Re: how to manage Cirrus on personal repository

On Fri, Sep 29, 2023 at 7:11 AM Daniel Gustafsson wrote: > > > On 29 Sep 2023, at 11:13, Peter Eisentraut wrote: > > > Does anyone have an idea how to manage this better? Is there maybe a way > > to globally set "only trigger manually", or could we make one? > > On my personal repo I only

Re: Eliminate redundant tuple visibility check in vacuum

nction or not. Oh, and, one more note. I've dropped the former patch 0001 which changed the function comment about off_loc above heap_page_prune(). I have plans to write a separate patch adding an error context callback for HOT pruning with the offset number and would include such a change in that patch. -

Re: Eager page freeze criteria clarification

On Fri, Sep 8, 2023 at 12:07 AM Andres Freund wrote: > > Hi, > > On 2023-09-06 10:35:17 -0400, Robert Haas wrote: > > On Wed, Sep 6, 2023 at 1:09 AM Andres Freund wrote: > > > Yea, it'd be useful to have a reasonably approximate wall clock time for > > > the > > > last modification of a page.

Re: Eager page freeze criteria clarification

On Wed, Sep 27, 2023 at 5:27 PM Peter Geoghegan wrote: > > On Wed, Sep 27, 2023 at 1:45 PM Andres Freund wrote: > > I am much more concerned about cases where > > opportunistic freezing requires an FPI - it'll often *still* be the right > > choice to freeze the page, but we need a way to prevent

Re: Eager page freeze criteria clarification

On Wed, Sep 27, 2023 at 7:39 PM Peter Geoghegan wrote: > > On Wed, Sep 27, 2023 at 4:09 PM Melanie Plageman > wrote: > > At the risk of seeming too execution-focused, I want to try and get more > > specific. Here is a description of an example implementation to test

Re: Eager page freeze criteria clarification

On Wed, Sep 27, 2023 at 3:25 PM Robert Haas wrote: > > On Wed, Sep 27, 2023 at 12:34 PM Andres Freund wrote: > > One way to deal with that would be to not track the average age in > > LSN-difference-bytes, but convert the value to some age metric at that > > time. If we e.g. were to convert the

Re: Eager page freeze criteria clarification

On Sat, Sep 23, 2023 at 3:53 PM Melanie Plageman wrote: > > Workload F: > > +--++-++--+ > | algo | WAL GB | cptr bgwriter writes| other reads/writes | IO ti

Re: Eager page freeze criteria clarification

On Mon, Aug 28, 2023 at 4:30 PM Melanie Plageman wrote: > On Mon, Aug 28, 2023 at 12:26 PM Robert Haas wrote: > > In row D, your algorithms are all bad, really bad. I don't quite > > understand how it can be that bad, actually. > > So, I realize now that this test was poo

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

On Fri, Sep 15, 2023 at 9:24 AM Nazir Bilal Yavuz wrote: > I found that pgBufferUsage.blk_{read|write}_time are zero although there are > pgBufferUsage.local_blks_{read|written} Yes, good catch. This is a bug. I will note that at least in 15 and likely before, pgBufferUsage.local_blks_written

Re: Eliminate redundant tuple visibility check in vacuum

On Fri, Sep 8, 2023 at 11:06 AM Robert Haas wrote: > > On Thu, Sep 7, 2023 at 6:23 PM Melanie Plageman > wrote: > > I mostly wanted to remove the NULL checks because I found them > > distracting (so, a stylistic complaint). However, upon further > > reflection, I

Re: Eliminate redundant tuple visibility check in vacuum

On Thu, Sep 7, 2023 at 3:30 PM Robert Haas wrote: > > On Thu, Sep 7, 2023 at 3:10 PM Melanie Plageman > wrote: > > I can fix it by changing the type of PruneResult->off_loc to be an > > OffsetNumber pointer. This does mean that PruneResult will be > > initialize

Re: Eliminate redundant tuple visibility check in vacuum

On Thu, Sep 7, 2023 at 1:37 PM Robert Haas wrote: > > On Wed, Sep 6, 2023 at 5:21 PM Melanie Plageman > wrote: > > Yeah, I think this is a fair concern. I have addressed it in the > > attached patches. > > > > I thought a lot about whether or not adding a Pr

Re: Eliminate redundant tuple visibility check in vacuum

On Wed, Sep 6, 2023 at 1:04 PM Robert Haas wrote: > > On Thu, Aug 31, 2023 at 6:29 PM Melanie Plageman > wrote: > > I have changed this. > > I spent a bunch of time today looking at this, thinking maybe I could > commit it. But then I got cold feet. > > With thes

Re: Opportunistically pruning page before update

On Wed, Jun 21, 2023 at 8:51 AM James Coleman wrote: > While at PGCon I was chatting with Andres (and I think Peter G. and a > few others who I can't remember at the moment, apologies) and Andres > noted that while we opportunistically prune a page when inserting a > tuple (before deciding we

Re: Show various offset arrays for heap WAL records

On Mon, Jul 10, 2023 at 3:44 AM Heikki Linnakangas wrote: > I'm late to the party, but regarding commit c03c2eae0a, which added the > guidelines for writing formatting desc functions: > > You moved the comment from rmgrdesc_utils.c into rmgrdesc_utils.h, but I > don't think that was a good idea.

Re: Commitfest 2023-09 starts soon

On Mon, Sep 4, 2023 at 1:01 PM Peter Eisentraut wrote: > > I have done several passes to make sure that patch statuses are more > accurate. As explained in a nearby message, I have set several patches > back from "Ready to Committer" to "Needs review" if additional > discussion happened past the

Re: Unlogged relation copy is not fsync'd

On Fri, Aug 25, 2023 at 8:47 AM Heikki Linnakangas wrote: > > I noticed another missing fsync() with unlogged tables, similar to the > one at [1]. > > RelationCopyStorage does this: > > > /* > >* When we WAL-logged rel pages, we must nonetheless fsync them. The > >* reason

Re: Why doesn't Vacuum FULL update the VM

On Fri, Sep 1, 2023 at 8:38 PM Peter Geoghegan wrote: > > On Fri, Sep 1, 2023 at 12:34 PM Melanie Plageman > wrote: > > I don't see why the visibility map shouldn't be updated so that all of > > the pages show all visible and all frozen for this relation afte

Why doesn't Vacuum FULL update the VM

Hi, I noticed that VACUUM FULL actually does freeze the tuples in the rewritten table (heap_freeze_tuple()) but then it doesn't mark them all visible or all frozen in the visibility map. I don't understand why. It seems like it would save us future work. Here is an example: create extension

Re: Eliminate redundant tuple visibility check in vacuum

On Thu, Aug 31, 2023 at 5:39 AM David Geier wrote: > Regarding the 2nd patch (disclaimer: I'm not too familiar with that area > of the code): I don't completely understand why the retry loop is not > needed anymore and how you now detect/handle the possible race > condition? It can still happen

Re: Eliminate redundant tuple visibility check in vacuum

y.c and pruneheap.c and looked at the commit that added the retry loop (8523492d4e349) to see everywhere it added comments and don't see anywhere else that needs updating. I have updated lazy_scan_prune()'s function header comment to describe the nature of the in-out and output parameters and the c

Re: Eliminate redundant tuple visibility check in vacuum

> On Tue, Aug 29, 2023 at 5:07 AM David Geier wrote: > > Could you measure any performance difference? > > > > If so could you provide your test case? > > I created a large table and then updated a tuple on every page in the > relation and vacuumed it. I saw a consistent slight improvement in >

Re: Sync scan & regression tests

On Wed, Aug 30, 2023 at 5:15 PM David Rowley wrote: > > On Tue, 29 Aug 2023 at 22:35, Heikki Linnakangas wrote: > > Looking the new heapgettup_advance_block() function and the code that it > > replaced, it's now skipping this ss_report_location() on the last call, > > when it has reached the end

Re: Eliminate redundant tuple visibility check in vacuum

Hi David, Thanks for taking a look! On Tue, Aug 29, 2023 at 5:07 AM David Geier wrote: > > Hi Melanie, > > On 8/29/23 01:49, Melanie Plageman wrote: > > While working on a set of patches to combine the freeze and visibility > > map WAL records into the prune record, I wr

Eliminate redundant tuple visibility check in vacuum

. It also gets rid of the retry loop in lazy_scan_prune(). - Melanie From bdb432b220571086077085b29d3c90a3a1c68c47 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Thu, 3 Aug 2023 15:37:39 -0400 Subject: [PATCH v1 2/2] Reuse heap_page_prune() tuple visibility statuses heap_page_prune() obtains

Re: Eager page freeze criteria clarification

On Mon, Aug 28, 2023 at 5:06 PM Peter Geoghegan wrote: > > On Mon, Aug 28, 2023 at 1:17 PM Robert Haas wrote: > > I'm sure this could be implemented, but it's unclear to me why you > > would expect it to perform well. Freezing a page that has no frozen > > tuples yet isn't cheaper than freezing

Re: Eager page freeze criteria clarification

On Mon, Aug 28, 2023 at 12:26 PM Robert Haas wrote: > > On Mon, Aug 28, 2023 at 10:00 AM Melanie Plageman > wrote: > Then there's the question of whether it's the right metric. My first > reaction is to think that it sounds pretty good. One thing I really > like about it is

Re: Commitfest manager for September

On Mon, Aug 28, 2023 at 11:36 AM Aleksander Alekseev wrote: > > Hi Peter, > > > I would like to be the commitfest manager for CF 2023-09. > > Many thanks for volunteering! If at some point you will require a bit > of help please let me know. I too had planned to volunteer to help. I volunteer to

Re: Eager page freeze criteria clarification

On Fri, Jul 28, 2023 at 3:27 PM Melanie Plageman wrote: > On Fri, Jul 28, 2023 at 3:00 PM Peter Geoghegan wrote: > > > Is this test meant to guard against unnecessary freezing or to avoid > > > freezing when the cost is too high? That is, are we trying to >

Re: Show WAL write and fsync stats in pg_stat_io

ntext of replication. > - Adding WAIT_EVENT_WAL_COPY_* operations to pg_stat_io if needed. > (This could be an another patch) Yes, I think it makes sense as another patch. > > On Sat, 22 Jul 2023 at 01:30, Melanie Plageman > wrote: > > I think it would be good to count WAL re

Re: Eager page freeze criteria clarification

On Fri, Jul 28, 2023 at 4:49 PM Peter Geoghegan wrote: > > On Fri, Jul 28, 2023 at 4:30 PM Andres Freund wrote: > > > Put differently, I can't see any reason to care whether pruning > > > emitted an FPI or not. Either way, it's very unlikely that freezing > > > needs to do so. > > > > +many > >

Re: Eager page freeze criteria clarification

On Fri, Jul 28, 2023 at 3:00 PM Peter Geoghegan wrote: > > On Fri, Jul 28, 2023 at 11:13 AM Melanie Plageman > wrote: > > if (pagefrz.freeze_required || tuples_frozen == 0 || > > (prunestate->all_visible && prunestate->all_frozen && &g

Eager page freeze criteria clarification

Hi, While hacking on the pruning and page-level freezing code and am a bit confused by the test guarding eager freezing [1]: /* * Freeze the page when heap_prepare_freeze_tuple indicates that at least * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also *

Re: Show WAL write and fsync stats in pg_stat_io

On Wed, Jun 28, 2023 at 6:09 AM Nazir Bilal Yavuz wrote: > This is a WIP patch to add WAL write and fsync stats to pg_stat_io > view. Thanks for working on this! I have some feedback on the content of the patch as well as some items that I feel are missing. I think it would be good to count WAL

stats test intermittent failure

rom cf41551431751a2b03e0fc08b7296301fba81992 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Mon, 10 Jul 2023 13:57:36 -0400 Subject: [PATCH v1] Fix pg_stat_io buffer reuse test instability The stats regression test attempts to ensure that Buffer Access Strategy "reuses" are being counted

Re: bgwriter doesn't flush WAL stats

On Wed, Jun 21, 2023 at 9:49 PM Kyotaro Horiguchi wrote: > Regarding the second patch, it introduces WAL IO time as a > IOCONTEXT_NORMAL/IOOBJECT_WAL, but it doesn't seem to follow the > convention or design of the pgstat_io component, which primarily > focuses on shared buffer IOs. I haven't

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

On Fri, Jun 9, 2023 at 3:28 AM Gregory Smith wrote: > On Thu, Jun 8, 2023 at 9:21 PM Andres Freund wrote: >> >> You might need to add --no-children to the perf report invocation, otherwise >> it'll show you the call graph inverted. > > > My problem was not writing kernel symbols out, I was only

Re: Wrong results from Parallel Hash Full Join

On Sun, Jun 11, 2023 at 11:24 PM Michael Paquier wrote: > > On Wed, Jun 07, 2023 at 05:16:12PM -0400, Melanie Plageman wrote: > > On Fri, May 19, 2023 at 8:05 PM Tom Lane wrote: > >> Considering that this is a parallel plan, I don't think there's any > >> mystery ab

Re: Major pgbench synthetic SELECT workload regression, Ubuntu 23.04+PG15

On Thu, Jun 8, 2023 at 3:09 PM Gregory Smith wrote: > Pushing SELECT statements at socket speeds with prepared statements is a > synthetic benchmark that normally demos big pgbench numbers. My benchmark > farm moved to Ubuntu 23.04/kernel 6.2.0-20 last month, and that test is badly > broken

Re: Wrong results from Parallel Hash Full Join

or match bit clearing for multi-batch hash joins [1]. It happens to also remove the test that failed on the buildfarm, which is why I thought to bring it up. -- Melanie [1] https://www.postgresql.org/message-id/CAAKRu_bdwDN_aHVctHcc9VoDP9av7LUMeuLbch1fHD2ESouw1g%40mail.gmail.com From d7b357c0f2e75bdfee1a58f98f

Re: Memory leak from ExecutorState context?

On Wed, May 17, 2023 at 6:35 PM Jehan-Guillaume de Rorthais wrote: > > On Wed, 17 May 2023 13:46:35 -0400 > Melanie Plageman wrote: > > > On Wed, May 17, 2023 at 07:10:08PM +0200, Jehan-Guillaume de Rorthais wrote: > > > On Tue, 16 May 2023 16:00:52 -0400 &g

Re: Memory leak from ExecutorState context?

On Wed, May 17, 2023 at 07:10:08PM +0200, Jehan-Guillaume de Rorthais wrote: > On Tue, 16 May 2023 16:00:52 -0400 > Melanie Plageman wrote: > > > From 309ad354b7a9e4dfa01b2985bd883829f5e0eba0 Mon Sep 17 00:00:00 2001 > > > From: Jehan-Guillaume de Rorthais > > >

Re: Memory leak from ExecutorState context?

On Sun, May 14, 2023 at 12:10:00AM +0200, Tomas Vondra wrote: > On 5/12/23 23:36, Melanie Plageman wrote: > > Thanks for continuing to work on this. > > > > Are you planning to modify what is displayed for memory usage in > > EXPLAIN ANALYZE? > > >

Re: Memory leak from ExecutorState context?

On Tue, May 16, 2023 at 04:00:51PM +0200, Jehan-Guillaume de Rorthais wrote: > From e5ecd466172b7bae2f1be294c1a5e70ce2b43ed8 Mon Sep 17 00:00:00 2001 > From: Melanie Plageman > Date: Thu, 30 Apr 2020 07:16:28 -0700 > Subject: [PATCH v8 1/3] Describe hash join implementation >

Re: Memory leak from ExecutorState context?

that this is what led them to running out of memory. On Wed, May 10, 2023 at 02:24:19PM +0200, Jehan-Guillaume de Rorthais wrote: > On Mon, 8 May 2023 11:56:48 -0400 Melanie Plageman > wrote: > > > > @@ -934,13 +943,16 @@ ExecHashIncreaseNumBatches(HashJoinTable hashtable) > > >

Re: pg_stat_io for the startup process

On Wed, May 03, 2023 at 04:11:33PM +0300, Melih Mutlu wrote: > Andres Freund , 27 Nis 2023 Per, 19:27 tarihinde şunu > yazdı: > > #ifdef WAL_DEBUG > > > if (XLOG_DEBUG || > > > (record->xl_rmid == RM_XACT_ID && > > trace_recovery_messages <=

Re: pg_stat_io for the startup process

gstat_io and pgBufferUsage. > > The difference comes from the case of RBM_ZERO pages. We should simply > align them. I would definitely make this a separate patch and probably a separate thread. It isn't related to the startup process and is worth a separate discussion. On Thu, Apr 27, 202

Re: Memory leak from ExecutorState context?

Thanks for continuing to work on this! On Thu, May 04, 2023 at 07:30:06PM +0200, Jehan-Guillaume de Rorthais wrote: > On Fri, 21 Apr 2023 16:44:48 -0400 Melanie Plageman > wrote: ... > > I think the biggest change that is needed is to implement this memory > > context usage

Re: pg_stat_io not tracking smgrwriteback() is confusing

v5 attached. On Thu, May 4, 2023 at 12:44 PM Andres Freund wrote: > On 2023-04-27 11:36:49 -0400, Melanie Plageman wrote: > > > > /* and finally tell the kernel to write the data to > > > > storage */ > > > > reln =

Re: pg_stat_io for the startup process

On Wed, Apr 26, 2023 at 2:53 PM Melih Mutlu wrote: > > Hi all, > > Robert Haas , 26 Nis 2023 Çar, 15:34 tarihinde şunu > yazdı: >> >> On Wed, Apr 26, 2023 at 5:47 AM Kyotaro Horiguchi >> wrote: >> > 3. When should we call pgstat_report_stats on the startup process? >> > >> > During recovery, I

Re: pg_stat_io not tracking smgrwriteback() is confusing

Thanks for the review! On Wed, Apr 26, 2023 at 10:22 PM Kyotaro Horiguchi wrote: > > At Wed, 26 Apr 2023 17:08:14 -0400, Melanie Plageman > wrote in > > On Mon, Apr 24, 2023 at 9:29 PM Melanie Plageman > > wrote: > > > I've yet to cook up a client backend te

Re: Should vacuum process config file reload more often

On Thu, Apr 27, 2023 at 8:55 AM Daniel Gustafsson wrote: > > > On 27 Apr 2023, at 14:10, Masahiko Sawada wrote: > > > > On Thu, Apr 27, 2023 at 6:30 PM John Naylor > > wrote: > >> > >> > >> On Fri, Apr 7, 2023 at 6:08 AM Daniel Gustafsson wrote: > >>> > >>> I had another read-through and

Re: pg_stat_io not tracking smgrwriteback() is confusing

On Mon, Apr 24, 2023 at 9:29 PM Melanie Plageman wrote: > I've yet to cook up a client backend test case (e.g. with COPY). I've taken > that as a todo. It was trivial to see client backend writebacks in almost any scenario once I set backend_flush_after. I wonder if it is worth ment

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Wed, Apr 26, 2023 at 8:31 AM Daniel Gustafsson wrote: > > On 26 Apr 2023, at 13:26, David Rowley wrote: > > On Wed, 26 Apr 2023, 8:48 pm Masahiko Sawada, > wrote: > > > It works but I think we might want to add the unit kB for > > understandability and

Re: pg_stat_io for the startup process

On Tue, Apr 25, 2023 at 10:51:14PM +0900, Fujii Masao wrote: > Hi, > > Regarding pg_stat_io for the startup process, I noticed that the counters > are only incremented after the startup process exits, not during WAL replay > in standby mode. This is because pgstat_flush_io() is only called when >

Re: pg_stat_io not tracking smgrwriteback() is confusing

On Mon, Apr 24, 2023 at 7:02 PM Melanie Plageman wrote: > On Mon, Apr 24, 2023 at 03:56:54PM -0700, Andres Freund wrote: > > > > I was thinking we'd track writeback separately from the write, rather > than > > attributing the writeback to "write". Otherwise it

Re: pg_stat_io not tracking smgrwriteback() is confusing

On Mon, Apr 24, 2023 at 03:56:54PM -0700, Andres Freund wrote: > Hi, > > On 2023-04-24 18:36:24 -0400, Melanie Plageman wrote: > > On Mon, Apr 24, 2023 at 6:13 PM Andres Freund wrote: > > > > Also, it seems like this (given the current code) is only reachable for >

Re: pg_stat_io not tracking smgrwriteback() is confusing

On Mon, Apr 24, 2023 at 6:13 PM Andres Freund wrote: > Hi, > > On 2023-04-24 17:37:48 -0400, Melanie Plageman wrote: > > On Mon, Apr 24, 2023 at 02:14:32PM -0700, Andres Freund wrote: > > > It starts blocking once "enough" IO is in flight. For things li

Re: pg_stat_io not tracking smgrwriteback() is confusing

On Mon, Apr 24, 2023 at 02:14:32PM -0700, Andres Freund wrote: > Hi, > > On 2023-04-24 16:39:36 -0400, Melanie Plageman wrote: > > On Wed, Apr 19, 2023 at 10:23:26AM -0700, Andres Freund wrote: > > > Hi, > > > > > > I noticed that the numbers i

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

On Mon, Apr 10, 2023 at 3:41 AM Pavel Luzanov wrote: > > On 05.04.2023 03:41, Melanie Plageman wrote: > > On Tue, Apr 4, 2023 at 4:35 PM Pavel Luzanov > > wrote: > > > >> After a little thought... I'm not sure about the term 'bootstrap > >> process'. I

Re: pg_stat_io not tracking smgrwriteback() is confusing

On Wed, Apr 19, 2023 at 10:23:26AM -0700, Andres Freund wrote: > Hi, > > I noticed that the numbers in pg_stat_io dont't quite add up to what I > expected in write heavy workloads. Particularly for checkpointer, the numbers > for "write" in log_checkpoints output are larger than what is visible

Re: could not extend file "base/5/3501" with FileFallocate(): Interrupted system call

On Mon, Apr 24, 2023 at 10:53:35AM +0200, Christoph Berg wrote: > Re: Andres Freund > > Add smgrzeroextend(), FileZero(), FileFallocate() > > Hi, > > I'm often seeing PG16 builds erroring out in the pgbench tests: > > 00:33:12 make[2]: Entering directory '/<>/build/src/bin/pgbench' > 00:33:12

Re: Move un-parenthesized syntax docs to "compatibility" for few SQL commands

On Fri, Apr 21, 2023 at 09:44:51PM -0700, Nathan Bossart wrote: > I've attached two patches. 0001 adds a parenthesized CLUSTER syntax that > doesn't require a table name. 0002 is your patch with a couple of small > adjustments. > > On Fri, Apr 21, 2023 at 07:29:59PM -0400, M

Re: Move un-parenthesized syntax docs to "compatibility" for few SQL commands

On Fri, Apr 21, 2023 at 6:55 PM Nathan Bossart wrote: > > On Fri, Apr 21, 2023 at 06:29:16PM -0400, Melanie Plageman wrote: > > Over in [2], it was suggested that moving the un-parenthesized syntax to > > the "Compatibility" section of their respective docs pages would

Move un-parenthesized syntax docs to "compatibility" for few SQL commands

pa.us From adef275c702833c6a4441d347ab8303403576fb2 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Fri, 21 Apr 2023 18:11:06 -0400 Subject: [PATCH v1] Doc: Move un-parenthesized syntax to compatibility for few SQL commands Move documentation of the un-parenthesized syntax for VACUUM, ANALYZE, and EXPLAIN to the &quo

Re: Memory leak from ExecutorState context?

On Fri, Apr 7, 2023 at 8:01 PM Jehan-Guillaume de Rorthais wrote: > > On Fri, 31 Mar 2023 14:06:11 +0200 > Jehan-Guillaume de Rorthais wrote: > > > > [...] > > > >> Hmmm, not sure is WARNING is a good approach, but I don't have a better > > > >> idea at the moment. > > > > > > > > I stepped it

Re: Commitfest 2023-03 starting tomorrow!

On Fri, Apr 21, 2023 at 9:50 AM Tom Lane wrote: > > Jehan-Guillaume de Rorthais writes: > > After catching up with this thread, where pending bugs are listed and > > discussed, > > I wonder if the current patches trying to lower the HashJoin memory > > explosion[1] > > could be added to the

Re: Memory leak from ExecutorState context?

On Thu, Apr 20, 2023 at 12:42 PM Konstantin Knizhnik wrote: > On 11.04.2023 8:14 PM, Jehan-Guillaume de Rorthais wrote: > > On Sat, 8 Apr 2023 02:01:19 +0200 > > Jehan-Guillaume de Rorthais wrote: > > > >> On Fri, 31 Mar 2023 14:06:11 +0200 > >> Jehan-Guillaume de Rorthais wrote: > >> > >>

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 19, 2023 at 8:43 PM Melanie Plageman wrote: > On Wed, Apr 19, 2023 at 3:20 PM Andres Freund wrote: >> On 2023-04-19 12:16:24 -0500, Justin Pryzby wrote: >> > On Wed, Apr 19, 2023 at 11:17:04AM -0400, Melanie Plageman wrote: >> > > Ultimately this

Re: Should we put command options in alphabetical order in the doc?

On Wed, Apr 19, 2023 at 05:33:47PM -0400, Melanie Plageman wrote: > On Wed, Apr 19, 2023 at 2:39 PM Peter Geoghegan wrote: > > > > On Wed, Apr 19, 2023 at 3:04 AM Alvaro Herrera > > wrote: > > > > While I'm certain that nobody will agree with me on every little

Re: Remove io prefix from pg_stat_io columns

On Thu, Apr 20, 2023 at 10:13:04AM +0900, Michael Paquier wrote: > On Wed, Apr 19, 2023 at 08:50:13PM -0400, Melanie Plageman wrote: > > I thought about changing parameter and local variable names to remove > > the prefix, but in the original discussion folks seemed to think it

Re: Remove io prefix from pg_stat_io columns

On Wed, Apr 19, 2023 at 8:42 PM Michael Paquier wrote: > > On Wed, Apr 19, 2023 at 01:54:21PM -0300, Fabrízio de Royes Mello wrote: > > On Wed, Apr 19, 2023 at 1:27 PM Melanie Plageman > > wrote: > >> Over in [1], we discussed removing the "io" pre

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 19, 2023 at 8:41 PM Justin Pryzby wrote: > > On Wed, Apr 19, 2023 at 12:20:51PM -0700, Andres Freund wrote: > > On 2023-04-19 12:16:24 -0500, Justin Pryzby wrote: > > > On Wed, Apr 19, 2023 at 11:17:04AM -0400, Melanie Plageman wrote: > > > > Ultima

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 19, 2023 at 3:20 PM Andres Freund wrote: > Hi, > > On 2023-04-19 12:16:24 -0500, Justin Pryzby wrote: > > On Wed, Apr 19, 2023 at 11:17:04AM -0400, Melanie Plageman wrote: > > > Ultimately this is probably fine. If we wanted to modify one of the >

Re: Should we put command options in alphabetical order in the doc?

On Wed, Apr 19, 2023 at 2:39 PM Peter Geoghegan wrote: > > On Wed, Apr 19, 2023 at 3:04 AM Alvaro Herrera > wrote: > > > While I'm certain that nobody will agree with me on every little > > > detail, I have to imagine that most would find my preferred ordering > > > quite understandable and

Remove io prefix from pg_stat_io columns

7543675435826022.horikyota.ntt%40gmail.com From 632102f3a9ad5f0246d544c0e29c85915ccde495 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Wed, 19 Apr 2023 11:43:13 -0400 Subject: [PATCH v1] Remove io prefix from pg_stat_io columns a9c70b46 added the statistics view pg_stat_io which contained colum

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 12, 2023 at 08:31:26PM -0400, Melanie Plageman wrote: > On Wed, Apr 12, 2023 at 6:50 PM Thomas Munro wrote: > > And if we're going to > > exercise/test that case, should we do the non-parallel version too? > > I've added this. I thought if we were adding the

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

On Sat, Apr 15, 2023 at 12:59:52PM +1200, David Rowley wrote: > On Fri, 14 Apr 2023 at 19:20, Peter Eisentraut > wrote: > > > > I came across these new options and had a little bit of trouble figuring > > them out from the documentation. Maybe this could be polished a bit. > > > > vacuumdb

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 12, 2023 at 6:50 PM Thomas Munro wrote: > > On Thu, Apr 13, 2023 at 9:48 AM Melanie Plageman > wrote: > > Attached patch includes the fix for ExecParallelHashTableInsert() as > > well as a test. I toyed with adapting one of the existing parallel full > &g

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 12, 2023 at 2:59 PM Melanie Plageman wrote: > > On Wed, Apr 12, 2023 at 2:14 PM Andres Freund wrote: > > > > Hi, > > > > On 2023-04-12 10:57:17 -0400, Melanie Plageman wrote: > > > HeapTupleHeaderHasMatch() checks if HEAP_TUPLE_HAS_MATCH is

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 12, 2023 at 2:14 PM Andres Freund wrote: > > Hi, > > On 2023-04-12 10:57:17 -0400, Melanie Plageman wrote: > > HeapTupleHeaderHasMatch() checks if HEAP_TUPLE_HAS_MATCH is set. > > > > In htup_details.h, you will see that HEAP_TUPLE_HAS_MATCH is

Re: Wrong results from Parallel Hash Full Join

On Wed, Apr 12, 2023 at 7:36 AM Richard Guo wrote: > > I came across $subject and reduced the repro query as below. > > create table a (i int); > create table b (i int); > insert into a values (1); > insert into b values (2); > update b set i = 2; > > set min_parallel_table_scan_size to 0; > set

Re: Show various offset arrays for heap WAL records

On Tue, Apr 11, 2023 at 1:35 PM Peter Geoghegan wrote: > > On Tue, Apr 11, 2023 at 7:40 AM Melanie Plageman > wrote: > > Not the fault of this patch, but I also noticed that heap UPDATE and > > HOT_UPDATE records have xmax twice and don't differentiate between new

Re: ERROR messages in VACUUM's PARALLEL option

On Tue, Apr 11, 2023 at 4:00 AM David Rowley wrote: > > Over in [1], Horiguchisan mentioned a few things about VACUUM's new > BUFFER_USAGE_LIMIT option. > > 1) buffer_usage_limit in the ERROR messages should be consistently in > uppercase. I did notice that all the other VACUUM options don't do

Re: Show various offset arrays for heap WAL records

Hi, static void infobits_desc(StringInfo buf, uint8 infobits, const char *keyname) { appendStringInfo(buf, "%s: [", keyname); Why can we assume that there will be no space at the end here? I know we need to be able to avoid doing the comma overwriting if no flags were set. In general, we

Re: Show various offset arrays for heap WAL records

On Mon, Apr 10, 2023 at 04:31:44PM -0700, Peter Geoghegan wrote: > On Mon, Apr 10, 2023 at 3:04 PM Melanie Plageman > wrote: > > > > I will say that the prefix of p in "ptid" makes it sound like pointer to > > a tid, which I don't believe is what you meant.

Re: Should vacuum process config file reload more often

On Fri, Apr 7, 2023 at 9:07 AM Melanie Plageman wrote: > > On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada wrote: > > > > On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > > > > > On 7 Apr 2023, at 00:12, Melanie Plageman > > > >

Re: Show various offset arrays for heap WAL records

On Sun, Apr 9, 2023 at 8:12 PM Peter Geoghegan wrote: > > On Fri, Apr 7, 2023 at 4:46 PM Peter Geoghegan wrote: > > Pushed that one too. > > I noticed that the nbtree VACUUM and DELETE record types have their > update/xl_btree_update arrays output incorrectly. We cannot use the > generic

Re: Parallel Full Hash Join

On Sat, Apr 8, 2023 at 1:30 PM Melanie Plageman wrote: > > On Sat, Apr 8, 2023 at 12:33 PM Tom Lane wrote: > > > > Thomas Munro writes: > > > I committed the main patch. > > > > BTW, it was easy to miss in all the buildfarm noise from > > last-poss

Re: Parallel Full Hash Join

On Sat, Apr 8, 2023 at 12:33 PM Tom Lane wrote: > > Thomas Munro writes: > > I committed the main patch. > > BTW, it was easy to miss in all the buildfarm noise from > last-possible-minute patches, but chimaera just showed something > that looks like a bug in this code [1]: > > 2023-04-08

Re: Show various offset arrays for heap WAL records

On Fri, Apr 7, 2023 at 7:09 PM Peter Geoghegan wrote: > > On Fri, Apr 7, 2023 at 4:01 PM Melanie Plageman > wrote: > > LGTM > > Pushed, thanks. It's come to my attention that I forgot to include the btree patch earlier. PFA From 4f502b2513ba79d738e7ed87aaf7d18ed2a2e30f Mon

Re: Show various offset arrays for heap WAL records

On Fri, Apr 7, 2023 at 5:43 PM Peter Geoghegan wrote: > > On Fri, Apr 7, 2023 at 1:33 PM Melanie Plageman > wrote: > > Attached v3 is cleaned up and includes a pg_walinspect docs update as > > well as some edited comments in rmgr_utils.c > > Attached v4 has some smal

Re: Minimal logical decoding on standbys

Code review only of 0001-0005. I noticed you had two 0008, btw. On Fri, Apr 07, 2023 at 11:12:26AM -0700, Andres Freund wrote: > Hi, > > On 2023-04-07 08:47:57 -0700, Andres Freund wrote: > > Integrated all of these. > > From 0e038eb5dfddec500fbf4625775d1fa508a208f6 Mon Sep 17 00:00:00 2001 >

Re: Show various offset arrays for heap WAL records

Attached v3 is cleaned up and includes a pg_walinspect docs update as well as some edited comments in rmgr_utils.c On Mon, Mar 27, 2023 at 6:27 PM Peter Geoghegan wrote: > > On Mon, Mar 27, 2023 at 2:29 PM Melanie Plageman > wrote: > > I went to add dedup records and not

Re: Track IO times in pg_stat_io

ng having the buffer pinned and not having those flags set, though. On Tue, Apr 4, 2023 at 8:59 PM Andres Freund wrote: > > Hi, > > On 2023-03-31 15:44:58 -0400, Melanie Plageman wrote: > > From 789d4bf1fb749a26523dbcd2c69795916b711c68 Mon Sep 17 00:00:00 2001 > > From: Mel

Re: Should vacuum process config file reload more often

On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada wrote: > > On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > > > On 7 Apr 2023, at 00:12, Melanie Plageman > > > wrote: > > > > > > On Thu, Apr 6, 2023 at 5:45 PM Daniel Gustafsso

<    1   2   3   4   5   6   7   >