On 31.07.2025 16:31, Melanie Plageman wrote:
If you eager scan and fail to freeze 3% of each
region, you'll have eager scanned and failed to freeze 3% of the
blocks in the table -- so the total cap is the same.
OhmyGod!Whata stupidmistakeI made. Imultipliedthe numberof regionsby3%:-)
Documentin
On Thu, Jul 31, 2025 at 7:05 AM Pavel Luzanov wrote:
>
> I have a question about the documentation.
>
> From description of the vacuum_max_eager_freeze_failure_rate [1]
>
> "Specifies the maximum number of pages (as a fraction of total pages in
> the relation) that VACUUM may scan and fail to set
Hello,
I have a question about the documentation.
From description of the vacuum_max_eager_freeze_failure_rate [1]
"Specifies the maximum number of pages (as a fraction of total pages in
the relation) that VACUUM may scan and fail to set all-frozen in the
visibility map before disabling eager
On Tue, Feb 11, 2025 at 11:31 AM Andres Freund wrote:
>
> On 2025-02-05 12:23:29 -0500, Melanie Plageman wrote:
> > Attached v16 implements the logic to not count pages we failed to
> > freeze because of cleanup lock contention as eager freeze failures.
>
> That looks good to me.
Cool. Committed
On 2025-02-05 12:23:29 -0500, Melanie Plageman wrote:
> Attached v16 implements the logic to not count pages we failed to
> freeze because of cleanup lock contention as eager freeze failures.
That looks good to me.
Hi,
On 2025-02-10 14:30:15 -0500, Robert Haas wrote:
> On Wed, Feb 5, 2025 at 12:23 PM Melanie Plageman
> wrote:
> > I started getting worried thinking about this. If you have a cursor
> > for select * from a table and fetch forward far enough, couldn't
> > vacuum fail to get cleanup locks on a w
n.
Anyway, I've updated the comment to be more correct I think.
- Melanie
From f713f7b3ca33e3cf5825706b86740c5f8f1e5ad6 Mon Sep 17 00:00:00 2001
From: Melanie Plageman
Date: Mon, 27 Jan 2025 12:23:00 -0500
Subject: [PATCH v17] Eagerly scan all-visible pages to amortize aggressive
vacuum
Aggressiv
Hi,
On 2025-02-04 12:44:22 -0500, Melanie Plageman wrote:
> On Mon, Feb 3, 2025 at 9:09 PM Andres Freund wrote:
> > > + /*
> > > + * Now calculate the eager scan start block. Start at a random spot
> > > + * somewhere within the first eager scan region. This avoids eager
> > > +
On Wed, Feb 5, 2025 at 12:23 PM Melanie Plageman
wrote:
> I started getting worried thinking about this. If you have a cursor
> for select * from a table and fetch forward far enough, couldn't
> vacuum fail to get cleanup locks on a whole range of blocks?
I don't think so. A given scan holds at m
27;m correct about that, but if I had to
> guess, my guess would be "rare scenario".
I've not done this in the attached v16. I have added a comment about it.
I think not doing it is a judgment call and not a bug, right?
- Melanie
From 8bfe510f260066de91f77d704b4eb8c01d67de40 M
On Tue, Feb 4, 2025 at 5:34 PM Melanie Plageman
wrote:
> I think I misspoke when I said we are unlikely to have contended
> all-visible pages. I suppose it is trivial to concoct a scenario where
> there are many pinned all-visible pages.
It's hard to keep heap pages pinned for a really long time,
On Tue, Feb 4, 2025 at 3:55 PM Robert Haas wrote:
>
> On Tue, Feb 4, 2025 at 2:57 PM Robert Treat wrote:
> > > Yea, I thought that counting them as failures made sense because we
> > > did fail to freeze them. However, now that you mention it, we didn't
> > > fail to freeze them because of age, s
On Tue, Feb 4, 2025 at 2:57 PM Robert Treat wrote:
> > Yea, I thought that counting them as failures made sense because we
> > did fail to freeze them. However, now that you mention it, we didn't
> > fail to freeze them because of age, so maybe we don't want to count
> > them as failures. I don't
On Tue, Feb 4, 2025 at 12:44 PM Melanie Plageman
wrote:
> On Mon, Feb 3, 2025 at 9:09 PM Andres Freund wrote:
> >
> > On 2025-01-29 14:12:52 -0500, Melanie Plageman wrote:
> > > From 71f32189aad510b73d221fb0478ffd916e5e5dde Mon Sep 17 00:00:00 2001
> > > From: Melanie Plageman
>
> > > @@ -1064,7
:23:00 -0500
> > Subject: [PATCH v14] Eagerly scan all-visible pages to amortize aggressive
> > vacuum
> >
> > Amortize the cost of an aggressive vacuum by eagerly scanning some
> > all-visible but not all-frozen pages during normal vacuums.
>
> I think it'd be g
On Mon, Feb 3, 2025 at 9:09 PM Andres Freund wrote:
> > + > xreflabel="vacuum_max_eager_freeze_failure_rate">
> > + vacuum_max_eager_freeze_failure_rate
> > (floating point)
> > +
> > + vacuum_max_eager_freeze_failure_rate
> > configuration parameter
> > +
> > +
Hi,
On 2025-01-29 14:12:52 -0500, Melanie Plageman wrote:
> From 71f32189aad510b73d221fb0478ffd916e5e5dde Mon Sep 17 00:00:00 2001
> From: Melanie Plageman
> Date: Mon, 27 Jan 2025 12:23:00 -0500
> Subject: [PATCH v14] Eagerly scan all-visible pages to amortize aggressive
> vacuu
On Wed, Jan 29, 2025 at 12:59 PM Masahiko Sawada wrote:
>
> I missed to mention; it's about the tab completion for
> vacuum_max_easter_freeze_failure storage parameter. We still need to
> update the tab-complete.in.c file (I mentioned the wrong file
> previously) for that.
Ah, thanks! I didn't re
On Wed, Jan 29, 2025 at 6:08 AM Melanie Plageman
wrote:
>
> On Tue, Jan 28, 2025 at 4:38 PM Masahiko Sawada wrote:
> >
> >
> > Thank you for updating the patch! These updates look good to me.
>
> Thanks for taking another look!
>
> > BTW I realized that we need to update tab-complete.c too to sup
On Fri, Jan 24, 2025 at 11:20 AM Robert Haas wrote:
>
> On Fri, Jan 24, 2025 at 9:15 AM Melanie Plageman
> wrote:
> > So, in this case, there is only one table in question, so 1 autovacuum
> > worker (and up to 2 maintenance parallel workers for index vacuuming).
> > The duration I provided is ju
version 13 attached.
On Tue, Jan 28, 2025 at 1:49 PM Robert Treat wrote:
>
> On Mon, Jan 27, 2025 at 12:45 PM Melanie Plageman
> wrote:
> >
> > On Fri, Jan 24, 2025 at 3:43 PM Robert Haas wrote:
> > >
> > > On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
> > > wrote:
> > attached v11 uses a f
On Tue, Jan 28, 2025 at 4:38 PM Masahiko Sawada wrote:
>
>
> Thank you for updating the patch! These updates look good to me.
Thanks for taking another look!
> BTW I realized that we need to update tab-complete.c too to support
> the tab-completion for vacuum_max_eager_freeze_failure_rate.
I'm
On Mon, Jan 27, 2025 at 1:22 PM Melanie Plageman
wrote:
>
> On Mon, Jan 27, 2025 at 12:52 PM Masahiko Sawada
> wrote:
> >
> > Thank you for updating the patch. I was reviewing the v10 patch and
> > had some comments. I believe these comments are still valid for v11,
> > but please ignore them if
On Mon, Jan 27, 2025 at 12:45 PM Melanie Plageman
wrote:
>
> On Fri, Jan 24, 2025 at 3:43 PM Robert Haas wrote:
> >
> > On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
> > wrote:
> attached v11 uses a fraction with this name. It follows the
> conventions and I find it descriptive.
>
> Changing
On Mon, Jan 27, 2025 at 12:52 PM Masahiko Sawada wrote:
>
> Thank you for updating the patch. I was reviewing the v10 patch and
> had some comments. I believe these comments are still valid for v11,
> but please ignore them if outdated.
Thanks so much for the review!
> + if (TransactionIdI
On Mon, Jan 27, 2025 at 9:45 AM Melanie Plageman
wrote:
>
> On Fri, Jan 24, 2025 at 3:43 PM Robert Haas wrote:
> >
> > On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
> > wrote:
> > > I think you're right. I would go with a percentage. I don't see many
> > > other GUCs that are percents. What w
On Fri, Jan 24, 2025 at 3:43 PM Robert Haas wrote:
>
> On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
> wrote:
> > I think you're right. I would go with a percentage. I don't see many
> > other GUCs that are percents. What would you call it? Perhaps
> > vacuum_eager_scan_fail_threshold? The % o
On Sun, Jan 26, 2025 at 3:11 PM Robert Treat wrote:
> Hmm... isn't "don't try" really more logically equivalent to "stop
> after zero failures", with "try until zero failures" more of the
> inverse... more like "never stop never stopping"?
No. Or at least, I don't think that's how English works.
On Fri, Jan 24, 2025 at 3:43 PM Robert Haas wrote:
> On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
> wrote:
> > This thought exercise made me realize something is wrong with my
> > current patch, though. If you set the failure tolerance
> > (vacuum_eager_scan_max_fails) to 0 right now, it disa
On Fri, Jan 24, 2025 at 3:43 PM Robert Haas wrote:
> On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
> wrote:
> >
> > > + ereport(INFO,
> > > + (errmsg("Vacuum successfully froze %u eager scanned blocks of
> > > \"%s.%s.%s\". Now disabling eager scanning.",
> > >
> > > I predict that if Tom sees
On Fri, Jan 24, 2025 at 3:02 PM Melanie Plageman
wrote:
> This thought exercise made me realize something is wrong with my
> current patch, though. If you set the failure tolerance
> (vacuum_eager_scan_max_fails) to 0 right now, it disables eager
> scanning altogether. That might be unexpected. Yo
Thanks for the review! Attached v10 addresses the review feedback
except that about the GUC/table option meaning and format.
On Thu, Jan 23, 2025 at 2:22 PM Robert Haas wrote:
>
> On Thu, Jan 23, 2025 at 1:31 PM Robert Haas wrote:
> >
> > On Wed, Jan 22, 2025 at 5:48 PM Melanie Plageman
> > wro
On Fri, Jan 24, 2025 at 9:15 AM Melanie Plageman
wrote:
> So, in this case, there is only one table in question, so 1 autovacuum
> worker (and up to 2 maintenance parallel workers for index vacuuming).
> The duration I provided is just the absolute duration from start of
> vacuum to finish -- not
On Thu, Jan 23, 2025 at 12:16 PM Robert Haas wrote:
>
> On Wed, Jan 22, 2025 at 5:48 PM Melanie Plageman
> wrote:
> > Circling back to benchmarking, I've been running the most adversarial
> > benchmarks I could devise and can share a bit of what I've found.
> >
> > I created a "hot tail" benchmar
On Wed, Jan 22, 2025 at 5:48 PM Melanie Plageman
wrote:
> Circling back to benchmarking, I've been running the most adversarial
> benchmarks I could devise and can share a bit of what I've found.
>
> I created a "hot tail" benchmark where 16 clients insert some data and
> then update some data ol
On Thu, Jan 23, 2025 at 1:31 PM Robert Haas wrote:
>
> On Wed, Jan 22, 2025 at 5:48 PM Melanie Plageman
> wrote:
> > Thanks! Attached v9 incorporates all your suggested changes.
>
> I'm not exactly sure what to do about it, but I feel like the
> documentation of vacuum_eager_scan_max_fails is goi
On Wed, Jan 22, 2025 at 5:48 PM Melanie Plageman
wrote:
> Thanks! Attached v9 incorporates all your suggested changes.
I'm not exactly sure what to do about it, but I feel like the
documentation of vacuum_eager_scan_max_fails is going to be
incomprehensible to someone who doesn't already have a d
have
to assume that the extra time spent vacuuming comes with no benefit to
offset the cost in the worst case.
The question is, is this extra time spent vacuuming in the worst case
acceptable?
- Melanie
From 3171aa3ac3380506c24c37c95ab21d8f3d10508d Mon Sep 17 00:00:00 2001
From: Melanie Plage
On Wed, Jan 15, 2025 at 3:56 PM Melanie Plageman
wrote:
> On Wed, Jan 15, 2025 at 3:03 PM Melanie Plageman
> wrote:
> >
> > On Wed, Jan 15, 2025 at 2:36 PM Marcos Pegoraro wrote:
> > >>
> > > There is a typo on committed vacuumlazy.c file
> > > * If the TID store fills up in phase I, vacuum sus
On 15.01.2025 23:09, Melanie Plageman wrote:
On Wed, Jan 15, 2025 at 12:08 PM Alena Rybakina
wrote:
This is interesting, but I think it might belong as commentary in
vacuumparallel.c instead.
I added some description about it, I hope it is fine. I attached
vacuum_description.diff
Alena, tha
On Wed, Jan 15, 2025 at 3:03 PM Melanie Plageman
wrote:
>
> On Wed, Jan 15, 2025 at 2:36 PM Marcos Pegoraro wrote:
> >>
> > There is a typo on committed vacuumlazy.c file
> > * If the TID store fills up in phase I, vacuum suspends phase I, proceeds
> > to
> > * phases II and II, cleaning up th
On Wed, Jan 15, 2025 at 12:08 PM Alena Rybakina
wrote:
>
> This is interesting, but I think it might belong as commentary in
> vacuumparallel.c instead.
>
> I added some description about it, I hope it is fine. I attached
> vacuum_description.diff
Alena, thanks again for your review. I pushed th
On Wed, Jan 15, 2025 at 2:36 PM Marcos Pegoraro wrote:
>>
>> On 14.01.2025 22:51, Melanie Plageman wrote:
>
>
> There is a typo on committed vacuumlazy.c file
> * If the TID store fills up in phase I, vacuum suspends phase I, proceeds to
> * phases II and II, cleaning up the dead tuples referenc
>
> On 14.01.2025 22:51, Melanie Plageman wrote:
>
There is a typo on committed vacuumlazy.c file
* If the TID store fills up in phase I, vacuum suspends phase I, proceeds
to
* phases II and II, cleaning up the dead tuples referenced in the current
TID
* store. This empties the TID store resume
On 14.01.2025 22:51, Melanie Plageman wrote:
On Mon, Jan 13, 2025 at 5:37 PM Alena Rybakina
wrote:
Thank you for working on this patch, without this explanation it is difficult
to understand what is happening, to put it mildly.
Thanks for the review! I've incorporated most of them into attac
hink it might belong as commentary in
vacuumparallel.c instead.
Thanks again for your close reading and detailed thoughts!
- Melanie
From d579e6fa22982f66b7e23efc80e7e528d2059d51 Mon Sep 17 00:00:00 2001
From: Melanie Plageman
Date: Tue, 7 Jan 2025 09:48:34 -0500
Subject: [PATCH v7 2/2] Eagerly
Hi!
On 14.01.2025 00:46, Melanie Plageman wrote:
On Thu, Jan 9, 2025 at 1:24 PM Andres Freund wrote:
On 2025-01-07 15:46:26 -0500, Melanie Plageman wrote:
For table storage options, those related to vacuum but not autovacuum
are in the main StdRdOptions struct. Of those, some are overridden b
hot tail workload with
a job that deletes old data.
- Melanie
From a5080bb6c630af932451d56a0931c9bc96eb8417 Mon Sep 17 00:00:00 2001
From: Melanie Plageman
Date: Tue, 7 Jan 2025 09:48:34 -0500
Subject: [PATCH v6 2/2] Eagerly scan all-visible pages to amortize aggressive
vacuum
Introduce eager
Hi,
On 2025-01-07 15:46:26 -0500, Melanie Plageman wrote:
> For table storage options, those related to vacuum but not autovacuum
> are in the main StdRdOptions struct. Of those, some are overridden by
> VACUUM command parameters which are parsed out into the VacuumParams
> struct. Though the memb
lageman
Date: Tue, 7 Jan 2025 09:48:34 -0500
Subject: [PATCH v5 3/3] Eagerly scan all-visible pages to amortize aggressive
vacuum
Introduce eager scanning normal vacuums, in which vacuum scans some of
the all-visible but not all-frozen pages in the relation to amortize the
cost of an aggressive va
On Sat, Dec 21, 2024 at 10:28 AM Robert Treat wrote:
>
> On Tue, Dec 17, 2024 at 5:51 PM Melanie Plageman
> wrote:
> >
> It feels to me like eager vacuums are not so much a distinct thing,
> but that, like how all vacuum do index cleanup unless told not to, all
> vacuums are optimistic that they
e in the visibility map. We may choose not
+ * to skip pages if the range of skippable pages is below
+ * SKIP_PAGES_THRESHOLD.
+ *
+ * Once vacuum has decided to scan a given block, it must read in the block
+ * and obtain a cleanup lock to prune tuples on the page. A non-aggressive
+ * vacuum ma
On Tue, Dec 17, 2024 at 5:51 PM Melanie Plageman
wrote:
>
> Thanks for taking a look!
>
> I've rebased and attached an updated v3 which also addresses review feedback.
>
> On Sun, Dec 15, 2024 at 1:05 AM Robert Treat wrote:
> > On Fri, Dec 13, 2024 at 5:53 PM Melanie Plageman
> > wrote:
>
> So,
On Tue, Dec 17, 2024 at 5:54 PM Melanie Plageman
wrote:
> Makes sense. I've attempted to clarify as you suggest in v3.
I would just commit 0001. There's nothing to be gained by waiting around.
I don't care about 0002 much. It doesn't seem particularly better or
worse. I would suggest that if you
Thanks for the review!
On Tue, Dec 17, 2024 at 10:57 AM Nazir Bilal Yavuz wrote:
>
> Here are couple of code comments:
>
> === [PATCH v2 07/10] ===
>
> It took me a while to understand that heap_vac_scan_next_block() loops
> until rel_pages. What do you think about adding
> Assert(vacrel->current
Hi,
Thank you for working on this!
On Sat, 14 Dec 2024 at 01:53, Melanie Plageman
wrote:
>
> On Thu, Nov 7, 2024 at 10:42 AM Andres Freund wrote:
> >
> > Hi,
>
> Thanks for the review!
> Attached v2 should address your feedback and also fixes a few bugs with v1.
Here are couple of code comment
16
> >
> > Hm. It's not clear to me why other_rw is higher with the patch? After all,
> > given the workload, there's no chance of unnecessarily freezing tuples? Is
> > that just because at the end of the benchmark there's leftover work?
>
> So oth
uumed in three main phases. In the first phase,
> > + * vacuum scans relation pages, pruning and freezing tuples and saving dead
> > + * tuples' TIDs in a TID store. If that TID store fills up or vacuum
> > finishes
> > + * scanning the relation, it progresses to
in three main phases. In the first phase,
> + * vacuum scans relation pages, pruning and freezing tuples and saving dead
> + * tuples' TIDs in a TID store. If that TID store fills up or vacuum finishes
> + * scanning the relation, it progresses to the second phase: index vacuuming.
>
59 matches
Mail list logo
