Re: Checkpointer write combining

2025-11-19 Thread Chao Li
> On Nov 20, 2025, at 06:12, Melanie Plageman wrote: > > Are you sure you applied 0005? It is the one that adds dbId to CkptSortItem. > My bad. Yes, I missed to download and apply 0005. > - Melanie > I went through the whole patch set again, and got a few more comments: 1 - 0002 ``` +/*

Re: Checkpointer write combining

2025-11-19 Thread Melanie Plageman
Thanks for the review! On Tue, Nov 18, 2025 at 9:10 PM Chao Li wrote: > > > On Nov 19, 2025, at 10:00, Chao Li wrote: > > > I just reviewed 0007. It removes the second parameter "bool > > skip_recently_used” from SyncOneBuffer. The function is static and is only > > called in one place with s

Re: Checkpointer write combining

2025-11-19 Thread BharatDB
Hi Melanie, Thank you for the detailed clarifications which helped me understand the constraints much more clearly. You are absolutely right regarding the key points you specified. My initial concern came from trying to avoid cases where strategy pin limits were unexpectedly small (0 or negative)

Re: Checkpointer write combining

2025-11-18 Thread Chao Li
> On Nov 19, 2025, at 10:00, Chao Li wrote: > > > >> On Nov 19, 2025, at 02:49, Melanie Plageman >> wrote: >> >> I no longer remember why I made that patch WIP, so I've removed that >> designation. > > I just reviewed 0007. It removes the second parameter "bool > skip_recently_used” fr

Re: Checkpointer write combining

2025-11-18 Thread Chao Li
> On Nov 19, 2025, at 02:49, Melanie Plageman wrote: > > I no longer remember why I made that patch WIP, so I've removed that > designation. I just reviewed 0007. It removes the second parameter "bool skip_recently_used” from SyncOneBuffer. The function is static and is only called in one p

Re: Checkpointer write combining

2025-11-18 Thread Melanie Plageman
On Thu, Nov 13, 2025 at 3:30 AM Chao Li wrote: > > > On Nov 4, 2025, at 07:34, Melanie Plageman > > wrote: > > > > > > uint32 > > StrategyMaxWriteBatchSize(BufferAccessStrategy strategy) > > { > >uint32max_write_batch_size = Min(io_combine_limit, > > MAX_IO_COMBINE_LIMIT); > >int

Re: Checkpointer write combining

2025-11-18 Thread Melanie Plageman
On Wed, Nov 12, 2025 at 6:40 AM BharatDB wrote: > > Considering the CI failures in earlier patch versions around “max batch > size”, upon my observation I found the failures arise either from boundary > conditions when io_combine_limit (GUC) is set larger than the compile-time > MAX_IO_COMBINE_

Re: Checkpointer write combining

2025-11-13 Thread Chao Li
> On Nov 4, 2025, at 07:34, Melanie Plageman wrote: > > Thanks for continuing to review! I've revised the patches to > incorporate all of your feedback except for where I mention below. > > There were failures in CI due to issues with max batch size, so > attached v8 also seeks to fix those.

Re: Checkpointer write combining

2025-11-12 Thread BharatDB
Hi all, Considering the CI failures in earlier patch versions around “max batch size”, upon my observation I found the failures arise either from boundary conditions when io_combine_limit (GUC) is set larger than the compile-time MAX_IO_COMBINE_LIMIT or when pin limits return small/zero values due

Re: Checkpointer write combining

2025-11-03 Thread Melanie Plageman
Thanks for continuing to review! I've revised the patches to incorporate all of your feedback except for where I mention below. There were failures in CI due to issues with max batch size, so attached v8 also seeks to fix those. - Melanie On Thu, Oct 16, 2025 at 12:25 AM Chao Li wrote: > > 3 -

Re: Checkpointer write combining

2025-10-18 Thread Chao Li
Hi Milanie, Thanks for updating the patch set. I review 1-6 and got a few more small comments. I didn’t review 0007 as it’s marked as WIP. > > - Melanie > 1 - 0001 ``` --- a/src/include/storage/buf_internals.h +++ b/src/include/storage/buf_internals.h @@ -421,6 +421,12 @@ ResourceOwnerForgetB

Re: Checkpointer write combining

2025-10-17 Thread Melanie Plageman
On Thu, Sep 11, 2025 at 11:33 PM Chao Li wrote: > > I don’t understand why the two versions are different: > > if (XLogNeedsFlush(lsn)) > { > /* > * Remove the dirty buffer from the ring; necessary to prevent an > * infinite loop if all ring members are dirty. > */ > strategy->buffers[strategy->cu

Re: Checkpointer write combining

2025-09-20 Thread Chao Li
> On Sep 10, 2025, at 01:55, Melanie Plageman wrote: > > > [1] > https://www.postgresql.org/message-id/flat/CAAKRu_Yjn4mvN9NBxtmsCQSGwup45CoA4e05nhR7ADP-v0WCig%40mail.gmail.com > 1 - 0001 ``` --- a/src/backend/storage/buffer/freelist.c +++ b/src/backend/storage/buffer/freelist.c + * The b

Re: Checkpointer write combining

2025-09-19 Thread Melanie Plageman
On Tue, Sep 9, 2025 at 9:27 AM Nazir Bilal Yavuz wrote: > Thanks for the review! > From 053dd9d15416d76ce4b95044d848f51ba13a2d20 Mon Sep 17 00:00:00 2001 > From: Melanie Plageman > Date: Tue, 2 Sep 2025 11:00:44 -0400 > Subject: [PATCH v2 1/9] Refactor goto into for loop in GetVictimBuffer() >

Re: Checkpointer write combining

2025-09-11 Thread Chao Li
> On Sep 12, 2025, at 07:11, Melanie Plageman wrote: > > On Wed, Sep 10, 2025 at 4:24 AM Chao Li wrote: >> > > Thanks for the review! > > For any of your feedback that I simply implemented, I omitted an > inline comment about it. Those changes are included in the attached > v6. My inline re

Re: Checkpointer write combining

2025-09-11 Thread Melanie Plageman
On Wed, Sep 10, 2025 at 4:24 AM Chao Li wrote: > Thanks for the review! For any of your feedback that I simply implemented, I omitted an inline comment about it. Those changes are included in the attached v6. My inline replies below are only for feedback requiring more discussion. > On Sep 10,

Re: Checkpointer write combining

2025-09-09 Thread Jeff Davis
On Tue, 2025-09-09 at 13:55 -0400, Melanie Plageman wrote: > On Tue, Sep 9, 2025 at 11:16 AM Melanie Plageman > wrote: > > > > One more fix and a bit more cleanup in attached v4. > > Okay one more version: I updated the thread on eager flushing the > bulkwrite ring [1], and some updates were nee

Re: Checkpointer write combining

2025-09-09 Thread Melanie Plageman
On Tue, Sep 9, 2025 at 11:16 AM Melanie Plageman wrote: > > One more fix and a bit more cleanup in attached v4. Okay one more version: I updated the thread on eager flushing the bulkwrite ring [1], and some updates were needed here. - Melanie [1] https://www.postgresql.org/message-id/flat/CAAK

Re: Checkpointer write combining

2025-09-09 Thread Melanie Plageman
On Tue, Sep 9, 2025 at 9:39 AM Melanie Plageman wrote: > > Oops, you're right. v3 attached with that mistake fixed. One more fix and a bit more cleanup in attached v4. - Melanie From fef00f5bd61dc0e3cac95fa39a75b3d73ba7d0a6 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Tue, 2 Sep 2025 1

Re: Checkpointer write combining

2025-09-09 Thread Nazir Bilal Yavuz
Hi, Thank you for working on this! On Tue, 9 Sept 2025 at 02:44, Melanie Plageman wrote: > > On Tue, Sep 2, 2025 at 5:10 PM Melanie Plageman > wrote: > > > > The attached patchset implements checkpointer write combining -- which > > makes immediate checkpoints at least 20% faster in my tests. >

Re: Checkpointer write combining

2025-09-08 Thread Melanie Plageman
On Tue, Sep 2, 2025 at 5:10 PM Melanie Plageman wrote: > > The attached patchset implements checkpointer write combining -- which > makes immediate checkpoints at least 20% faster in my tests. > Checkpointer achieves higher write throughput and higher write IOPs > with the patch. These needed a r