Re: non-HOT update not looking at FSM for large tuple update

2021-03-27 Thread Noah Misch
On Sat, Mar 27, 2021 at 11:26:47AM -0400, John Naylor wrote: > On Sat, Mar 27, 2021 at 3:00 AM Noah Misch wrote: > > Does anyone have a strong opinion on whether to back-patch?  I am weakly > > inclined not to back-patch, because today's behavior might happen to perform > > better when

Re: non-HOT update not looking at FSM for large tuple update

2021-03-27 Thread John Naylor
On Sat, Mar 27, 2021 at 3:00 AM Noah Misch wrote: > > Does anyone have a strong opinion on whether to back-patch? I am weakly > inclined not to back-patch, because today's behavior might happen to perform > better when large_upd_rate-small_ins_rate<0. It's not a clear case. The present behavior

RE: non-HOT update not looking at FSM for large tuple update

2021-03-27 Thread Floris Van Nee
Hi Noah, Thanks for taking a look at this patch. > > In evaluating whether this is a good choice of value, I think about the > expected page lifecycle. A tuple barely larger than fillfactor (roughly > len=1+BLCKSZ*fillfactor/100) will start on a roughly-empty page. As long as > the tuple

Re: non-HOT update not looking at FSM for large tuple update

2021-03-27 Thread Noah Misch
I gather this is important when large_upd_rate=rate(cross-page update bytes for tuples larger than fillfactor) exceeds small_ins_rate=rate(insert bytes for tuples NOT larger than fillfactor). That is a plausible outcome when inserts are rare, and table bloat then accrues at

Re: non-HOT update not looking at FSM for large tuple update

2021-03-19 Thread Matthias van de Meent
On Fri, 19 Mar 2021 at 19:16, John Naylor wrote: > > On Thu, Mar 18, 2021 at 5:30 PM Matthias van de Meent > wrote: > > > > This is slightly more verbose, but I think this clarifies the > > reasoning why we need this a bit better. Feel free to reject or adapt > > as needed. > > I like this in

Re: non-HOT update not looking at FSM for large tuple update

2021-03-19 Thread John Naylor
On Thu, Mar 18, 2021 at 5:30 PM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > > > + * The minimum space on a page for it to be considered "empty" regardless > > + * of fillfactor. We base this on MaxHeapTupleSize, minus a small amount > > + * of slack. We allow slack

Re: non-HOT update not looking at FSM for large tuple update

2021-03-18 Thread Matthias van de Meent
On Wed, 17 Mar 2021 at 21:52, John Naylor wrote: > > On Fri, Mar 12, 2021 at 8:45 AM Matthias van de Meent > wrote: > > > > If this case isn't added, the lower else branch will fail to find > > fitting pages for len > maxPaddedFsmRequest tuples; potentially > > extending the relation when there

Re: non-HOT update not looking at FSM for large tuple update

2021-03-17 Thread John Naylor
On Fri, Mar 12, 2021 at 8:45 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > > If this case isn't added, the lower else branch will fail to find > fitting pages for len > maxPaddedFsmRequest tuples; potentially > extending the relation when there is actually still enough space >

Re: non-HOT update not looking at FSM for large tuple update

2021-03-12 Thread Matthias van de Meent
On Thu, 11 Mar 2021 at 16:16, John Naylor wrote: > > On Thu, Mar 11, 2021 at 9:46 AM Matthias van de Meent > wrote: > > > Regarding the 2% slack logic, could we change it to use increments of > > line pointers instead? That makes it more clear what problem this > > solution is trying to work

Re: non-HOT update not looking at FSM for large tuple update

2021-03-11 Thread John Naylor
On Thu, Mar 11, 2021 at 9:46 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > Regarding the 2% slack logic, could we change it to use increments of > line pointers instead? That makes it more clear what problem this > solution is trying to work around; that is to say line

Re: non-HOT update not looking at FSM for large tuple update

2021-03-11 Thread Matthias van de Meent
On Tue, 9 Mar 2021 at 18:39, John Naylor wrote: > > I wrote: > > > That seems like the proper fix, and I see you've started a thread for that. > > I don't think that change in behavior would be backpatchable, but patch > > here might have a chance at that. > > I remembered after the fact that

Re: non-HOT update not looking at FSM for large tuple update

2021-03-09 Thread John Naylor
I wrote: > That seems like the proper fix, and I see you've started a thread for that. I don't think that change in behavior would be backpatchable, but patch here might have a chance at that. I remembered after the fact that truncating line pointers would only allow for omitting the 2% slack

Re: non-HOT update not looking at FSM for large tuple update

2021-03-09 Thread John Naylor
On Tue, Mar 9, 2021 at 9:40 AM Floris Van Nee wrote: > > Hi, > > > > > This patch fails to consider that len may be bigger than MaxHeapTupleSize * > > 0.98, which in this case triggers a reproducable > > PANIC: > > Good catch! I've adapted the patch with your suggested fix. Thank you both for

RE: non-HOT update not looking at FSM for large tuple update

2021-03-09 Thread Floris Van Nee
Hi, > > This patch fails to consider that len may be bigger than MaxHeapTupleSize * > 0.98, which in this case triggers a reproducable > PANIC: Good catch! I've adapted the patch with your suggested fix. > > One different question I have, though, is why we can't "just" teach vacuum > to clean

Re: non-HOT update not looking at FSM for large tuple update

2021-03-08 Thread Matthias van de Meent
On Mon, 8 Mar 2021 at 16:25, Floris Van Nee wrote: > > > I've added this to the commitfest as a bug fix and added you as an author. > > Thanks. Patch looks good to me, but I guess there needs to be someone else > reviewing too? > Also, would this be a backpatchable bugfix? > > -Floris > This

RE: non-HOT update not looking at FSM for large tuple update

2021-03-08 Thread Floris Van Nee
> I've added this to the commitfest as a bug fix and added you as an author. Thanks. Patch looks good to me, but I guess there needs to be someone else reviewing too? Also, would this be a backpatchable bugfix? -Floris

Re: non-HOT update not looking at FSM for large tuple update

2021-02-26 Thread John Naylor
On Wed, Feb 24, 2021 at 6:29 PM Floris Van Nee wrote: > > > > That makes sense, although the exact number seems precisely tailored to your use case. 2% gives 164 bytes of slack and doesn't seem too large. Updated patch attached. > > Yeah, I tried picking it as conservative as I could, but 2% is

RE: non-HOT update not looking at FSM for large tuple update

2021-02-24 Thread Floris Van Nee
> That makes sense, although the exact number seems precisely tailored to your > use case. 2% gives 164 bytes of slack and doesn't seem too large. Updated > patch attached. Yeah, I tried picking it as conservative as I could, but 2% is obviously great too. :-) I can't think of any large

Re: non-HOT update not looking at FSM for large tuple update

2021-02-24 Thread John Naylor
On Wed, Feb 24, 2021 at 4:52 PM Floris Van Nee wrote: > I also understand the temptation to define it based on the relation's fill factor, as you did in the patch. However, upon some further thought I wonder if that's useful? A relation with a higher fill factor will have a lower 'saveFreeSpace'

RE: non-HOT update not looking at FSM for large tuple update

2021-02-24 Thread Floris Van Nee
> In this case, the vast majority has between 8050-8099 bytes free according to > the FSM. That means that, for this particular case, for a fillfactor of 10, > we’d need to subtract ~120 bytes or so in order to properly recycle the pages. Also, I think this "fudge" factor would need to be

RE: non-HOT update not looking at FSM for large tuple update

2021-02-24 Thread Floris Van Nee
Hi John, > One idea is to take your -50 idea and make it more general and safe, by > scaling the fudge factor based on fillfactor, such that if fillfactor is less > than 100, the requested freespace is a bit smaller than the max. It's still a > bit of a hack, though. I've attached a draft of

Re: non-HOT update not looking at FSM for large tuple update

2021-02-24 Thread John Naylor
On Wed, Feb 24, 2021 at 10:44 AM Floris Van Nee wrote: > > The problem here is two-folded: for any non-HOT update of a tuple that’s larger than the size of the fillfactor, the fsm will not be used, but instead a new page will be chosen. I confirmed this not only non-HOT updates, but regular