Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-07-15 Thread Simon Riggs
On Wed, 14 Jul 2021 at 17:22, vignesh C wrote: > > On Thu, Apr 8, 2021 at 11:40 PM Simon Riggs wrote: > > > > On Thu, 8 Apr 2021 at 18:15, Alvaro Herrera wrote: > > > > > > On 2021-Apr-08, Simon Riggs wrote: > > > > > > > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > > > > > > > It's

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-07-14 Thread vignesh C
On Thu, Apr 8, 2021 at 11:40 PM Simon Riggs wrote: > > On Thu, 8 Apr 2021 at 18:15, Alvaro Herrera wrote: > > > > On 2021-Apr-08, Simon Riggs wrote: > > > > > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > > > > > It's not clear to me which patch is which, so perhaps move one CF entry > >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 18:15, Alvaro Herrera wrote: > > On 2021-Apr-08, Simon Riggs wrote: > > > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > > > It's not clear to me which patch is which, so perhaps move one CF entry > > > to next CF and clarify which patch is current? > > > > Entry:

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Simon Riggs wrote: > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > It's not clear to me which patch is which, so perhaps move one CF entry > > to next CF and clarify which patch is current? > > Entry: Maximize page freezing > has this patch, perfectly fine, awaiting

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 17:44, David Steele wrote: > > On 4/8/21 12:29 PM, Simon Riggs wrote: > > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > > It has been five months since this patch was updated, so marking > Returned with Feedback. > > Please resubmit to the next CF

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread David Steele
On 4/8/21 12:29 PM, Simon Riggs wrote: On Thu, 8 Apr 2021 at 16:58, David Steele wrote: It has been five months since this patch was updated, so marking Returned with Feedback. Please resubmit to the next CF when you have a new patch. There are 2 separate patch-sets on this thread, with

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > >> It has been five months since this patch was updated, so marking > >> Returned with Feedback. > >> > >> Please resubmit to the next CF when you have a new patch. > > > > There are 2 separate patch-sets on this thread, with separate CF

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread David Steele
On 4/8/21 11:41 AM, Simon Riggs wrote: On Thu, 8 Apr 2021 at 16:23, David Steele wrote: On 3/17/21 4:50 PM, Simon Riggs wrote: On Fri, 12 Mar 2021 at 22:16, Tomas Vondra wrote: On 1/28/21 2:33 PM, Simon Riggs wrote: On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: This entry has

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 16:23, David Steele wrote: > > On 3/17/21 4:50 PM, Simon Riggs wrote: > > On Fri, 12 Mar 2021 at 22:16, Tomas Vondra > > wrote: > >> > >> On 1/28/21 2:33 PM, Simon Riggs wrote: > >>> On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada > >>> wrote: > >>> > This entry has

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread David Steele
On 3/17/21 4:50 PM, Simon Riggs wrote: On Fri, 12 Mar 2021 at 22:16, Tomas Vondra wrote: On 1/28/21 2:33 PM, Simon Riggs wrote: On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: This entry has been "Waiting on Author" status and the patch has not been updated since Nov 30. Are you

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-03-17 Thread Simon Riggs
On Fri, 12 Mar 2021 at 22:16, Tomas Vondra wrote: > > On 1/28/21 2:33 PM, Simon Riggs wrote: > > On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: > > > >> This entry has been "Waiting on Author" status and the patch has not > >> been updated since Nov 30. Are you still planning to work on

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-03-12 Thread Tomas Vondra
On 1/28/21 2:33 PM, Simon Riggs wrote: > On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: > >> This entry has been "Waiting on Author" status and the patch has not >> been updated since Nov 30. Are you still planning to work on this? > > Yes, new patch version tomorrow. Thanks for the nudge

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-01-28 Thread Simon Riggs
On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: > This entry has been "Waiting on Author" status and the patch has not > been updated since Nov 30. Are you still planning to work on this? Yes, new patch version tomorrow. Thanks for the nudge and the review. -- Simon Riggs

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-01-28 Thread Masahiko Sawada
Hi Simon, On Mon, Jan 4, 2021 at 11:45 PM Masahiko Sawada wrote: > > On Tue, Dec 1, 2020 at 10:45 AM Masahiko Sawada wrote: > > > > On Fri, Nov 20, 2020 at 8:47 PM Simon Riggs wrote: > > > > > > On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > > > > > > > On Fri, 20 Nov 2020 at 01:40,

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-01-04 Thread Masahiko Sawada
On Tue, Dec 1, 2020 at 10:45 AM Masahiko Sawada wrote: > > On Fri, Nov 20, 2020 at 8:47 PM Simon Riggs wrote: > > > > On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > > > > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-12-28 Thread Masahiko Sawada
Hi Simon, On Mon, Nov 30, 2020 at 1:53 AM Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 15:29, Alvaro Herrera wrote: > > > > Note on heap_prepare_freeze_tuple()'s fifth parameter, it's not valid to > > pass OldestXmin; you need a multixact limit there, not an Xid limit. I > > think the return

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-30 Thread Masahiko Sawada
On Fri, Nov 20, 2020 at 8:47 PM Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote:

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-29 Thread Simon Riggs
On Fri, 20 Nov 2020 at 15:29, Alvaro Herrera wrote: > > Note on heap_prepare_freeze_tuple()'s fifth parameter, it's not valid to > pass OldestXmin; you need a multixact limit there, not an Xid limit. I > think the return value of GetOldestMultiXactId is a good choice. AFAICS > this means that

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-29 Thread Simon Riggs
On Fri, 20 Nov 2020 at 11:47, Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Robert Haas
On Fri, Nov 20, 2020 at 11:02 AM Alvaro Herrera wrote: > On 2020-Nov-20, Robert Haas wrote: > > Yeah, I think dirtying the page fewer times is a big win. However, a > > page may have tuples that are not yet all-visible, and we can't freeze > > those just because we are freezing others. > > Of

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Alvaro Herrera
On 2020-Nov-20, Robert Haas wrote: > Yeah, I think dirtying the page fewer times is a big win. However, a > page may have tuples that are not yet all-visible, and we can't freeze > those just because we are freezing others. Of course! We should only freeze tuples that are freezable. I thought

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Robert Haas
On Fri, Nov 20, 2020 at 9:08 AM Alvaro Herrera wrote: > There are two costs associated with this processing. One is dirtying > the page (which means it needs to be written down when evicted), and the > other is to write WAL records for each change. The cost for the latter > is going to be the

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Alvaro Herrera
On 2020-Nov-20, Simon Riggs wrote: > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > Since we use the term aggressive scan in the docs, I personally don't > > feel unnatural about that. But since this option also disables index > > cleanup when not enabled explicitly, I’m concerned a

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Alvaro Herrera
Note on heap_prepare_freeze_tuple()'s fifth parameter, it's not valid to pass OldestXmin; you need a multixact limit there, not an Xid limit. I think the return value of GetOldestMultiXactId is a good choice. AFAICS this means that you'll need to add a new output argument to

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
On Fri, 20 Nov 2020 at 14:07, Alvaro Herrera wrote: > > On 2020-Nov-20, Masahiko Sawada wrote: > > > I'm concerned that always freezing all tuples when we're going to make > > the page dirty would affect the existing vacuum workload much. The > > additional cost of freezing multiple tuples would

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Alvaro Herrera
On 2020-Nov-20, Masahiko Sawada wrote: > I'm concerned that always freezing all tuples when we're going to make > the page dirty would affect the existing vacuum workload much. The > additional cost of freezing multiple tuples would be low but if we > freeze tuples we would also need to write

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > > > > > > > On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs > > >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
On Fri, 20 Nov 2020 at 03:54, Masahiko Sawada wrote: > > > So +1 for this idea. > > > > Patch to do this attached, for discussion. > > Thank you for the patch! > > +* > +* Once we decide to dirty the data block we may as well > freeze > +* any

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > > > > > On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs > > > wrote: > > > > Patches attached. > > > > 1.

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-19 Thread Masahiko Sawada
On Fri, Nov 20, 2020 at 6:02 AM Simon Riggs wrote: > > On Wed, 18 Nov 2020 at 02:04, Alvaro Herrera wrote: > > > > On 2020-Nov-17, Simon Riggs wrote: > > > > > As an additional optimization, if we do find a row that needs freezing > > > on a data block, we should simply freeze *all* row versions

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-19 Thread Masahiko Sawada
On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > > > On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs wrote: > > > Patches attached. > > > 1. vacuum_anti_wraparound.v2.patch > > > 2. vacuumdb_anti_wrap.v1.patch - depends upon (1) > > > > I

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-19 Thread Simon Riggs
On Wed, 18 Nov 2020 at 02:04, Alvaro Herrera wrote: > > On 2020-Nov-17, Simon Riggs wrote: > > > As an additional optimization, if we do find a row that needs freezing > > on a data block, we should simply freeze *all* row versions on the > > page, not just the ones below the selected cutoff.

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-19 Thread Simon Riggs
On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs wrote: > > Patches attached. > > 1. vacuum_anti_wraparound.v2.patch > > 2. vacuumdb_anti_wrap.v1.patch - depends upon (1) > > I don't like the use of ANTI_WRAPAROUND as a name for this new option. >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-18 Thread Robert Haas
On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs wrote: > Patches attached. > 1. vacuum_anti_wraparound.v2.patch > 2. vacuumdb_anti_wrap.v1.patch - depends upon (1) I don't like the use of ANTI_WRAPAROUND as a name for this new option. Wouldn't it make more sense to call it AGGRESSIVE? Or maybe

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-18 Thread Simon Riggs
On Wed, 18 Nov 2020 at 10:28, Masahiko Sawada wrote: > > So we have 3 ways to reset relfrozenxid by a user action: > > VACUUM (DISABLE_PAGE_SKIPPING ON) - scans all blocks, deliberately > > ignoring the ones it could have skipped. This certainly slows it down. > > VACU

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-18 Thread Masahiko Sawada
s the vacuum to reset > relfrozenxid, but (2) actually slows down the scan by making it freeze > more blocks than it would do normally. > > So we have 3 ways to reset relfrozenxid by a user action: > VACUUM (DISABLE_PAGE_SKIPPING ON) - scans all blocks, deliberately > ignori

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-17 Thread Alvaro Herrera
On 2020-Nov-17, Simon Riggs wrote: > As an additional optimization, if we do find a row that needs freezing > on a data block, we should simply freeze *all* row versions on the > page, not just the ones below the selected cutoff. This is justified > since writing the block is the biggest cost and

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-17 Thread Simon Riggs
, but (2) actually slows down the scan by making it freeze more blocks than it would do normally. So we have 3 ways to reset relfrozenxid by a user action: VACUUM (DISABLE_PAGE_SKIPPING ON) - scans all blocks, deliberately ignoring the ones it could have skipped. This certainly slows it down. VACUUM (FR

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-16 Thread Masahiko Sawada
On Tue, Nov 17, 2020 at 5:52 AM Simon Riggs wrote: > > The docs are misleading for this feature, since they say: > "This option disables all page-skipping behavior, and is > intended to be used only when the contents of the visibility map are > suspect, which should happen only if there is a

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-16 Thread David G. Johnston
On Mon, Nov 16, 2020 at 1:52 PM Simon Riggs wrote: > The docs are misleading for this feature, since they say: > "This option disables all page-skipping behavior, and is > intended to be used only when the contents of the visibility map are > suspect, which should happen only if there is a

VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-16 Thread Simon Riggs
The docs are misleading for this feature, since they say: "This option disables all page-skipping behavior, and is intended to be used only when the contents of the visibility map are suspect, which should happen only if there is a hardware or software issue causing database corruption." The docs