Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Jun 26, 2018 at 4:11 PM Darafei "Komяpa" Praliaskouski wrote: > вт, 26 июн. 2018 г. в 15:42, Alexander Korotkov : >> >> On Tue, Jun 26, 2018 at 1:46 PM Masahiko Sawada >> wrote: >> > On Fri, Jun 22, 2018 at 6:55 PM, Alexander Korotkov >> > wrote: >> > > So, I propose to just >> > >

Re: [HACKERS] GUC for cleanup indexes threshold.

вт, 26 июн. 2018 г. в 15:42, Alexander Korotkov : > On Tue, Jun 26, 2018 at 1:46 PM Masahiko Sawada > wrote: > > On Fri, Jun 22, 2018 at 6:55 PM, Alexander Korotkov > > wrote: > > > So, I propose to just > > > increase maximum value for both GUC and reloption. See the attached > > > patch. It

Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Jun 26, 2018 at 1:46 PM Masahiko Sawada wrote: > On Fri, Jun 22, 2018 at 6:55 PM, Alexander Korotkov > wrote: > > So, I propose to just > > increase maximum value for both GUC and reloption. See the attached > > patch. It also changes calculations _bt_vacuum_needs_cleanup() for > >

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Jun 22, 2018 at 6:55 PM, Alexander Korotkov wrote: > On Wed, Jun 20, 2018 at 12:00 PM Alexander Korotkov > wrote: >> On Wed, Jun 20, 2018 at 8:32 AM Masahiko Sawada >> wrote: >> > On Wed, Jun 20, 2018 at 1:00 AM, Alexander Korotkov >> > > Ok. I've rephrased comment a bit. Also, you

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Jun 20, 2018 at 12:00 PM Alexander Korotkov wrote: > On Wed, Jun 20, 2018 at 8:32 AM Masahiko Sawada wrote: > > On Wed, Jun 20, 2018 at 1:00 AM, Alexander Korotkov > > > Ok. I've rephrased comment a bit. Also, you created "index vacuum" > > > subsection in the "resource usage" section.

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Jun 20, 2018 at 8:32 AM Masahiko Sawada wrote: > On Wed, Jun 20, 2018 at 1:00 AM, Alexander Korotkov > > Ok. I've rephrased comment a bit. Also, you created "index vacuum" > > subsection in the "resource usage" section. I think it's not > > appropriate for this option to be in

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Jun 20, 2018 at 1:00 AM, Alexander Korotkov wrote: > On Tue, Jun 19, 2018 at 12:25 PM Masahiko Sawada > wrote: >> On Tue, Jun 19, 2018 at 5:43 PM, Alexander Korotkov >> wrote: >> > On Tue, Jun 19, 2018 at 11:34 AM Masahiko Sawada >> > wrote: >> >> On Mon, Jun 18, 2018 at 1:56 PM,

Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Jun 19, 2018 at 12:25 PM Masahiko Sawada wrote: > On Tue, Jun 19, 2018 at 5:43 PM, Alexander Korotkov > wrote: > > On Tue, Jun 19, 2018 at 11:34 AM Masahiko Sawada > > wrote: > >> On Mon, Jun 18, 2018 at 1:56 PM, Alexander Korotkov > >> > So, I'm proposing to raise maximum valus of >

Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Jun 19, 2018 at 11:34 AM Masahiko Sawada wrote: > On Mon, Jun 18, 2018 at 1:56 PM, Alexander Korotkov > > So, I'm proposing to raise maximum valus of > > vacuum_cleanup_index_scale_factor to DBL_MAX. Any objections? > > > > I agree to expand the maximum value. But if users don't want

Re: [HACKERS] GUC for cleanup indexes threshold.

Hi! On Sat, Jun 16, 2018 at 11:23 PM Darafei "Komяpa" Praliaskouski wrote: > It is cool to see this in Postgres 11. However: > >> >> 4) vacuum_cleanup_index_scale_factor can be set either by GUC or reloption. >> Default value is 0.1. So, by default cleanup scan is triggered after >> increasing

Re: [HACKERS] GUC for cleanup indexes threshold.

Hi! It is cool to see this in Postgres 11. However: > 4) vacuum_cleanup_index_scale_factor can be set either by GUC or > reloption. > Default value is 0.1. So, by default cleanup scan is triggered after > increasing of > table size by 10%. > vacuum_cleanup_index_scale_factor can be set to the

Re: [HACKERS] GUC for cleanup indexes threshold.

At Fri, 6 Apr 2018 10:52:58 +0900, Masahiko Sawada wrote in > On Thu, Apr 5, 2018 at 7:23 PM, Teodor Sigaev wrote: > > Thanks to everyone, fixes are pushed except nodeMerge.c, I don't

Re: [HACKERS] GUC for cleanup indexes threshold.

On Thu, Apr 5, 2018 at 7:23 PM, Teodor Sigaev wrote: > Thanks to everyone, fixes are pushed except nodeMerge.c, I don't wish to > increase entropy around MERGE patch :) > Thank you! Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source

Re: [HACKERS] GUC for cleanup indexes threshold.

Hello. The commit leaves three warnings for -Wunused-but-set-variable. Two of them are not assertion-only but really not used at all. I also found that nodeMerge.c has one such variable. regards. At Thu, 5 Apr 2018 15:43:55 +0900, Masahiko Sawada wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

On Thu, Apr 5, 2018 at 2:40 PM, Masahiko Sawada wrote: > On Thu, Apr 5, 2018 at 1:30 AM, Teodor Sigaev wrote: >> Thanks for everyone, pushed with minor editorization >> > > Thank you for committing! > I found a typo in nbtpage.c and attached a patch fixes

Re: [HACKERS] GUC for cleanup indexes threshold.

On Thu, Apr 5, 2018 at 1:30 AM, Teodor Sigaev wrote: > Thanks for everyone, pushed with minor editorization > Thank you for committing! I found a typo in nbtpage.c and attached a patch fixes it. s/overritten/overwritten/ Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND

Re: [HACKERS] GUC for cleanup indexes threshold.

Thanks for everyone, pushed with minor editorization Alexander Korotkov wrote: On Tue, Apr 3, 2018 at 6:42 PM, Masahiko Sawada > wrote: On Tue, Mar 27, 2018 at 8:01 PM, Alexander Korotkov > So, I would like to clarify why could my

Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Apr 3, 2018 at 6:42 PM, Masahiko Sawada wrote: > On Tue, Mar 27, 2018 at 8:01 PM, Alexander Korotkov > > So, I would like to clarify why could my patch block future improvements > > in this area? For instance, if we would decide to make btree able to > skip > >

Re: [HACKERS] GUC for cleanup indexes threshold.

Sorry for the late response. On Tue, Mar 27, 2018 at 8:01 PM, Alexander Korotkov wrote: > On Tue, Mar 27, 2018 at 11:55 AM, Masahiko Sawada > wrote: >> >> On Thu, Mar 22, 2018 at 9:24 PM, Alexander Korotkov >> wrote:

Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Mar 27, 2018 at 11:55 AM, Masahiko Sawada wrote: > On Thu, Mar 22, 2018 at 9:24 PM, Alexander Korotkov > wrote: > > However, I see that you are comparing relative change of num_heap_tuples > > before and after vacuum to

Re: [HACKERS] GUC for cleanup indexes threshold.

On Thu, Mar 22, 2018 at 9:24 PM, Alexander Korotkov wrote: > On Mon, Mar 19, 2018 at 5:12 AM, Masahiko Sawada > wrote: >> >> On Wed, Mar 14, 2018 at 9:25 PM, Alexander Korotkov >> wrote: >> > On Wed, Mar 14, 2018 at

Re: [HACKERS] GUC for cleanup indexes threshold.

On Mon, Mar 19, 2018 at 8:45 AM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > At Mon, 19 Mar 2018 11:12:58 +0900, Masahiko Sawada > wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

On Mon, Mar 19, 2018 at 5:12 AM, Masahiko Sawada wrote: > On Wed, Mar 14, 2018 at 9:25 PM, Alexander Korotkov > wrote: > > On Wed, Mar 14, 2018 at 7:40 AM, Masahiko Sawada > > wrote: > >> > >> On Sat, Mar 10, 2018 at 3:40

Re: [HACKERS] GUC for cleanup indexes threshold.

At Tue, 20 Mar 2018 13:57:19 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180320.135719.90053076.horiguchi.kyot...@lab.ntt.co.jp> > At Mon, 19 Mar 2018 20:50:48 +0900, Masahiko Sawada > wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

At Mon, 19 Mar 2018 20:50:48 +0900, Masahiko Sawada wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

On Mon, Mar 19, 2018 at 8:50 AM, Masahiko Sawada wrote: >>> require the bulk-delete method of scanning whole index and of logging >>> WAL. But it leads some extra overhead. With this patch we no longer >>> need to depend on the full scan on b-tree index. This might be

Re: [HACKERS] GUC for cleanup indexes threshold.

On Mon, Mar 19, 2018 at 2:45 PM, Kyotaro HORIGUCHI wrote: > At Mon, 19 Mar 2018 11:12:58 +0900, Masahiko Sawada > wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

Sorry I'd like to make a trivial but critical fix. At Mon, 19 Mar 2018 14:45:05 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180319.144505.166111203.horiguchi.kyot...@lab.ntt.co.jp> > At Mon, 19 Mar 2018 11:12:58 +0900, Masahiko Sawada

Re: [HACKERS] GUC for cleanup indexes threshold.

At Mon, 19 Mar 2018 11:12:58 +0900, Masahiko Sawada wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Mar 14, 2018 at 9:25 PM, Alexander Korotkov wrote: > On Wed, Mar 14, 2018 at 7:40 AM, Masahiko Sawada > wrote: >> >> On Sat, Mar 10, 2018 at 3:40 AM, Alexander Korotkov >> wrote: >> > On Fri, Mar 9, 2018 at

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Mar 14, 2018 at 7:40 AM, Masahiko Sawada wrote: > On Sat, Mar 10, 2018 at 3:40 AM, Alexander Korotkov > wrote: > > On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada > > wrote: > >> > >> On Fri, Mar 9, 2018 at 8:43

Re: [HACKERS] GUC for cleanup indexes threshold.

On Sat, Mar 10, 2018 at 3:40 AM, Alexander Korotkov wrote: > On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada > wrote: >> >> On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov >> wrote: >> > 2) These parameters are

Re: [HACKERS] GUC for cleanup indexes threshold.

On Sat, Mar 10, 2018 at 5:49 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Fri, Mar 9, 2018 at 9:40 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada >> wrote: >> >>> Attached an updated

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Mar 9, 2018 at 9:40 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada > wrote: > >> Attached an updated patch >> > fixed these issue. Will review the patch again. > > > Thank you! > I've fixed a bug:

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada wrote: > On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov > wrote: > > 2) These parameters are reset during btbulkdelete() and set during > > btvacuumcleanup(). > > Can't we set these parameters

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov wrote: > Hi! > Sorry for my late reply. > I'd like to propose a revised patch based on various ideas upthread. Thank you for proposing the patch! > > This patch works as following. > > 1) B-tree meta page is

Re: [HACKERS] GUC for cleanup indexes threshold.

Hi! I'd like to propose a revised patch based on various ideas upthread. This patch works as following. 1) B-tree meta page is extended with 2 additional parameters: * btm_oldest_btpo_xact – oldest btpo_xact among of deleted pages, * btm_last_cleanup_num_heap_tuples – number of heap tuples

Re: [HACKERS] GUC for cleanup indexes threshold.

On Mon, Mar 5, 2018 at 5:56 AM, Masahiko Sawada wrote: > On Sun, Mar 4, 2018 at 8:59 AM, Alexander Korotkov > wrote: > > On Fri, Mar 2, 2018 at 10:53 AM, Masahiko Sawada > > wrote: > >> > >> > 2) In the append-only case,

Re: [HACKERS] GUC for cleanup indexes threshold.

On Sun, Mar 4, 2018 at 8:59 AM, Alexander Korotkov wrote: > On Fri, Mar 2, 2018 at 10:53 AM, Masahiko Sawada > wrote: >> >> > 2) In the append-only case, index statistics can lag indefinitely. >> >> The original proposal proposed a new GUC that

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Mar 2, 2018 at 10:53 AM, Masahiko Sawada wrote: > > 2) In the append-only case, index statistics can lag indefinitely. > > The original proposal proposed a new GUC that specifies a fraction of > the modified pages to trigger a cleanup indexes. Regarding original

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Feb 28, 2018 at 1:45 AM, Alexander Korotkov wrote: > On Wed, Nov 29, 2017 at 6:06 PM, Masahiko Sawada > wrote: >> >> On Wed, Nov 29, 2017 at 11:05 PM, Simon Riggs >> wrote: >> > On 25 September 2017 at 22:34,

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Jan 12, 2018 at 7:05 AM, Masahiko Sawada wrote: > On Wed, Jan 10, 2018 at 11:28 AM, Masahiko Sawada > wrote: > > On Sun, Jan 7, 2018 at 8:40 AM, Peter Geoghegan wrote: > >> On Sat, Jan 6, 2018 at 2:20 PM, Stephen Frost

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Nov 29, 2017 at 6:06 PM, Masahiko Sawada wrote: > On Wed, Nov 29, 2017 at 11:05 PM, Simon Riggs > wrote: > > On 25 September 2017 at 22:34, Kyotaro HORIGUCHI > > wrote: > > > >>> > Here is a small patch that

Re: [HACKERS] GUC for cleanup indexes threshold.

On Fri, Jan 12, 2018 at 1:05 PM, Masahiko Sawada wrote: > On Wed, Jan 10, 2018 at 11:28 AM, Masahiko Sawada > wrote: >> On Sun, Jan 7, 2018 at 8:40 AM, Peter Geoghegan wrote: >>> On Sat, Jan 6, 2018 at 2:20 PM, Stephen Frost

Re: [HACKERS] GUC for cleanup indexes threshold.

On Wed, Jan 10, 2018 at 11:28 AM, Masahiko Sawada wrote: > On Sun, Jan 7, 2018 at 8:40 AM, Peter Geoghegan wrote: >> On Sat, Jan 6, 2018 at 2:20 PM, Stephen Frost wrote: > IIRC the patches that makes the cleanup scan skip has a

Re: [HACKERS] GUC for cleanup indexes threshold.

On Sun, Jan 7, 2018 at 8:40 AM, Peter Geoghegan wrote: > On Sat, Jan 6, 2018 at 2:20 PM, Stephen Frost wrote: >>> > IIRC the patches that makes the cleanup scan skip has a problem >>> > pointed by Peter[1], that is that we stash an XID when a btree page is >>> >

Re: [HACKERS] GUC for cleanup indexes threshold.

Hi Stephen, On Sat, Jan 6, 2018 at 4:02 PM, Stephen Frost wrote: > Perhaps it should really be in Needs review state then..? Probably. As I pointed out already some time ago, this RecentGlobalXmin interlock stuff is our particular implementation of what Lanin & Shasha call

Re: [HACKERS] GUC for cleanup indexes threshold.

Greetings Peter, * Peter Geoghegan (p...@bowt.ie) wrote: > On Sat, Jan 6, 2018 at 2:20 PM, Stephen Frost wrote: > >> > IIRC the patches that makes the cleanup scan skip has a problem > >> > pointed by Peter[1], that is that we stash an XID when a btree page is > >> > deleted,

Re: [HACKERS] GUC for cleanup indexes threshold.

On Sat, Jan 6, 2018 at 2:20 PM, Stephen Frost wrote: >> > IIRC the patches that makes the cleanup scan skip has a problem >> > pointed by Peter[1], that is that we stash an XID when a btree page is >> > deleted, which is used to determine when it's finally safe to recycle >> >

Re: [HACKERS] GUC for cleanup indexes threshold.

Greetings, (pruned the CC list) * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Nov 30, 2017 at 12:06 AM, Masahiko Sawada > wrote: > > On Wed, Nov 29, 2017 at 11:05 PM, Simon Riggs wrote: > >> Unless there is disagreement on the

Re: [HACKERS] GUC for cleanup indexes threshold.

On Thu, Nov 30, 2017 at 12:06 AM, Masahiko Sawada wrote: > On Wed, Nov 29, 2017 at 11:05 PM, Simon Riggs wrote: >> Unless there is disagreement on the above, it seems we should apply >> Yura's patch (an edited version, perhaps). >> > > IIRC the