Re: [HACKERS] GUC for cleanup indexes threshold.

2017-10-31 Thread Robert Haas
On Mon, Oct 16, 2017 at 3:02 PM, Masahiko Sawada wrote: > I guess that is the patch I proposed. However I think that there still > is room for discussion because the patch cannot skip to cleanup vacuum > when aggressive vacuum, which is one of the situation that I really >

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-10-16 Thread Masahiko Sawada
On Fri, Oct 13, 2017 at 1:14 AM, Robert Haas wrote: > On Tue, Oct 10, 2017 at 5:55 AM, Pavel Golub wrote: >> DP> The new status of this patch is: Ready for Committer >> >> Seems like, we may also going to hit it and it would be cool this >> vacuum

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-10-12 Thread Robert Haas
On Tue, Oct 10, 2017 at 5:55 AM, Pavel Golub wrote: > DP> The new status of this patch is: Ready for Committer > > Seems like, we may also going to hit it and it would be cool this > vacuum issue solved for next PG version. Exactly which patch on this thread is someone

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-10-10 Thread Pavel Golub
Hello, Darafei. You wrote: DP> The following review has been posted through the commitfest application: DP> make installcheck-world: tested, passed DP> Implements feature: tested, passed DP> Spec compliant: tested, passed DP> Documentation:tested, passed DP> We're

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-25 Thread Kyotaro HORIGUCHI
At Fri, 22 Sep 2017 17:15:08 +0300, Sokolov Yura wrote in > On 2017-09-22 16:22, Sokolov Yura wrote: > > On 2017-09-22 11:21, Masahiko Sawada wrote: > >> On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro HORIGUCHI > >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-25 Thread Kyotaro HORIGUCHI
At Mon, 25 Sep 2017 19:20:07 +0900, Masahiko Sawada wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-25 Thread Masahiko Sawada
On Fri, Sep 22, 2017 at 5:31 PM, Kyotaro HORIGUCHI wrote: > At Fri, 22 Sep 2017 17:21:04 +0900, Masahiko Sawada > wrote in >> On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Darafei Praliaskouski
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed We're using Postgres with this patch for some time. In our

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Sokolov Yura
On 2017-09-22 16:22, Sokolov Yura wrote: On 2017-09-22 11:21, Masahiko Sawada wrote: On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro HORIGUCHI wrote: At Fri, 22 Sep 2017 15:00:20 +0900, Masahiko Sawada wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Sokolov Yura
On 2017-09-22 11:21, Masahiko Sawada wrote: On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro HORIGUCHI wrote: At Fri, 22 Sep 2017 15:00:20 +0900, Masahiko Sawada wrote in On

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Claudio Freire
On Fri, Sep 22, 2017 at 4:46 AM, Kyotaro HORIGUCHI wrote: > I apologize in advance of possible silliness. > > At Thu, 21 Sep 2017 13:54:01 -0300, Claudio Freire > wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Kyotaro HORIGUCHI
At Fri, 22 Sep 2017 17:21:04 +0900, Masahiko Sawada wrote in > On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro HORIGUCHI > wrote: > > At Fri, 22 Sep 2017 15:00:20 +0900,

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Masahiko Sawada
On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro HORIGUCHI wrote: > At Fri, 22 Sep 2017 15:00:20 +0900, Masahiko Sawada > wrote in >> On Tue, Sep 19, 2017 at 3:31 PM, Kyotaro

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Kyotaro HORIGUCHI
I apologize in advance of possible silliness. At Thu, 21 Sep 2017 13:54:01 -0300, Claudio Freire wrote in

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Kyotaro HORIGUCHI
At Fri, 22 Sep 2017 15:00:20 +0900, Masahiko Sawada wrote in > On Tue, Sep 19, 2017 at 3:31 PM, Kyotaro HORIGUCHI > wrote: > > I was just looking the thread since it is

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-22 Thread Masahiko Sawada
On Tue, Sep 19, 2017 at 3:31 PM, Kyotaro HORIGUCHI wrote: > I was just looking the thread since it is found left alone for a > long time in the CF app. > > At Mon, 18 Sep 2017 16:35:58 -0700, Peter Geoghegan wrote in >

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-21 Thread Claudio Freire
On Tue, Sep 19, 2017 at 8:55 PM, Peter Geoghegan wrote: > On Tue, Sep 19, 2017 at 4:47 PM, Claudio Freire > wrote: >> Maybe this is looking at the problem from the wrong direction. >> >> Why can't the page be added to the FSM immediately and the check be >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-21 Thread Kyotaro HORIGUCHI
Hi, At Tue, 19 Sep 2017 16:55:38 -0700, Peter Geoghegan wrote in > On Tue, Sep 19, 2017 at 4:47 PM, Claudio Freire > wrote: > > Maybe this is looking at the problem from the wrong

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-19 Thread Peter Geoghegan
On Tue, Sep 19, 2017 at 4:47 PM, Claudio Freire wrote: > Maybe this is looking at the problem from the wrong direction. > > Why can't the page be added to the FSM immediately and the check be > done at runtime when looking for a reusable page? > > Index FSMs currently

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-19 Thread Claudio Freire
On Tue, Sep 19, 2017 at 3:31 AM, Kyotaro HORIGUCHI wrote: > I was just looking the thread since it is found left alone for a > long time in the CF app. > > At Mon, 18 Sep 2017 16:35:58 -0700, Peter Geoghegan wrote in >

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-19 Thread Kyotaro HORIGUCHI
I was just looking the thread since it is found left alone for a long time in the CF app. At Mon, 18 Sep 2017 16:35:58 -0700, Peter Geoghegan wrote in > On Wed, Apr 5, 2017 at 3:50 PM, Andres Freund

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-18 Thread Peter Geoghegan
On Wed, Apr 5, 2017 at 3:50 PM, Andres Freund wrote: > Hi, > > On 2017-04-01 03:05:07 +0900, Masahiko Sawada wrote: >> On Fri, Mar 31, 2017 at 11:44 PM, Robert Haas wrote: >> [ lots of valuable discussion ] > > I think this patch clearly still is in the

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-04-05 Thread Andres Freund
Hi, On 2017-04-01 03:05:07 +0900, Masahiko Sawada wrote: > On Fri, Mar 31, 2017 at 11:44 PM, Robert Haas wrote: > [ lots of valuable discussion ] I think this patch clearly still is in the design stage, and has received plenty feedback this CF. I'll therefore move this

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread Masahiko Sawada
On Fri, Mar 31, 2017 at 11:44 PM, Robert Haas wrote: > On Wed, Mar 29, 2017 at 2:23 AM, Masahiko Sawada > wrote: >> I was thinking that the status of this patch is still "Needs review" >> because I sent latest version patch[1]. > > I think you're

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread Robert Haas
On Wed, Mar 29, 2017 at 2:23 AM, Masahiko Sawada wrote: > I was thinking that the status of this patch is still "Needs review" > because I sent latest version patch[1]. I think you're right. I took a look at this today. I think there is some problem with the design of

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread David Steele
On 3/29/17 2:23 AM, Masahiko Sawada wrote: On Wed, Mar 29, 2017 at 12:23 AM, David Steele wrote: On 3/23/17 1:54 AM, Masahiko Sawada wrote: On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-29 Thread Masahiko Sawada
On Wed, Mar 29, 2017 at 12:23 AM, David Steele wrote: > On 3/23/17 1:54 AM, Masahiko Sawada wrote: >> >> On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: >>> >>> On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: We already

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-28 Thread David Steele
On 3/23/17 1:54 AM, Masahiko Sawada wrote: On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: We already have BTPageOpaqueData.btpo, a union whose contained type varies based on the page being dead. We

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-22 Thread Masahiko Sawada
On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: > On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: >> We already have BTPageOpaqueData.btpo, a union whose contained type >> varies based on the page being dead. We could just do the same with >> some

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-22 Thread Robert Haas
On Tue, Mar 21, 2017 at 8:18 PM, Peter Geoghegan wrote: >> Not only might that be unnecessary, but if we don't have a test >> demonstrating the problem, we also don't have a test demonstrating >> that a given approach fixes it. > > Preventing recycling from happening until we feel

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Peter Geoghegan
On Tue, Mar 21, 2017 at 12:15 PM, Robert Haas wrote: > Wouldn't it break on-disk compatibility with existing btree indexes? Yes, it would, but see my later remarks on pd_prune_xid. I think that that would be safe. > I think we're still trying to solve a problem that Simon

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Robert Haas
On Tue, Mar 14, 2017 at 6:10 PM, Peter Geoghegan wrote: > On Tue, Mar 14, 2017 at 2:48 PM, Peter Geoghegan wrote: >> I think that that's safe, but it is a little disappointing that it >> does not allow us to skip work in the case that you really had in mind >> when

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Masahiko Sawada
On Wed, Mar 22, 2017 at 2:29 AM, David Steele wrote: > Hi, > > On 3/15/17 9:50 AM, Amit Kapila wrote: >> >> >> What about if somebody does manual vacuum and there are no garbage >> tuples to clean, won't in that case also you want to avoid skipping >> the lazy_cleanup_index?

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread David Steele
Hi, On 3/15/17 9:50 AM, Amit Kapila wrote: What about if somebody does manual vacuum and there are no garbage tuples to clean, won't in that case also you want to avoid skipping the lazy_cleanup_index? Another option could be to skip updating the relfrozenxid if we have skipped the index

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Masahiko Sawada
On Wed, Mar 15, 2017 at 4:50 PM, Amit Kapila wrote: > On Thu, Mar 9, 2017 at 10:21 PM, Masahiko Sawada > wrote: >> On Wed, Mar 8, 2017 at 1:43 AM, Peter Geoghegan wrote: >>> On Sat, Mar 4, 2017 at 1:30 AM, Amit Kapila

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-15 Thread Amit Kapila
On Thu, Mar 9, 2017 at 10:21 PM, Masahiko Sawada wrote: > On Wed, Mar 8, 2017 at 1:43 AM, Peter Geoghegan wrote: >> On Sat, Mar 4, 2017 at 1:30 AM, Amit Kapila wrote: While I can't see this explained anywhere, I'm pretty

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-14 Thread Peter Geoghegan
On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: > We already have BTPageOpaqueData.btpo, a union whose contained type > varies based on the page being dead. We could just do the same with > some other field in that struct, and then store epoch there. Clearly > nobody really

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-14 Thread Peter Geoghegan
On Tue, Mar 14, 2017 at 2:48 PM, Peter Geoghegan wrote: > I think that that's safe, but it is a little disappointing that it > does not allow us to skip work in the case that you really had in mind > when writing the patch. Better than nothing, though, and perhaps still > a good

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-14 Thread Peter Geoghegan
On Thu, Mar 9, 2017 at 8:51 AM, Masahiko Sawada wrote: >> pg_class.relfrozenxid is InvalidTransactionId for indexes because >> indexes generally don't store XIDs. This is the one exception that I'm >> aware of, presumably justified by the fact that it's only for >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-09 Thread Masahiko Sawada
On Wed, Mar 8, 2017 at 1:43 AM, Peter Geoghegan wrote: > On Sat, Mar 4, 2017 at 1:30 AM, Amit Kapila wrote: >>> While I can't see this explained anywhere, I'm >>> pretty sure that that's supposed to be impossible, which this patch >>> changes. >>> >> >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-07 Thread Peter Geoghegan
On Sat, Mar 4, 2017 at 1:30 AM, Amit Kapila wrote: >> While I can't see this explained anywhere, I'm >> pretty sure that that's supposed to be impossible, which this patch >> changes. >> > > What makes you think that patch will allow pg_class.relfrozenxid to be > advanced

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-06 Thread Masahiko Sawada
On Sat, Mar 4, 2017 at 4:37 AM, Peter Geoghegan wrote: > On Fri, Mar 3, 2017 at 8:13 AM, Masahiko Sawada wrote: >> Thank you for clarification. Let me check my understanding. IIUC, >> skipping second index vacuum path (lazy_cleanup_index) can not be >> cause

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-04 Thread Amit Kapila
On Sat, Mar 4, 2017 at 8:55 AM, Peter Geoghegan wrote: > On Fri, Mar 3, 2017 at 6:58 PM, Amit Kapila wrote: >> You are right that we don't want the number of unclaimed-by-FSM >> recyclable pages to grow forever, but I think that won't happen with >> this

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Peter Geoghegan
On Fri, Mar 3, 2017 at 6:58 PM, Amit Kapila wrote: > You are right that we don't want the number of unclaimed-by-FSM > recyclable pages to grow forever, but I think that won't happen with > this patch. As soon as there are more deletions (in heap), in the > next vacuum

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Amit Kapila
On Sat, Mar 4, 2017 at 5:59 AM, Peter Geoghegan wrote: > On Fri, Mar 3, 2017 at 2:41 PM, Peter Geoghegan wrote: >> In other words, the number of B-Tree pages that the last VACUUM >> deleted, and thus made eligible to recycle by the next VACUUM has no >> relationship

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Peter Geoghegan
On Fri, Mar 3, 2017 at 2:41 PM, Peter Geoghegan wrote: > In other words, the number of B-Tree pages that the last VACUUM > deleted, and thus made eligible to recycle by the next VACUUM has no > relationship with the number of pages the next VACUUM will itself end > up deleting, in

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Peter Geoghegan
On Fri, Mar 3, 2017 at 11:49 AM, Peter Geoghegan wrote: >> Please verify my understanding of your thought process: We don't have >> to freeze indexes at all, ever, so if we see index bloat as a separate >> problem, we also see that there is no need to *link* index needs to >> the

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Peter Geoghegan
On Fri, Mar 3, 2017 at 11:37 AM, Peter Geoghegan wrote: > Please verify my understanding of your thought process: We don't have > to freeze indexes at all, ever, so if we see index bloat as a separate > problem, we also see that there is no need to *link* index needs to > the need

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Peter Geoghegan
On Fri, Mar 3, 2017 at 8:13 AM, Masahiko Sawada wrote: > Thank you for clarification. Let me check my understanding. IIUC, > skipping second index vacuum path (lazy_cleanup_index) can not be > cause of leaving page as half-dead state but could leave recyclable > pages that

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Masahiko Sawada
On Fri, Mar 3, 2017 at 10:45 PM, David Steele wrote: > On 2/27/17 12:46 PM, Peter Geoghegan wrote: >> On Sat, Feb 25, 2017 at 10:51 PM, Robert Haas wrote: >> >>> Do you have an idea about that, or any ideas for experiments we could try? >> >> Nothing

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread Masahiko Sawada
On Sat, Feb 25, 2017 at 7:10 AM, Peter Geoghegan wrote: > On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: >> I think this thread is pretty short on evidence that would let us make >> a smart decision about what to do here. I see three possibilities. >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-03 Thread David Steele
On 2/27/17 12:46 PM, Peter Geoghegan wrote: > On Sat, Feb 25, 2017 at 10:51 PM, Robert Haas wrote: > >> Do you have an idea about that, or any ideas for experiments we could try? > > Nothing occurs to me right now, unfortunately. However, my general > sense is that it

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-27 Thread Peter Geoghegan
On Sat, Feb 25, 2017 at 10:51 PM, Robert Haas wrote: > The thing that strikes me based on what you wrote is that our page > recycling seems to admit of long delays even as things stand. There's > no bound on how much time could pass between one index vacuum and the > next,

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-25 Thread Robert Haas
On Sat, Feb 25, 2017 at 3:40 AM, Peter Geoghegan wrote: > On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: >> I think this thread is pretty short on evidence that would let us make >> a smart decision about what to do here. I see three possibilities. >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Amit Kapila
On Sat, Feb 25, 2017 at 3:40 AM, Peter Geoghegan wrote: > On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: >> I think this thread is pretty short on evidence that would let us make >> a smart decision about what to do here. I see three possibilities. >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Peter Geoghegan
On Fri, Feb 24, 2017 at 9:26 AM, Robert Haas wrote: > I think this thread is pretty short on evidence that would let us make > a smart decision about what to do here. I see three possibilities. > The first is that this patch is a good idea whether we do something > about

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Jim Nasby
On 2/24/17 11:26 AM, Robert Haas wrote: I think we need to come up with some set of tests to figure out what actually works well in practice here. Theories are a good starting point, but good vacuum behavior is really important, and a patch that changes it ought to be backed up by at least some

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-24 Thread Robert Haas
On Fri, Feb 24, 2017 at 8:49 AM, Amit Kapila wrote: >> IIUC, I think that we need to have the number of half-dead pages in meta >> page. > > Don't you think we need to consider backward compatibility if we want > to do that? Yeah, that would be an on-disk format break.

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-23 Thread Amit Kapila
On Thu, Feb 23, 2017 at 5:15 PM, Masahiko Sawada wrote: > On Wed, Feb 22, 2017 at 12:01 AM, Amit Kapila wrote: > >> I understand that there could be some delay in reclaiming dead pages >> but do you think it is such a big deal that we completely

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-23 Thread Masahiko Sawada
On Wed, Feb 22, 2017 at 12:01 AM, Amit Kapila wrote: > On Tue, Feb 21, 2017 at 1:09 AM, Simon Riggs wrote: >> On 20 February 2017 at 10:27, Amit Kapila wrote: >>> On Mon, Feb 20, 2017 at 3:01 PM, Simon Riggs

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-21 Thread Amit Kapila
On Tue, Feb 21, 2017 at 1:09 AM, Simon Riggs wrote: > On 20 February 2017 at 10:27, Amit Kapila wrote: >> On Mon, Feb 20, 2017 at 3:01 PM, Simon Riggs wrote: >>> On 20 February 2017 at 09:15, Amit Kapila

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Peter Geoghegan
On Thu, Feb 16, 2017 at 10:41 AM, Robert Haas wrote: >> 2. The current btree vacuum code requires 2 vacuums to fully reuse >> half-dead pages. So skipping an index vacuum might mean that second >> index scan never happens at all, which would be bad. > > Maybe. If there are

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Simon Riggs
On 20 February 2017 at 10:27, Amit Kapila wrote: > On Mon, Feb 20, 2017 at 3:01 PM, Simon Riggs wrote: >> On 20 February 2017 at 09:15, Amit Kapila wrote: >>> On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Masahiko Sawada
On Mon, Feb 20, 2017 at 6:15 PM, Amit Kapila wrote: > On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada > wrote: >> On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: >>> On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Amit Kapila
On Mon, Feb 20, 2017 at 3:01 PM, Simon Riggs wrote: > On 20 February 2017 at 09:15, Amit Kapila wrote: >> On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada >> wrote: >>> On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Simon Riggs
On 20 February 2017 at 09:15, Amit Kapila wrote: > On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada > wrote: >> On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: >>> On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Amit Kapila
On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada wrote: > On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: >> On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs wrote: >>> On 15 February 2017 at 08:07, Masahiko Sawada

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Masahiko Sawada
On Mon, Feb 20, 2017 at 11:35 AM, Jim Nasby wrote: > On 2/19/17 7:56 PM, Masahiko Sawada wrote: >> >> The half-dead pages are never cleaned up if the ratio of pages >> containing garbage is always lower than threshold. Also in gin index >> the pending list is never

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Jim Nasby
On 2/19/17 7:56 PM, Masahiko Sawada wrote: The half-dead pages are never cleaned up if the ratio of pages containing garbage is always lower than threshold. Also in gin index the pending list is never cleared, which become big problem. I guess that we should take action for each type of indexes.

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Masahiko Sawada
On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: > On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs wrote: >> On 15 February 2017 at 08:07, Masahiko Sawada wrote: >>> It's a bug. Attached latest version patch, which passed make

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-16 Thread Robert Haas
On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs wrote: > On 15 February 2017 at 08:07, Masahiko Sawada wrote: >> It's a bug. Attached latest version patch, which passed make check. > > In its current form, I'm not sure this is a good idea. Problems... >

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-16 Thread Simon Riggs
On 15 February 2017 at 08:07, Masahiko Sawada wrote: > > It's a bug. Attached latest version patch, which passed make check. In its current form, I'm not sure this is a good idea. Problems... 1. I'm pretty sure the world doesn't need another VACUUM parameter I suggest

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-16 Thread Kuntal Ghosh
On Mon, Feb 13, 2017 at 1:01 PM, Masahiko Sawada wrote: Thanks for the explanation. I've looked into the referred code. I'm still in doubt. vacuumed_pages is incremented only when there are no indexes, i.e. nindexes=0. Now, look at the following part in the patch. +

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-15 Thread Masahiko Sawada
On Wed, Feb 15, 2017 at 4:39 PM, Ideriha, Takeshi wrote: > Hi, I tried regression test and found some errors concerning brin and gin, > though I didn't look into this. > > Here's a log: > > *** /home/ideriha/postgres-master/src/test/regress/expected/brin.out >

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-14 Thread Ideriha, Takeshi
Hi, I tried regression test and found some errors concerning brin and gin, though I didn't look into this. Here's a log: *** /home/ideriha/postgres-master/src/test/regress/expected/brin.out 2017-02-13 11:33:43.270942937 +0900 ---

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-12 Thread Masahiko Sawada
On Fri, Feb 10, 2017 at 8:01 PM, Kuntal Ghosh wrote: > On Wed, Jan 4, 2017 at 1:51 PM, Masahiko Sawada wrote: > >> Attached patch introduces new GUC parameter parameter >> vacuum_cleanup_index_scale_factor which specifies the fraction of the >>

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-10 Thread Kuntal Ghosh
On Wed, Jan 4, 2017 at 1:51 PM, Masahiko Sawada wrote: > Attached patch introduces new GUC parameter parameter > vacuum_cleanup_index_scale_factor which specifies the fraction of the > table pages containing dead tuple needed to trigger a cleaning up > indexes. The default

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-01-05 Thread Robert Haas
On Wed, Jan 4, 2017 at 3:21 AM, Masahiko Sawada wrote: > Hi and happy new year. > > The lazy vacuum calls lazy_cleanup_index to update statistics of > indexes on a table such as relpages, reltuples at the end of the > lazy_scan_heap. In all type of indexes the

[HACKERS] GUC for cleanup indexes threshold.

2017-01-04 Thread Masahiko Sawada
Hi and happy new year. The lazy vacuum calls lazy_cleanup_index to update statistics of indexes on a table such as relpages, reltuples at the end of the lazy_scan_heap. In all type of indexes the lazy_cleanup_index scans all index pages. It happens even if table has not been updated at all since