Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-03 Thread Alvaro Herrera
Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: I am rather surprised that nobody has reported this problem before. I am now of the mind that this is clearly a bug that should be fixed all the way back. I'm coming around to that also, however, should we worry

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 9:54 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: So in essence what we're going to do is that the balance mechanism considers only tables that don't have per-table configuration options; for those that do, we will use the values configured

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Thu, Oct 2, 2014 at 9:54 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: So in essence what we're going to do is that the balance mechanism considers only tables that don't have per-table configuration options; for

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Alvaro Herrera
Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: I agree with both of those arguments. I have run into very few customers who have used the autovacuum settings to customize behavior for particular tables, and anyone who hasn't should see no change (right?), so my guess

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Alvaro Herrera wrote: Basically, if you are on 9.3.5 or earlier any per-table options for autovacuum cost delay will misbehave (meaning: any such table will be processed with settings flattened according to balancing of the standard

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-01 Thread Robert Haas
On Tue, Sep 30, 2014 at 6:16 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: I favor option (a). There's something to be said for your proposal in terms of logical consistency with what we have now, but to be honest I'm not sure it's the behavior anyone wants (I would

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-09-30 Thread Alvaro Herrera
Robert Haas wrote: I favor option (a). There's something to be said for your proposal in terms of logical consistency with what we have now, but to be honest I'm not sure it's the behavior anyone wants (I would welcome more feedback on what people actually want). I think we should view an

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-09-21 Thread Gregory Smith
On 8/28/14, 12:18 PM, Robert Haas wrote: At least in situations that I've encountered, it's typical to be able to determine the frequency with which a given table needs to be vacuumed to avoid runaway bloat, and from that you can work backwards to figure out how fast you must process it in

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-30 Thread Amit Kapila
On Tue, Aug 26, 2014 at 9:49 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So my proposal is a bit more complicated. First we introduce the notion of a single number, to enable sorting and computations: the delay equivalent, which is the cost_limit divided by cost_delay. The highest

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-30 Thread Amit Kapila
On Thu, Aug 28, 2014 at 11:06 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: Now, in the case where you are setting an overall limit, there is at least an argument to be made that you can determine the overall rate of autovacuum-induced I/O activity that the system

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-29 Thread Alvaro Herrera
Mark Kirkwood wrote: On 29/08/14 08:56, Alvaro Herrera wrote: Robert Haas wrote: I agree that you might not like that. But you might not like having the table vacuumed slower than the configured rate, either. My impression is that the time between vacuums isn't really all that negotiable

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-28 Thread Robert Haas
On Tue, Aug 26, 2014 at 12:19 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: On Mon, May 5, 2014 at 11:57 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: I could think of 2 ways to change this: a. if user has specified cost_limit value for table, then it just uses it rather

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-28 Thread Alvaro Herrera
Robert Haas wrote: Now, in the case where you are setting an overall limit, there is at least an argument to be made that you can determine the overall rate of autovacuum-induced I/O activity that the system can tolerate, and set your limits to stay within that budget, and then let the system

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-28 Thread Robert Haas
On Thu, Aug 28, 2014 at 1:36 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: Now, in the case where you are setting an overall limit, there is at least an argument to be made that you can determine the overall rate of autovacuum-induced I/O activity that the system can

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-28 Thread Alvaro Herrera
Robert Haas wrote: I agree that you might not like that. But you might not like having the table vacuumed slower than the configured rate, either. My impression is that the time between vacuums isn't really all that negotiable for some people. I had one customer who had horrible bloat

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-28 Thread Robert Haas
On Thu, Aug 28, 2014 at 4:56 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: I agree that you might not like that. But you might not like having the table vacuumed slower than the configured rate, either. My impression is that the time between vacuums isn't really all

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-28 Thread Mark Kirkwood
On 29/08/14 08:56, Alvaro Herrera wrote: Robert Haas wrote: I agree that you might not like that. But you might not like having the table vacuumed slower than the configured rate, either. My impression is that the time between vacuums isn't really all that negotiable for some people. I had

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-26 Thread Alvaro Herrera
Mark Kirkwood wrote: On 06/05/14 16:28, Amit Kapila wrote: On Mon, May 5, 2014 at 11:57 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: I could think of 2 ways to change this: a. if user has specified cost_limit value for table, then it just uses it rather than rebalancing

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-26 Thread Mark Kirkwood
On 27/08/14 10:27, Alvaro Herrera wrote: Alvaro Herrera wrote: So my proposal is a bit more complicated. First we introduce the notion of a single number, to enable sorting and computations: the delay equivalent, which is the cost_limit divided by cost_delay. Here's a patch that implements

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-26 Thread Haribabu Kommi
On Wed, Aug 27, 2014 at 8:27 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: So my proposal is a bit more complicated. First we introduce the notion of a single number, to enable sorting and computations: the delay equivalent, which is the cost_limit divided by

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-08-26 Thread Mark Kirkwood
On 27/08/14 10:27, Alvaro Herrera wrote: Alvaro Herrera wrote: So my proposal is a bit more complicated. First we introduce the notion of a single number, to enable sorting and computations: the delay equivalent, which is the cost_limit divided by cost_delay. Here's a patch that implements

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-05-05 Thread Mark Kirkwood
On 05/05/14 15:22, Amit Kapila wrote: Here what I could understand is that sum of cost_limit for all autovacuum workers should never exceed the value of autovacuum_vacuum_cost_limit which seems to be always the case in current code but same is not true for proposed patch. Right, but have a

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-05-05 Thread Amit Kapila
On Mon, May 5, 2014 at 11:57 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: On 05/05/14 15:22, Amit Kapila wrote: Right, but have a look at the 1st message in this thread - the current behavior (and to a large extent the above condition) means that setting cost limits per table does

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-05-05 Thread Mark Kirkwood
On 06/05/14 16:28, Amit Kapila wrote: On Mon, May 5, 2014 at 11:57 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: On 05/05/14 15:22, Amit Kapila wrote: Right, but have a look at the 1st message in this thread - the current behavior (and to a large extent the above condition) means that

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-05-04 Thread Amit Kapila
On Mon, Feb 17, 2014 at 7:38 AM, Haribabu Kommi kommi.harib...@gmail.com wrote: I modified the autovac_balance_cost function to balance the costs using the number of running workers, instead of default vacuum cost parameters. Lets assume there are 4 workers running currently with default cost

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-05-04 Thread Haribabu Kommi
On Mon, May 5, 2014 at 1:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Feb 17, 2014 at 7:38 AM, Haribabu Kommi kommi.harib...@gmail.com wrote: I modified the autovac_balance_cost function to balance the costs using the number of running workers, instead of default vacuum cost

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-05-04 Thread Amit Kapila
On Mon, May 5, 2014 at 6:35 AM, Haribabu Kommi kommi.harib...@gmail.com wrote: On Mon, May 5, 2014 at 1:09 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Feb 17, 2014 at 7:38 AM, Haribabu Kommi kommi.harib...@gmail.com wrote: I modified the autovac_balance_cost function to balance the

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-04-09 Thread Alvaro Herrera
Haribabu Kommi wrote: I modified the autovac_balance_cost function to balance the costs using the number of running workers, instead of default vacuum cost parameters. Just as a heads-up, this patch wasn't part of the commitfest, but I intend to review it and possibly commit for 9.4. Not

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-02-14 Thread Alvaro Herrera
Haribabu Kommi escribió: I changed the balance cost calculations a little bit to give priority to the user provided per table autovacuum parameters. If any user specified per table vacuum parameters exists and those are different with guc vacuum parameters then the balance cost calculations

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-02-14 Thread Haribabu Kommi
On Feb 15, 2014 9:19 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Haribabu Kommi escribió: I changed the balance cost calculations a little bit to give priority to the user provided per table autovacuum parameters. If any user specified per table vacuum parameters exists and those

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-02-13 Thread Alvaro Herrera
I hadn't noticed this thread. I will give this a look. Thanks for providing a patch. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to