Re: [HACKERS] autovacuum_work_mem

2013-12-16 Thread Heikki Linnakangas
On 12/13/2013 08:40 PM, Alvaro Herrera wrote: Heikki Linnakangas escribió: I haven't been following this thread in detail, but would it help if we implemented a scheme to reduce (auto)vacuum's memory usage? Such schemes have been discussed in the past, packing the list of dead items more

Re: [HACKERS] autovacuum_work_mem

2013-12-16 Thread Simon Riggs
On 16 December 2013 10:12, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/13/2013 08:40 PM, Alvaro Herrera wrote: Heikki Linnakangas escribió: I haven't been following this thread in detail, but would it help if we implemented a scheme to reduce (auto)vacuum's memory usage? Such

Re: [HACKERS] autovacuum_work_mem

2013-12-13 Thread Bruce Momjian
On Wed, Dec 11, 2013 at 10:35:32AM -0800, Josh Berkus wrote: On 12/11/2013 09:57 AM, Robert Haas wrote: I don't agree with that assessment. Anything that involves changing the scheduling of autovacuum is a major project that will legitimately provoke much controversy. Extensive testing

Re: [HACKERS] autovacuum_work_mem

2013-12-13 Thread Heikki Linnakangas
On 12/13/2013 08:24 PM, Bruce Momjian wrote: On Wed, Dec 11, 2013 at 10:35:32AM -0800, Josh Berkus wrote: On 12/11/2013 09:57 AM, Robert Haas wrote: I don't agree with that assessment. Anything that involves changing the scheduling of autovacuum is a major project that will legitimately

Re: [HACKERS] autovacuum_work_mem

2013-12-13 Thread Alvaro Herrera
Heikki Linnakangas escribió: I haven't been following this thread in detail, but would it help if we implemented a scheme to reduce (auto)vacuum's memory usage? Such schemes have been discussed in the past, packing the list of dead items more tightly. Well, it would help some, but it

Re: [HACKERS] autovacuum_work_mem

2013-12-12 Thread Simon Riggs
On 11 December 2013 22:23, Peter Geoghegan p...@heroku.com wrote: On Wed, Dec 11, 2013 at 1:28 PM, Simon Riggs si...@2ndquadrant.com wrote: But nobody has given a sensible answer to my questions, other than to roll out the same general points again. In practice, its a knob that does not do

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Sun, Oct 20, 2013 at 7:21 AM, Magnus Hagander mag...@hagander.net wrote: It seemed neater to me to create a new flag, so that in principle any vacuum() code path can request autovacuum_work_mem, rather than having lazyvacuum.c code call IsAutoVacuumWorkerProcess() for the same purpose. To

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 25 November 2013 21:51, Peter Geoghegan p...@heroku.com wrote: On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead tuples, limiting their numbers. In what circumstances will the memory usage

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 9:43 AM, Simon Riggs si...@2ndquadrant.com wrote: On 25 November 2013 21:51, Peter Geoghegan p...@heroku.com wrote: On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 14:50, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 11, 2013 at 9:43 AM, Simon Riggs si...@2ndquadrant.com wrote: On 25 November 2013 21:51, Peter Geoghegan p...@heroku.com wrote: On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 10:41 AM, Simon Riggs si...@2ndquadrant.com wrote: It looks fairly easy to estimate the memory needed for an auto vacuum, since we know the scale factor and the tuple estimate. We can then use the memory estimate to alter the scheduling of work. And/or we can use actual

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 09:57 AM, Robert Haas wrote: I don't agree with that assessment. Anything that involves changing the scheduling of autovacuum is a major project that will legitimately provoke much controversy. Extensive testing will be needed to prove that the new algorithm doesn't perform

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Peter Geoghegan
On Wed, Dec 11, 2013 at 7:41 AM, Simon Riggs si...@2ndquadrant.com wrote: That's about 2-3 days work and I know Peter can hack it. So the situation is not perfection-sought-blocking-good, this is more like fairly poor solution being driven through when a better solution is available within the

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in any important cases. Agreed, but the amount of testing seems equivalent in both cases, assuming we weren't

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 2:37 PM, Simon Riggs si...@2ndquadrant.com wrote: On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in any important cases. Agreed, but

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 11:37 AM, Simon Riggs wrote: On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in any important cases. Agreed, but the amount of testing seems

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: And, for that matter, accepting this patch by no means blocks doing something more sophisticated in the future. Yeah. I think the only real argument against it is do we really need yet another knob?. Since Josh, who's usually the voicer of that argument,

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 12:40 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: And, for that matter, accepting this patch by no means blocks doing something more sophisticated in the future. Yeah. I think the only real argument against it is do we really need yet another knob?. Since Josh,

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 19:54, Josh Berkus j...@agliodbs.com wrote: On 12/11/2013 11:37 AM, Simon Riggs wrote: On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Peter Geoghegan
On Wed, Dec 11, 2013 at 1:28 PM, Simon Riggs si...@2ndquadrant.com wrote: But nobody has given a sensible answer to my questions, other than to roll out the same general points again. In practice, its a knob that does not do very much of use for us. At best it is addressing the symptoms, not

Re: [HACKERS] autovacuum_work_mem

2013-11-25 Thread Peter Geoghegan
On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead tuples, limiting their numbers. In what circumstances will the memory usage from multiple concurrent VACUUMs become a problem? In those

Re: [HACKERS] autovacuum_work_mem

2013-11-24 Thread Simon Riggs
On 19 October 2013 19:22, Peter Geoghegan p...@heroku.com wrote: I won't repeat the rationale for the patch here. I can't see the problem that this patch is trying to solve. I'm having trouble understanding when I would use this. VACUUM uses 6 bytes per dead tuple. And autovacuum regularly

Re: [HACKERS] autovacuum_work_mem

2013-11-15 Thread Peter Geoghegan
On Mon, Oct 21, 2013 at 6:44 AM, Magnus Hagander mag...@hagander.net wrote: +1. If changing at all, then maybe just autovacuum_mem? I would like to stick with autovacuum_work_mem - it reflects the fact that it's a setting shadowed by maintenance_work_mem, without being too verbose. -- Peter

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Peter Eisentraut
On 10/19/13 8:22 PM, Peter Geoghegan wrote: I don't think it's a problem that autovacuum_work_mem is kind of similar to vacuum_mem in name. maintenance_work_mem was last spelt vacuum_mem about 10 years ago. Enough time has passed that I think it very unlikely that someone might conflate the

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Robert Haas
On Mon, Oct 21, 2013 at 9:36 AM, Peter Eisentraut pete...@gmx.net wrote: On 10/19/13 8:22 PM, Peter Geoghegan wrote: I don't think it's a problem that autovacuum_work_mem is kind of similar to vacuum_mem in name. maintenance_work_mem was last spelt vacuum_mem about 10 years ago. Enough time

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Magnus Hagander
On Mon, Oct 21, 2013 at 3:42 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Oct 21, 2013 at 9:36 AM, Peter Eisentraut pete...@gmx.net wrote: On 10/19/13 8:22 PM, Peter Geoghegan wrote: I don't think it's a problem that autovacuum_work_mem is kind of similar to vacuum_mem in name.

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: +1. If changing at all, then maybe just autovacuum_mem? It's not like there's a different parameter to control a different kind of memory in autovac, is there? +1 on that, for my 2c. Thanks, Stephen signature.asc

Re: [HACKERS] autovacuum_work_mem

2013-10-20 Thread Magnus Hagander
On Sun, Oct 20, 2013 at 2:22 AM, Peter Geoghegan p...@heroku.com wrote: There has recently been considerable discussion around auto-tuning. Throughout the course of this discussion, I raised the idea of creating a new GUC to separately control autovacuum's usage of maintenance_work_mem [1],