Re: [HACKERS] autovacuum_work_mem

2013-12-16 Thread Simon Riggs
On 16 December 2013 10:12, Heikki Linnakangas wrote: > On 12/13/2013 08:40 PM, Alvaro Herrera wrote: >> >> Heikki Linnakangas escribió: >> >>> I haven't been following this thread in detail, but would it help if >>> we implemented a scheme to reduce (auto)vacuum's memory usage? Such >>> schemes ha

Re: [HACKERS] autovacuum_work_mem

2013-12-16 Thread Heikki Linnakangas
On 12/13/2013 08:40 PM, Alvaro Herrera wrote: Heikki Linnakangas escribió: I haven't been following this thread in detail, but would it help if we implemented a scheme to reduce (auto)vacuum's memory usage? Such schemes have been discussed in the past, packing the list of dead items more tightl

Re: [HACKERS] autovacuum_work_mem

2013-12-13 Thread Alvaro Herrera
Heikki Linnakangas escribió: > I haven't been following this thread in detail, but would it help if > we implemented a scheme to reduce (auto)vacuum's memory usage? Such > schemes have been discussed in the past, packing the list of dead > items more tightly. Well, it would help some, but it woul

Re: [HACKERS] autovacuum_work_mem

2013-12-13 Thread Heikki Linnakangas
On 12/13/2013 08:24 PM, Bruce Momjian wrote: On Wed, Dec 11, 2013 at 10:35:32AM -0800, Josh Berkus wrote: On 12/11/2013 09:57 AM, Robert Haas wrote: I don't agree with that assessment. Anything that involves changing the scheduling of autovacuum is a major project that will legitimately provok

Re: [HACKERS] autovacuum_work_mem

2013-12-13 Thread Bruce Momjian
On Wed, Dec 11, 2013 at 10:35:32AM -0800, Josh Berkus wrote: > On 12/11/2013 09:57 AM, Robert Haas wrote: > > I don't agree with that assessment. Anything that involves changing > > the scheduling of autovacuum is a major project that will legitimately > > provoke much controversy. Extensive test

Re: [HACKERS] autovacuum_work_mem

2013-12-12 Thread Simon Riggs
On 11 December 2013 22:23, Peter Geoghegan wrote: > On Wed, Dec 11, 2013 at 1:28 PM, Simon Riggs wrote: >> But nobody has given a sensible answer to my questions, other than to >> roll out the same general points again. In practice, its a knob that >> does not do very much of use for us. At best

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Peter Geoghegan
On Wed, Dec 11, 2013 at 1:28 PM, Simon Riggs wrote: > But nobody has given a sensible answer to my questions, other than to > roll out the same general points again. In practice, its a knob that > does not do very much of use for us. At best it is addressing the > symptoms, not the cause. IMHO. I

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 19:54, Josh Berkus wrote: > On 12/11/2013 11:37 AM, Simon Riggs wrote:> On 11 December 2013 17:57, > Robert Haas wrote: >> >>> Extensive testing will be needed to prove >>> that the new algorithm doesn't perform worse than the current >>> algorithm in any important cases. >> >

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 12:40 PM, Tom Lane wrote: > Josh Berkus writes: >> And, for that matter, accepting this patch by no means blocks doing >> something more sophisticated in the future. > > Yeah. I think the only real argument against it is "do we really need > yet another knob?". Since Josh, who's u

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Tom Lane
Josh Berkus writes: > And, for that matter, accepting this patch by no means blocks doing > something more sophisticated in the future. Yeah. I think the only real argument against it is "do we really need yet another knob?". Since Josh, who's usually the voicer of that argument, is for this on

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 11:37 AM, Simon Riggs wrote:> On 11 December 2013 17:57, Robert Haas wrote: > >> Extensive testing will be needed to prove >> that the new algorithm doesn't perform worse than the current >> algorithm in any important cases. > > Agreed, but the amount of testing seems equivalent in b

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 2:37 PM, Simon Riggs wrote: > On 11 December 2013 17:57, Robert Haas wrote: >> Extensive testing will be needed to prove >> that the new algorithm doesn't perform worse than the current >> algorithm in any important cases. > > Agreed, but the amount of testing seems equiva

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 17:57, Robert Haas wrote: > Extensive testing will be needed to prove > that the new algorithm doesn't perform worse than the current > algorithm in any important cases. Agreed, but the amount of testing seems equivalent in both cases, assuming we weren't going to skip it for

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Peter Geoghegan
On Wed, Dec 11, 2013 at 7:41 AM, Simon Riggs wrote: > That's about 2-3 days work and I know Peter can hack it. So the > situation is not perfection-sought-blocking-good, this is more like > fairly poor solution being driven through when a better solution is > available within the time and skills a

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 09:57 AM, Robert Haas wrote: > I don't agree with that assessment. Anything that involves changing > the scheduling of autovacuum is a major project that will legitimately > provoke much controversy. Extensive testing will be needed to prove > that the new algorithm doesn't perform

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 10:41 AM, Simon Riggs wrote: > It looks fairly easy to estimate the memory needed for an auto vacuum, > since we know the scale factor and the tuple estimate. We can then use > the memory estimate to alter the scheduling of work. And/or we can use > actual memory usage and

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 14:50, Robert Haas wrote: > On Wed, Dec 11, 2013 at 9:43 AM, Simon Riggs wrote: >> On 25 November 2013 21:51, Peter Geoghegan wrote: >>> On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs wrote: VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead tu

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 9:43 AM, Simon Riggs wrote: > On 25 November 2013 21:51, Peter Geoghegan wrote: >> On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs wrote: >>> VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes >>> dead tuples, limiting their numbers. >>> >>> In what circums

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 25 November 2013 21:51, Peter Geoghegan wrote: > On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs wrote: >> VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes >> dead tuples, limiting their numbers. >> >> In what circumstances will the memory usage from multiple concurrent >> VAC

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Sun, Oct 20, 2013 at 7:21 AM, Magnus Hagander wrote: >> It seemed neater to me to create a new flag, so that in principle any >> vacuum() code path can request autovacuum_work_mem, rather than having >> lazyvacuum.c code call IsAutoVacuumWorkerProcess() for the same >> purpose. To date, that's

Re: [HACKERS] autovacuum_work_mem

2013-11-25 Thread Peter Geoghegan
On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs wrote: > VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes > dead tuples, limiting their numbers. > > In what circumstances will the memory usage from multiple concurrent > VACUUMs become a problem? In those circumstances, reducing >

Re: [HACKERS] autovacuum_work_mem

2013-11-24 Thread Simon Riggs
On 19 October 2013 19:22, Peter Geoghegan wrote: > I won't repeat the rationale for the patch here. I can't see the problem that this patch is trying to solve. I'm having trouble understanding when I would use this. VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead tuple

Re: [HACKERS] autovacuum_work_mem

2013-11-15 Thread Peter Geoghegan
On Mon, Oct 21, 2013 at 6:44 AM, Magnus Hagander wrote: > +1. If changing at all, then maybe just "autovacuum_mem"? I would like to stick with autovacuum_work_mem - it reflects the fact that it's a setting shadowed by maintenance_work_mem, without being too verbose. -- Peter Geoghegan -- Sen

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > +1. If changing at all, then maybe just "autovacuum_mem"? It's not > like there's a different parameter to control a different kind of > memory in autovac, is there? +1 on that, for my 2c. Thanks, Stephen signature.asc De

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Magnus Hagander
On Mon, Oct 21, 2013 at 3:42 PM, Robert Haas wrote: > On Mon, Oct 21, 2013 at 9:36 AM, Peter Eisentraut wrote: >> On 10/19/13 8:22 PM, Peter Geoghegan wrote: >>> I don't think it's a problem that >>> autovacuum_work_mem is kind of similar to vacuum_mem in name. >>> maintenance_work_mem was last s

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Robert Haas
On Mon, Oct 21, 2013 at 9:36 AM, Peter Eisentraut wrote: > On 10/19/13 8:22 PM, Peter Geoghegan wrote: >> I don't think it's a problem that >> autovacuum_work_mem is kind of similar to vacuum_mem in name. >> maintenance_work_mem was last spelt vacuum_mem about 10 years ago. >> Enough time has pass

Re: [HACKERS] autovacuum_work_mem

2013-10-21 Thread Peter Eisentraut
On 10/19/13 8:22 PM, Peter Geoghegan wrote: > I don't think it's a problem that > autovacuum_work_mem is kind of similar to vacuum_mem in name. > maintenance_work_mem was last spelt vacuum_mem about 10 years ago. > Enough time has passed that I think it very unlikely that someone > might conflate t

Re: [HACKERS] autovacuum_work_mem

2013-10-20 Thread Magnus Hagander
On Sun, Oct 20, 2013 at 2:22 AM, Peter Geoghegan wrote: > There has recently been considerable discussion around auto-tuning. > Throughout the course of this discussion, I raised the idea of > creating a new GUC to separately control autovacuum's usage of > maintenance_work_mem [1], explaining the

[HACKERS] autovacuum_work_mem

2013-10-19 Thread Peter Geoghegan
There has recently been considerable discussion around auto-tuning. Throughout the course of this discussion, I raised the idea of creating a new GUC to separately control autovacuum's usage of maintenance_work_mem [1], explaining the rationale in some detail [2]. At the time Magnus seemed to think