Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Lew
On 02/23/2012 07:28 AM, k...@rice.edu wrote: On Thu, Feb 23, 2012 at 05:25:46PM +0200, Reuven M. Lerner wrote: What is the distribution of end_dates? It might be worth running this in several steps, deleting records older than, say, 90 days, 60 days, 30 days. I've suggested something

Re: [PERFORM] set autovacuum=off

2012-02-24 Thread Peter van Hardenberg
On Thu, Feb 23, 2012 at 1:07 PM, Alessandro Gagliardi alessan...@path.com wrote: ...Apparently the last four columns don't exist in my database. As for the first four, that is somewhat illuminating Then you are not running a current version of PostgreSQL so the first step to performance

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Samuel Gendler
On Thu, Feb 23, 2012 at 10:39 PM, Reuven M. Lerner reu...@lerner.co.ilwrote: Hi, everyone. So it turns out that we're not using 25 GB of virtual memory. (That's what I had been shown yesterday, and it was a bit surprising, to say the least...) A few statistics that I managed to get from

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Reuven M. Lerner
Hi, everyone. Samuel wrote: What is work_mem set to? If all the other values were set so low, I'd expect work_mem to also be small, which could be causing all kind of disk activity when steps don't fit into a work_mem segment. I just checked, and work_mem is set to 30 MB. That seems a bit

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Claudio Freire
On Fri, Feb 24, 2012 at 9:37 AM, Reuven M. Lerner reu...@lerner.co.il wrote: I just checked, and work_mem is set to 30 MB.  That seems a bit low to me, given the size of the database and the fact that we're doing so much sorting and subselecting.  Am I right that we should push that up some

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Shaun Thomas
On 02/24/2012 12:39 AM, Reuven M. Lerner wrote: - CPU is virtually idle when running the deletes, using about 1% of CPU I think you found your problem. See if you can get the Windows admins to give you some info on how busy the disks are (percent utilization, IOPS, something) the next time

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Reuven M. Lerner
Hi, everyone. Shaun said: On 02/24/2012 12:39 AM, Reuven M. Lerner wrote: - CPU is virtually idle when running the deletes, using about 1% of CPU I think you found your problem. See if you can get the Windows admins to give you some info on how busy the disks are (percent utilization,

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Shaun Thomas
On 02/24/2012 08:54 AM, Reuven M. Lerner wrote: I/O writes: process1: 820,000, process2: 1Milion Process3: 33,000 That's not especially helpful, unfortunately. That doesn't really tell us how saturated the controller is. However I suspect it's being effectively slammed based simply on your

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-24 Thread Steve Crawford
On 02/23/2012 12:39 AM, Reuven M. Lerner wrote: Hi, everyone. ... ...at one installation, we're quickly running out of disk space. The database is already taking up about 200 GB of space, and is growing by 1 GB or so a day. I've been following the discussion of approaches and tuning for