Re: [PERFORM] Vacuums on large busy databases

2006-09-18 Thread Francisco Reyes
Jim C. Nasby writes: BTW, on some good raid controllers (with battery backup and write-caching), putting pg_xlog on a seperate partition doesn't really help, so you might want to try combining everything. Planning to put a busy database on second raid or perhaps some index files. So far the se

Re: [PERFORM] Vacuums on large busy databases

2006-09-18 Thread Jim C. Nasby
On Thu, Sep 14, 2006 at 11:23:01AM -0400, Francisco Reyes wrote: > My setup: > Freebsd 6.1 > Postgresql 8.1.4 > Memory: 8GB > SATA Disks > > Raid 1 10 spindles (2 as hot spares) > 500GB disks (16MB buffer), 7200 rpm > Raid 10 > > Raid 2 4 spindles > 150GB 10K rpm disks > Raid 10 > > shared_buff

Re: [PERFORM] Vacuums on large busy databases

2006-09-15 Thread Markus Schaber
Hi, Francisco, Francisco Reyes wrote: > I am looking to either improve the time of the vacuum or decrease it's > impact on the loads. > Are the variables: > #vacuum_cost_delay = 0 # 0-1000 milliseconds > #vacuum_cost_page_hit = 1 # 0-1 credits > #vacuum_cost_pag

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Jeff Davis
On Thu, 2006-09-14 at 21:04 -0400, Michael Stone wrote: > On Thu, Sep 14, 2006 at 05:52:02PM -0700, Jeff Davis wrote: > >Any long-running system will have very little "free" memory. Free memory > >is wasted memory, so the OS finds some use for it. > > The important part of the output of "free" in

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Jeff Davis
On Thu, 2006-09-14 at 19:50 -0400, Francisco Reyes wrote: > > regarding shared buffers I'd make this much bigger, like 2GB or more > > Will do 2GB on the weekend. From what I read this requires shared memory so > have to restart my machine (FreeBSD). > You should be able to do: # sysctl -w kern

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Michael Stone
On Thu, Sep 14, 2006 at 05:52:02PM -0700, Jeff Davis wrote: Any long-running system will have very little "free" memory. Free memory is wasted memory, so the OS finds some use for it. The important part of the output of "free" in this context isn't how much is free, it's how much is cache vs h

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Jeff Davis
On Thu, 2006-09-14 at 20:07 -0400, Dave Cramer wrote: > On 14-Sep-06, at 7:50 PM, Francisco Reyes wrote: > > > Dave Cramer writes: > > > >> personally, I'd set this to about 6G. This doesn't actually > >> consume memory it is just a setting to tell postgresql how much > >> memory is being us

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Jeff Davis
On Thu, 2006-09-14 at 20:04 -0400, Francisco Reyes wrote: > Michael Stone writes: > > > On Thu, Sep 14, 2006 at 04:30:46PM -0400, Francisco Reyes wrote: > >>Right now adding up from ps the memory I have about 2GB. > > > > That's not how you find out how much memory you have. Try "free" or > > so

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Jeff Davis
On Thu, 2006-09-14 at 19:30 -0400, Francisco Reyes wrote: > Will have to talk to the developers. In particular for every insert there > are updates. I know they have at least one table that gets udpated to have > summarized totals. > If the table being updated is small, you have no problems at

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Michael Stone
On Thu, Sep 14, 2006 at 08:04:39PM -0400, Francisco Reyes wrote: Wasn't trying to get an accurate value, just a ballpark figure. Won't even be a ballpark. When you say "free" are you refering to the free value from top? or some program called free? Depends on your OS. Mike Stone -

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Dave Cramer
On 14-Sep-06, at 7:50 PM, Francisco Reyes wrote: Dave Cramer writes: personally, I'd set this to about 6G. This doesn't actually consume memory it is just a setting to tell postgresql how much memory is being used for cache and kernel buffers Gotcha. Will increase further. regarding

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Francisco Reyes
Michael Stone writes: On Thu, Sep 14, 2006 at 04:30:46PM -0400, Francisco Reyes wrote: Right now adding up from ps the memory I have about 2GB. That's not how you find out how much memory you have. Try "free" or somesuch. Wasn't trying to get an accurate value, just a ballpark figure. Whe

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Francisco Reyes
Dave Cramer writes: personally, I'd set this to about 6G. This doesn't actually consume memory it is just a setting to tell postgresql how much memory is being used for cache and kernel buffers Gotcha. Will increase further. regarding shared buffers I'd make this much bigger, like 2GB

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Francisco Reyes
Jeff Davis writes: shared_buffers = 1 Why so low? My initial research was not thorough enough with regards to how to compute how many to use. You have a lot of memory, and shared_buffers are an important performance setting. I have a machine with 4GB of RAM, and I found my best perf

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Dave Cramer
Francisco On 14-Sep-06, at 4:30 PM, Francisco Reyes wrote: Dave Cramer writes: What is a reasonable number? I estimate I have at least 1 to 2 GB free of memory. You are using 6G of memory for something else ? Right now adding up from ps the memory I have about 2GB. Have an occassional progr

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Jeff Davis
On Thu, 2006-09-14 at 11:23 -0400, Francisco Reyes wrote: > My setup: > Freebsd 6.1 > Postgresql 8.1.4 > Memory: 8GB > SATA Disks > > Raid 1 10 spindles (2 as hot spares) > 500GB disks (16MB buffer), 7200 rpm > Raid 10 > > Raid 2 4 spindles > 150GB 10K rpm disks > Raid 10 > > shared_buffers = 1

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Michael Stone
On Thu, Sep 14, 2006 at 04:30:46PM -0400, Francisco Reyes wrote: Right now adding up from ps the memory I have about 2GB. That's not how you find out how much memory you have. Try "free" or somesuch. Mike Stone ---(end of broadcast)--- TIP 5:

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Francisco Reyes
Dave Cramer writes: What is a reasonable number? I estimate I have at least 1 to 2 GB free of memory. You are using 6G of memory for something else ? Right now adding up from ps the memory I have about 2GB. Have an occassional program which uses up to 2GB. Then I want to give some breathing

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Dave Cramer
Francisco On 14-Sep-06, at 1:36 PM, Francisco Reyes wrote: Dave Cramer writes: What is effective_cache set to ? Increasing this seems to have helped significantly a web app. Load times seem magnitudes faster. Increased it to effective_cache_size = 12288 # 96MB What is a reasonable numbe

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Francisco Reyes
Dave Cramer writes: What is effective_cache set to ? Increasing this seems to have helped significantly a web app. Load times seem magnitudes faster. Increased it to effective_cache_size = 12288 # 96MB What is a reasonable number? I estimate I have at least 1 to 2 GB free of memory. Don't

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Francisco Reyes
Dave Cramer writes: What is effective_cache set to ? Default of 1000. Was just reading about this parameter. Will try increasing it to 8192 (8192 * 8K = 64MB) why not just let autovac do it's thing ? Have been playing with decresing the autovac values. With 100GB+ tables even 1% in autov

Re: [PERFORM] Vacuums on large busy databases

2006-09-14 Thread Dave Cramer
On 14-Sep-06, at 11:23 AM, Francisco Reyes wrote: My setup: Freebsd 6.1 Postgresql 8.1.4 Memory: 8GB SATA Disks Raid 1 10 spindles (2 as hot spares) 500GB disks (16MB buffer), 7200 rpm Raid 10 Raid 2 4 spindles 150GB 10K rpm disks Raid 10 shared_buffers = 1 shared buffers should be consi