Re: [PERFORM] Large Database Performance suggestions

2004-11-15 Thread Andrew Hammond
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Joshua Marsh wrote: | Thanks for all of your help so far. Here is some of the information | you guys were asking for: | | Test System: | 2x AMD Opteron 244 (1.8Ghz) | 8GB RAM | 7x 72GB SCSI HDD (Raid 5) You probably want to look at investing in a SAN.

Re: [PERFORM] Large Database Performance suggestions

2004-10-26 Thread Tom Lane
Joshua Marsh <[EMAIL PROTECTED]> writes: > shared_buffers = 1000 # min 16, at least max_connections*2, 8KB each This is on the small side for an 8G machine. I'd try 1 or so. > sort_mem = 4096000 Yikes. You do realize you just said that *each sort operation* can use 4G? (Actually,

Re: [PERFORM] Large Database Performance suggestions

2004-10-26 Thread Joshua Marsh
Thanks for all of your help so far. Here is some of the information you guys were asking for: Test System: 2x AMD Opteron 244 (1.8Ghz) 8GB RAM 7x 72GB SCSI HDD (Raid 5) postrgesql.conf information: #--- # RESOURCE USAGE (exc

Re: [PERFORM] Large Database Performance suggestions

2004-10-22 Thread Dave Cramer
Josh, Your hardware setup would be useful too. It's surprising how slow some big name servers really are. If you are seriously considering memory sizes over 4G you may want to look at an opteron. Dave Joshua Marsh wrote: Hello everyone, I am currently working on a data project that uses PostgreS

Re: [PERFORM] Large Database Performance suggestions

2004-10-21 Thread Scott Marlowe
On Thu, 2004-10-21 at 21:14, Joshua Marsh wrote: > Hello everyone, > > I am currently working on a data project that uses PostgreSQL > extensively to store, manage and maintain the data. We haven't had > any problems regarding database size until recently. The three major > tables we use never g

Re: [PERFORM] Large Database Performance suggestions

2004-10-21 Thread Tom Lane
Joshua Marsh <[EMAIL PROTECTED]> writes: > ... We did some original testing and with a server with 8GB or RAM and > found we can do operations on data file up to 50 million fairly well, > but performance drop dramatically after that. What you have to ask is *why* does it drop dramatically? There

Re: [PERFORM] Large Database Performance suggestions

2004-10-21 Thread Gavin Sherry
On Thu, 21 Oct 2004, Joshua Marsh wrote: > Recently, we have found customers who are wanting to use our service > with data files between 100 million and 300 million records. At that > size, each of the three major tables will hold between 150 million and > 700 million records. At this size, I c

[PERFORM] Large Database Performance suggestions

2004-10-21 Thread Joshua Marsh
Hello everyone, I am currently working on a data project that uses PostgreSQL extensively to store, manage and maintain the data. We haven't had any problems regarding database size until recently. The three major tables we use never get bigger than 10 million records. With this size, we can do