Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread Ansgar -59cobalt- Wiechers
On 2007-09-05 Scott Marlowe wrote: On 9/5/07, Ansgar -59cobalt- Wiechers [EMAIL PROTECTED] wrote: On 2007-09-05 Scott Marlowe wrote: On 9/5/07, Ansgar -59cobalt- Wiechers [EMAIL PROTECTED] wrote: On 2007-09-05 Scott Marlowe wrote: And there's the issue that with windows / NTFS that when one

[PERFORM] Hardware spec

2007-09-06 Thread Willo van der Merwe
Hi guys, I'm have the rare opportunity to spec the hardware for a new database server. It's going to replace an older one, driving a social networking web application. The current server (a quad opteron with 4Gb of RAM and 80Gb fast SCSI RAID10) is coping with an average load of ranging between

Re: [PERFORM] Hardware spec

2007-09-06 Thread Richard Huxton
Willo van der Merwe wrote: Hi guys, I'm have the rare opportunity to spec the hardware for a new database server. It's going to replace an older one, driving a social networking web application. The current server (a quad opteron with 4Gb of RAM and 80Gb fast SCSI RAID10) is coping with an

Re: [PERFORM] Hardware spec

2007-09-06 Thread Willo van der Merwe
Richard Huxton wrote: Willo van der Merwe wrote: Hi guys, I'm have the rare opportunity to spec the hardware for a new database server. It's going to replace an older one, driving a social networking web application. The current server (a quad opteron with 4Gb of RAM and 80Gb fast SCSI RAID10)

Re: [PERFORM] Hardware spec

2007-09-06 Thread Jean-David Beyer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Willo van der Merwe wrote: Richard Huxton wrote: Willo van der Merwe wrote: Hi guys, I'm have the rare opportunity to spec the hardware for a new database server. It's going to replace an older one, driving a social networking web

Re: [PERFORM] Hardware spec

2007-09-06 Thread Willo van der Merwe
Jean-David Beyer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Willo van der Merwe wrote: Richard Huxton wrote: Willo van der Merwe wrote: Hi guys, I'm have the rare opportunity to spec the hardware for a new database server. It's going to replace an older one,

[PERFORM] SAN vs Internal Disks

2007-09-06 Thread Harsh Azad
Hi, We are currently running our DB on a DualCore, Dual Proc 3.Ghz Xeon, 8GB RAM, 4x SAS 146 GB 15K RPM on RAID 5. The current data size is about 50GB, but we want to purchase the hardware to scale to about 1TB as we think our business will need to support that much soon. - Currently we have a

Re: [PERFORM] utilising multi-cpu/core machines?

2007-09-06 Thread Kevin Grittner
On Wed, Sep 5, 2007 at 5:41 PM, in message [EMAIL PROTECTED], Thomas Finneid [EMAIL PROTECTED] wrote: how does pg utilise multi cpus/cores, i.e. does it use more than one core? and possibly, how, are there any documentation about this. For portability reasons PostgreSQL doesn't use

Re: [PERFORM] Hardware spec

2007-09-06 Thread Florian Weimer
* Willo van der Merwe: Good advice. After running a vmstat and iostat, it is clear, to my mind anyway, that the most likely bottleneck is IO, next is probably some more RAM. Here's the output: procs ---memory-- ---swap-- -io --system-- cpu r b swpd

Re: [PERFORM] Hardware spec

2007-09-06 Thread Florian Weimer
* Willo van der Merwe: Florian Weimer wrote: You need to run vmstat 10 (for ten-second averages) and report a couple of lines. 2 80 1 5 0 61732 37052 28180 34319560014 987 2320 2021 38 sda3 3.30 0.0026.40 0264 sda8

Re: [PERFORM] [ADMIN] ADO -PostgreSQL OLE DB Provider

2007-09-06 Thread Richard Broersma Jr
--- Jayaram Bhat [EMAIL PROTECTED] wrote: I am using a postgres setup in Windows. And it is working fine usings ODBC drive, but not in ADO PostgreSQL OLE DB Provider giving error test connection 'Test connection failed because of an error in initializing provider. Unspecified error'

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Scott Marlowe
On 9/6/07, Harsh Azad [EMAIL PROTECTED] wrote: Hi, We are currently running our DB on a DualCore, Dual Proc 3.Ghz Xeon, 8GB RAM, 4x SAS 146 GB 15K RPM on RAID 5. The current data size is about 50GB, but we want to purchase the hardware to scale to about 1TB as we think our business will

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Harsh Azad
Thanks Mark. If I replicate a snapshot of Data and log files (basically the entire PG data directory) and I maintain same version of postgres on both servers, it should work right? I am also thinking that having SAN storage will provide me with facility of keeping a warm standby DB. By just

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Mark Lewis
On Thu, 2007-09-06 at 18:05 +0530, Harsh Azad wrote: Hi, We are currently running our DB on a DualCore, Dual Proc 3.Ghz Xeon, 8GB RAM, 4x SAS 146 GB 15K RPM on RAID 5. The current data size is about 50GB, but we want to purchase the hardware to scale to about 1TB as we think our business

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Harsh Azad
Thanks Scott, we have now requested IBM/EMC to provide test machines. Interestingly since you mentioned the importance of Raid controllers and the drivers; we are planning to use Cent OS 5 for hosting the DB. Firstly, I could only find postgres 8.1.x RPM for CentOS 5, could not find any RPM for

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Harsh Azad wrote: Thanks Scott, we have now requested IBM/EMC to provide test machines. Interestingly since you mentioned the importance of Raid controllers and the drivers; we are planning to use Cent OS 5 for hosting the DB. Firstly, I could

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Arjen van der Meijden
On 6-9-2007 14:35 Harsh Azad wrote: 2x Quad Xeon 2.4 Ghz (4-way only 2 populated right now) I don't understand this sentence. You seem to imply you might be able to fit more processors in your system? Currently the only Quad Core's you can buy are dual-processor processors, unless you

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Scott Marlowe
On 9/6/07, Harsh Azad [EMAIL PROTECTED] wrote: Thanks Scott, we have now requested IBM/EMC to provide test machines. Interestingly since you mentioned the importance of Raid controllers and the drivers; we are planning to use Cent OS 5 for hosting the DB. What RAID controllers have you looked

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Harsh Azad
Hi, How about the Dell Perc 5/i card, 512MB battery backed cache or IBM ServeRAID-8k Adapter? I hope I am sending relevant information here, I am not too well versed with RAID controllers. Regards, Harsh On 9/6/07, Scott Marlowe [EMAIL PROTECTED] wrote: On 9/6/07, Harsh Azad [EMAIL

Re: [PERFORM] Hardware spec]

2007-09-06 Thread Jean-David Beyer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Willo van der Merwe wrote: Jean-David Beyer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Willo van der Merwe wrote: Richard Huxton wrote: Willo van der Merwe wrote: Hi guys, I'm have the rare opportunity to spec the hardware for a

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Mark Lewis
On Thu, 2007-09-06 at 22:28 +0530, Harsh Azad wrote: Thanks Mark. If I replicate a snapshot of Data and log files (basically the entire PG data directory) and I maintain same version of postgres on both servers, it should work right? I am also thinking that having SAN storage will provide

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Scott Marlowe
On 9/6/07, Harsh Azad [EMAIL PROTECTED] wrote: Hi, How about the Dell Perc 5/i card, 512MB battery backed cache or IBM ServeRAID-8k Adapter? All Dell Percs have so far been based on either adaptec or LSI controllers, and have ranged from really bad to fairly decent performers. There were

Re: [PERFORM] Hardware spec

2007-09-06 Thread Willo van der Merwe
Florian Weimer wrote: You need to run vmstat 10 (for ten-second averages) and report a couple of lines. procs ---memory-- ---swap-- -io --system-- cpu r b swpd free buff cache si sobibo incs us sy id wa 1 0 61732 47388 27908

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Joe Uhl
Scott Marlowe wrote: On 9/6/07, Harsh Azad [EMAIL PROTECTED] wrote: Hi, How about the Dell Perc 5/i card, 512MB battery backed cache or IBM ServeRAID-8k Adapter? All Dell Percs have so far been based on either adaptec or LSI controllers, and have ranged from really bad to fairly

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Joel Fradkin
I am not sure I agree with that evaluation. I only have 2 dell database servers and they have been 100% reliable. Maybe he is referring to support which does tend be up to who you get. When I asked about performance on my new server they were very helpful but I did have a bad time on my NAS device

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Arjen van der Meijden
On 6-9-2007 20:42 Scott Marlowe wrote: On 9/6/07, Harsh Azad [EMAIL PROTECTED] wrote: Hi, How about the Dell Perc 5/i card, 512MB battery backed cache or IBM ServeRAID-8k Adapter? All Dell Percs have so far been based on either adaptec or LSI controllers, and have ranged from really bad to

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Arjen van der Meijden
On 6-9-2007 20:29 Mark Lewis wrote: Maybe I'm jaded by past experiences, but the only real use case I can see to justify a SAN for a database would be something like Oracle RAC, but I'm not aware of any PG equivalent to that. PG Cluster II seems to be able to do that, but I don't know whether

[PERFORM] Reasonable amount of indices

2007-09-06 Thread Patric
Hi, I've a question about amount of indices. I explain my issue based on an example: Table which contains person information, one row per person. There will be lots of SELECTS doing search by special criteria, e.g.: Age, Gender. Now there will be 4 User groups which will

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread James Mansion
Scott Marlowe wrote: And there's the issue that with windows / NTFS that when one process opens a file for read, it locks it for all other users. This means that things like virus scanners can cause odd, unpredictable failures of your database. Can you provide some justification for this?

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread James Mansion
Scott Marlowe wrote: Where unixes generally outperform windows is in starting up new backends, better file systems, and handling very large shared_buffer settings. Why do you think that UNIX systems are better at handling large shared buffers than Wndows? 32 bit Windows systems can suffer

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread Scott Marlowe
On 9/6/07, James Mansion [EMAIL PROTECTED] wrote: Scott Marlowe wrote: And there's the issue that with windows / NTFS that when one process opens a file for read, it locks it for all other users. This means that things like virus scanners can cause odd, unpredictable failures of your

Re: [PERFORM] Reasonable amount of indices

2007-09-06 Thread Alvaro Herrera
Patric wrote: My Question now is: Is it wise to do so, and create hundreds or maybe thousands of Indices which partition the table for the selections. No, this is not helpful -- basically what you are doing is taking the first level (the first couple of levels maybe) of the index out

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread Carlo Stonebanks
Wow - it's nice to hear someone say that... out loud. Thanks, you gave me hope! -Original Message- From: James Mansion [mailto:[EMAIL PROTECTED] Sent: September 6, 2007 4:55 PM To: Carlo Stonebanks Cc: Scott Marlowe; Alvaro Herrera; pgsql-performance@postgresql.org Subject: Re:

Re: [PERFORM] How planner decides left-anchored LIKE can use index

2007-09-06 Thread Tom Lane
Carlo Stonebanks [EMAIL PROTECTED] writes: There is an index on lower(last_name). I have seen the planner convert the LIKE to lower(last_name) = 'smith' and lower(last_name) 'smiti' on 8.2.4 systems, but a slow sequence scan and filter on 8.1.9 - is this related to the version difference

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Scott Marlowe
On 9/6/07, Joel Fradkin [EMAIL PROTECTED] wrote: I am not sure I agree with that evaluation. I only have 2 dell database servers and they have been 100% reliable. Maybe he is referring to support which does tend be up to who you get. When I asked about performance on my new server they were

Re: [PERFORM] Reasonable amount of indices

2007-09-06 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Patric wrote: My Question now is: Is it wise to do so, and create hundreds or maybe thousands of Indices which partition the table for the selections. No, this is not helpful -- basically what you are doing is taking the first level (the first

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread Carlo Stonebanks
If what you mean is that pg has a design that's heavily oriented towards things that tend to be cheap on POSIX and doesn't use the core Win32 features effectively, then let's track that as an optimisation opportunity for the Win32 port. Isn't it just easier to assume that Windows Server can't

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread James Mansion
Carlo Stonebanks wrote: Isn't it just easier to assume that Windows Server can't do anything right? ;-) Well, avoiding the ;-) - people do, and its remarkably foolish of them. Its a long-standing whinge that many people with a UNIX-background seem to just assume that Windows sucks, but you

Re: [PERFORM] SAN vs Internal Disks

2007-09-06 Thread Greg Smith
On Thu, 6 Sep 2007, Harsh Azad wrote: Firstly, I could only find postgres 8.1.x RPM for CentOS 5, could not find any RPM for 8.2.4. Is there any 8.2.4 RPM for CentOS 5? You've already been pointed in the right direction. Devrim, the person who handles this packaging, does a great job of

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-06 Thread Alvaro Herrera
James Mansion escribió: If what you mean is that pg has a design that's heavily oriented towards things that tend to be cheap on POSIX and doesn't use the core Win32 features effectively, then let's track that as an optimisation opportunity for the Win32 port. Already done for 8.3 (actual