Re: [PERFORM] best arrangement of 3 disks for (insert) performance
Having WAL on a separate drive from the database would be something of a win. I'd buy that 1 disk for OS+WAL and then RAID [something] across the other two drives for the database would be pretty helpful. Just my .02, I did a lot of testing before I deployed our ~50GB postgresql databases with various combinations of 6 15k SCSI drives. I did custom benchmarks to simulate our applications, I downloaded several benchmarks, etc. It might be a fluke, but I never got better performance with WALs on a different disk than I did with all 6 disks in a 0+1 configuration. Obviously that's not an option with 3 disks. =) I ended up going with that for easier space maintenance. Obviously YMMV, benchmark for your own situation. :) ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
the machine will be dealing with lots of inserts, basically as many as we can throw at it If you mean lots of _transactions_ with few inserts per transaction you should get a RAID controller w/ battery backed write-back cache. Nothing else will improve your write performance by nearly as much. You could sell the RAM and one of the CPU's to pay for it ;-) If you have lots of inserts but all in a few transactions then it's not quite so critical. M ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[PERFORM] best arrangement of 3 disks for (insert) performance
Hi all, I have some new hardware on the way and would like some advice on how to get the most out of it.. its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks should i mirror 2 of the disks for postgres data, and use the 3rd disk for the o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg and use the 3rd for o/s,logs,backups ? the machine will be dealing with lots of inserts, basically as many as we can throw at it thanks, Richard ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
The machine is coming from dell, and i have the option of a PERC 3/SC RAID Controller (32MB) or software raid. does anyone have any experience of this controller? its an additional £345 for this controller, i'd be interested to know what people think - my other option is to buy the raid controller separately, which appeals to me but i wouldnt know what to look for in a raid controller. that raid controller review site sounds like a good idea :) Richard. On Friday 12 September 2003 4:24 pm, Christopher Browne wrote: [EMAIL PROTECTED] (Richard Jones) writes: I have some new hardware on the way and would like some advice on how to get the most out of it.. its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks should i mirror 2 of the disks for postgres data, and use the 3rd disk for the o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg and use the 3rd for o/s,logs,backups ? the machine will be dealing with lots of inserts, basically as many as we can throw at it Having WAL on a separate drive from the database would be something of a win. I'd buy that 1 disk for OS+WAL and then RAID [something] across the other two drives for the database would be pretty helpful. After doing some [loose] benchmarking, the VERY best way to improve performance would involve a RAID controller with battery-backed cache. On a box with similar configuration to yours, it took ~3h for a particular set of data to load; on another one with battery-backed cache (and a dozen fast SCSI drives :-)), the same data took as little as 6 minutes to load. The BIG effect seemed to come from the controller. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] best arrangement of 3 disks for (insert) performance - Dell
The Dell PERC controllers have a very strong reputation for terrible performance. If you search the archives of the Dell Linux Power Edge list (dell.com/linux), you will find many, many people who get better performance from software RAID, rather than the hw RAID on the PERC. Having said that, the 3/SC might be one of the better PERC controllers. I would spend and hour or two and benchmark hw vs. sw before I committed to either one. Thom Dyson Director of Information Services Sybex, Inc. On 9/12/2003 9:55:40 AM, Richard Jones [EMAIL PROTECTED] wrote: The machine is coming from dell, and i have the option of a PERC 3/SC RAID Controller (32MB) or software raid. does anyone have any experience of this controller? its an additional £345 for this controller, i'd be interested to know what people think - my other option is to buy the raid controller separately, which appeals to me but i wouldnt know what to look for in a raid controller. that raid controller review site sounds like a good idea :) Richard. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
I would like to point out though on the PERC controllers that are LSI based ( Megaraid ) there -are- settings that can be changed to fix any o the performance issues. Check the linux megaraid driver list archives to see the full description. I've seen it come up many times and basically all the problems have turned up resolved. Will On Fri, 2003-09-12 at 10:03, Thom Dyson wrote: The Dell PERC controllers have a very strong reputation for terrible performance. If you search the archives of the Dell Linux Power Edge list (dell.com/linux), you will find many, many people who get better performance from software RAID, rather than the hw RAID on the PERC. Having said that, the 3/SC might be one of the better PERC controllers. I would spend and hour or two and benchmark hw vs. sw before I committed to either one. Thom Dyson Director of Information Services Sybex, Inc. On 9/12/2003 9:55:40 AM, Richard Jones [EMAIL PROTECTED] wrote: The machine is coming from dell, and i have the option of a PERC 3/SC RAID Controller (32MB) or software raid. does anyone have any experience of this controller? its an additional £345 for this controller, i'd be interested to know what people think - my other option is to buy the raid controller separately, which appeals to me but i wouldnt know what to look for in a raid controller. that raid controller review site sounds like a good idea :) Richard. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster signature.asc Description: This is a digitally signed message part
Re: [PERFORM] best arrangement of 3 disks for (insert) performance - Dell
[EMAIL PROTECTED] (Thom Dyson) writes: The Dell PERC controllers have a very strong reputation for terrible performance. If you search the archives of the Dell Linux Power Edge list (dell.com/linux), you will find many, many people who get better performance from software RAID, rather than the hw RAID on the PERC. Having said that, the 3/SC might be one of the better PERC controllers. I would spend and hour or two and benchmark hw vs. sw before I committed to either one. I can't agree with that. 1. If you search the archives for messages dated a couple of years ago, you can find lots of messages indicating terrible performance. Drivers are not cast in concrete; there has been a LOT of change to them since then. 2. The second MAJOR merit to hardware RAID is the ability to hot-swap drives. Software RAID doesn't help with that at all. 3. The _immense_ performance improvement that can be gotten out of these controllers comes from having fsync() turn into a near no-op since changes can be committed to the 128K battery-backed cache REALLY QUICKLY. That is something you should avoid doing with software RAID in any case where you actually care about your data. That third part is where Big Wins come. It is the very same sort of big win from cacheing that we saw, years ago, when we improved system performance _immensely_ by adding a mere 16 bytes of cache by buying serial controller cards with cacheing UUARTs. It is akin to the way SCSI controllers got pretty big performance improvements by adding 256 bytes of tagged command cache. -- output = (cbbrowne @ libertyrms.info) http://dev6.int.libertyrms.com/ Christopher Browne (416) 646 3304 x124 (land) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
On Fri, 2003-09-12 at 12:55, Richard Jones wrote: The machine is coming from dell, and i have the option of a PERC 3/SC RAID Controller (32MB) or software raid. does anyone have any experience of this controller? its an additional £345 for this controller, i'd be interested to know what people think - my other option is to buy the raid controller separately, which appeals to me but i wouldnt know what to look for in a raid controller. Hardware raid with the write cache, and sell a CPU if necessary to buy it (don't sell the ram though!). signature.asc Description: This is a digitally signed message part
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
The dual xeon arrangement is because the machine will also have to do some collaborative filtering which is very cpu intensive and very disk un-intensive, after loading the data into ram. On Friday 12 September 2003 5:49 pm, you wrote: RIchard, its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks should i mirror 2 of the disks for postgres data, and use the 3rd disk for the o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg and use the 3rd for o/s,logs,backups ? I'd mirror 2. Stripey RAID with few disks imposes a heavy performance penalty on data writes (particularly updates), sometimes as much as 50% for a RAID5-3disk config. I am a little curious why you've got a dual-xeon, but could only afford 3 disks ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
RIchard, its a dual xeon 2.4, 4gb ram and 3x identical 15k rpm scsi disks should i mirror 2 of the disks for postgres data, and use the 3rd disk for the o/s and the pg logs or raid5 the 3 disks or even stripe 2 disks for pg and use the 3rd for o/s,logs,backups ? I'd mirror 2. Stripey RAID with few disks imposes a heavy performance penalty on data writes (particularly updates), sometimes as much as 50% for a RAID5-3disk config. I am a little curious why you've got a dual-xeon, but could only afford 3 disks -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
