Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Hi, Charles, Charles Sprickman wrote: I've also got a 1U with a 9500SX-4 and 4 drives. I like how the 3Ware card scales there - started with 2 drives and got drive speed mirroring. Added two more and most of the bonnie numbers doubled. This is not what I'm used to with the Adaptec SCSI junk. Well, for sequential reading, you should be able to get double drive speed on a 2-disk mirror with a good controller, as it can balance the reads among the drives. Markus -- Markus Schaber | Logical TrackingTracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in EU! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Although I for one have yet to see a controller that actualy does this (I believe software RAID on linux doesn't either).Alex.On 8/7/06, Markus Schaber [EMAIL PROTECTED] wrote:Hi, Charles, Charles Sprickman wrote: I've also got a 1U with a 9500SX-4 and 4 drives.I like how the 3Ware card scales there - started with 2 drives and got drive speed mirroring. Added two more and most of the bonnie numbers doubled.This is not what I'm used to with the Adaptec SCSI junk.Well, for sequential reading, you should be able to get double drivespeed on a 2-disk mirror with a good controller, as it can balance thereads among the drives. Markus--Markus Schaber | Logical TrackingTracing International AGDipl. Inf. | Software Development GISFight against software patents in EU! www.ffii.org www.nosoftwarepatents.org---(end of broadcast)---TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
On Mon, Aug 07, 2006 at 04:02:52PM -0400, Alex Turner wrote: Although I for one have yet to see a controller that actualy does this (I believe software RAID on linux doesn't either). Linux' software RAID does. See earlier threads for demonstrations. /* Steinar */ -- Homepage: http://www.sesse.net/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig DDR PC3200
Hello.OS: Gentoo 2006.0 with gentoo's hardened kernelVersion: I haven't checked. Im guessing 8.0.8 (latest stable on all systems) or 8.1.4 which is the latest package.I'm still gonna try to run with smart array 5i. How can i find out that my performance with that is crappy? Without ripping down my systems, and using software raid? Kjell ToreOn 7/28/06, Claus Guttesen [EMAIL PROTECTED] wrote: As I have understood, there is alot of tuning using both postgres.conf and analyzing queries to make the values of postgres.conf fit my needs, system and hardware. This is where I need some help. I have looked into postgres.conf , and seen the tunings. But I'm still not sure what I should put into those variables (in postgres.conf) with my hardware. Any suggestions would be most appreciated!What OS is it running and what version is postgresql? regardsClaus-- Be nice to people on your way up because you meet them on your way down.
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Run bonnie++ version 1.03 and report results here. - Luke Sent from my GoodLink synchronized handheld (www.good.com) -Original Message- From: Kjell Tore Fossbakk [mailto:[EMAIL PROTECTED] Sent: Sunday, July 30, 2006 03:03 PM Eastern Standard Time To: Claus Guttesen Cc: pgsql-performance@postgresql.org Subject:Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig DDR PC3200 Hello. OS: Gentoo 2006.0 with gentoo's hardened kernel Version: I haven't checked. Im guessing 8.0.8 (latest stable on all systems) or 8.1.4 which is the latest package. I'm still gonna try to run with smart array 5i. How can i find out that my performance with that is crappy? Without ripping down my systems, and using software raid? Kjell Tore On 7/28/06, Claus Guttesen [EMAIL PROTECTED] wrote: As I have understood, there is alot of tuning using both postgres.confand analyzing queries to make the values of postgres.conf fit my needs, system and hardware. This is where I need some help. I have looked into postgres.conf , and seen the tunings. But I'm still not sure what I should put into those variables (in postgres.conf) with my hardware. Any suggestions would be most appreciated! What OS is it running and what version is postgresql? regards Claus -- Be nice to people on your way up because you meet them on your way down. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig DDR PC3200
Okey!The thing is, im on vacation. So ill report in about 3 weeks time.. Sry guys.. :-)Kjell ToreOn 7/30/06, Luke Lonergan [EMAIL PROTECTED] wrote:Run bonnie++ version 1.03 and report results here. - LukeSent from my GoodLink synchronized handheld (www.good.com) -Original Message-From: Kjell Tore Fossbakk [mailto: [EMAIL PROTECTED]]Sent: Sunday, July 30, 2006 03:03 PM Eastern Standard TimeTo: Claus GuttesenCc: pgsql-performance@postgresql.orgSubject:Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig DDR PC3200Hello.OS: Gentoo 2006.0 with gentoo's hardened kernelVersion: I haven't checked. Im guessing 8.0.8 (latest stable on all systems)or 8.1.4 which is the latest package. I'm still gonna try to run with smart array 5i. How can i find out that myperformance with that is crappy? Without ripping down my systems, and usingsoftware raid?Kjell ToreOn 7/28/06, Claus Guttesen [EMAIL PROTECTED] wrote: As I have understood, there is alot of tuning using both postgres.confand analyzing queries to make the values of postgres.conf fit my needs, system and hardware. This is where I need some help. I have looked into postgres.conf , and seen the tunings. But I'm still not sure what I should put into those variables (in postgres.conf) with my hardware. Any suggestions would be most appreciated! What OS is it running and what version is postgresql? regards Claus --Be nice to people on your way up because you meet them on your way down.-- Be nice to people on your way up because you meet them on your way down.
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
On Fri, 28 Jul 2006, Mikael Carneholm wrote: Luke, Yeah, I read those results, and I'm very disappointed with my results from the MSA1500. I would however be interested in other people's bonnie++ and benchmarksql results using a similar machine (2 cpu dual core opteron) with other off the shelf storage systems (EMC/Netapp/Xyratex/../). Could you run benchmarksql against that machine with the 16 SATA disk and 3Ware 9550SX SATA RAID adapters? It would be *very* interesting to see how the I/O performance correlates to benchmarksql (postgres) transaction throughout. FWIW, once our vendor gets all the pieces (having some issues with figuring out which multilane sata cables to get), I'll have a dual-core opteron box with a 3Ware 9500SX-12MI and 8 drives. I need to benchmark to compare this to our xeon/adaptec/scsi build we've been using. I've also got a 1U with a 9500SX-4 and 4 drives. I like how the 3Ware card scales there - started with 2 drives and got drive speed mirroring. Added two more and most of the bonnie numbers doubled. This is not what I'm used to with the Adaptec SCSI junk. These SATA RAID controllers 3Ware is making seem to be leaps and bounds beyond what the old guard is churning out (at much higher prices). Charles /Mikael -Original Message- From: Luke Lonergan [mailto:[EMAIL PROTECTED] Sent: den 28 juli 2006 11:17 To: Mikael Carneholm; Kjell Tore Fossbakk; pgsql-performance@postgresql.org Subject: RE: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig Mikael, -Original Message- From: Mikael Carneholm [mailto:[EMAIL PROTECTED] Sent: Friday, July 28, 2006 2:05 AM My bonnie++ results are found in this message: http://archives.postgresql.org/pgsql-performance/2006-07/msg00164.php Apologies if I've already said this, but those bonnie++ results are very disappointing. The sequential transfer rates between 20MB/s and 57MB/s are slower than a single SATA disk, and your SCSI disks might even do 80MB/s sequential transfer rate each. Random access is also very poor, though perhaps equal to 5 disk drives at 500/second. By comparison, we routinely get 950MB/s sequential transfer rate using 16 SATA disks and 3Ware 9550SX SATA RAID adapters on Linux. On Solaris ZFS on an X4500, we recently got this bonnie++ result on 36 SATA disk drives in RAID10 (single thread first): Version 1.03 --Sequential Output----Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 120453 99 467814 98 290391 58 109371 99 993344 94 1801 4 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ 30850 99 + +++ + +++ Bumping up the number of concurrent processes to 2, we get about 1.5x speed reads of RAID10 with a concurrent workload (you have to add the rates together): Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 111441 95 212536 54 171798 51 106184 98 719472 88 1233 2 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26085 90 + +++ 5700 98 21448 97 + +++ 4381 97 Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 116355 99 212509 54 171647 50 106112 98 715030 87 1274 3 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26082 99 + +++ 5588 98 21399 88 + +++ 4272 97 So that's 2500 seeks per second, 1440MB/s sequential block read, 212MB/s per character sequential read. - Luke ---(end of broadcast)--- TIP 6: explain analyze is your friend ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Charles, FWIW, once our vendor gets all the pieces (having some issues with figuring out which multilane sata cables to get), I'll have a dual-core opteron box with a 3Ware 9500SX-12MI and 8 drives. I need to benchmark to compare this to our xeon/adaptec/scsi build we've been using. Cool! You mean the 9550SX, not the 9500, right? A trick on the 9550SX with Linux is to set the max readahead to 512KB and no larger when using RAID10. If you use RAID5, set it to 16MB. Here is how you set it (put in /etc/rc.d/rc.local) for 512KB on /dev/sda: /sbin/blockdev --setra 512 /dev/sda I was able to go from 310MB/s on 8 drives to 475MB/s that way (using XFS). Also, you need to stay away from Linux Volume Manager (lvm and lvm2), they add a lot of overhead (!!?) to the block access. It took me a long time to figure that out! - Luke ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
systems could send me their bonnie + benchmarksql results! I am one of the authors of BenchmarkSQL, it is similar to a DBT2. But, its very easy to use (/or abuse). It's a multithreaded Java Swing client that can run the exact same benchmark (uses JDBC prepared statements) against Postgres/EnterpriseDB/Bizgres, MySQueeL, Horacle, Microsloth, etc, etc. You can find BenchmarkSQL on pgFoundry and SourceForge. As expected, Postgres is good on this benchmark and is getting better all the time. If you run an EnterpriseDB install right out of the box versus a PG install right out of the box you'll notice that EnterpriseDB outperforms PG by better than 2x. This does NOT mean that EnterpriseDB is 3x faster than Postgres... EnterpriseDB is the same speed as Postgres. We do something we call Dynatune at db startup time. The algorithm is pretty simple in our current GA version and really only considers the amount of RAM, SHARED Memory, and machine usage pattern. Manual tuning is required to really optimize performance Forgreat insightinto the basics of quickly tuning PostgreSQL for a reasonable starting point,check out the great instructions offered by Josh Berkus and Joe Conway at http://www.powerpostgresql.com/PerfList/. The moral of this unreasonably verbose email is that you shouldn't abuse BenchmarkSQL andmeasure runswithout making sure that, at least, quick/simple best practices have been applied to tuning the db's you are choosing to test. --Denis Lussier CTO http://www.enterprisedb.com
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Denis, On 7/29/06 11:09 AM, Denis Lussier [EMAIL PROTECTED] wrote: We do something we call Dynatune at db startup time. Sounds great - where do we download it? - Luke ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Kjell, I got 4 150GIG SCSI disks in a Smart Array 5i 1+0 RAID. The Smart Array 5i is a terrible performer on Linux. I would be surprised if you exceed the performance of a single hard drive with this controller when doing I/O from disk. Since your database working set is larger than memory on the machine, I would recommend you use a simple non-RAID U320 SCSI controller, like those from LSI Logic (which HP resells) and implement Linux software RAID. You should see a nearly 10x increase in performance as compared to the SmartArray 5i. If you have a good relationship with HP, please ask them for some documentation of RAID performance on Linux with the SmartArray 5i. I predict they will tell you what they've told me and others: the 5i is only useful for booting the OS. Alternately they could say: we have world record performance with our RAID controllers, in which case you should ask them if that was with the 5i on Linux or whether it was the 6-series on Windows. - Luke ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Mikael, -Original Message- From: Mikael Carneholm [mailto:[EMAIL PROTECTED] Sent: Friday, July 28, 2006 1:47 AM I would be interested in what numbers you would get out of bonnie++ (http://www.coker.com.au/bonnie++) and BenchmarkSQL (http://sourceforge.net/projects/benchmarksql) on that hardware, for comparison with our DL385 (2xOpteron 280, 16Gb ram) and MSA1500. If you need help building benchmarksql, I can assist you with that. Me too. Can you post your MSA1500 results? The MSA500/1000 come with two SmartArray 6402 controllers, but the RAID is done inside the MSA500/1000 chassis from what I understand. I have heard that the performance on Linux is pretty good, but I've not seen the benchmarks to prove it. Bonnie++ is fine - should tell us what we need to know. Also, I am *very* interested in seeing what the P600 SAS controller results look like when coupled with an MSA50 SAS chassis with 10 disks. This is the new SAS controller that can be configured on the DL385 and 585. - Luke ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Mikael, -Original Message- From: Mikael Carneholm [mailto:[EMAIL PROTECTED] Sent: Friday, July 28, 2006 2:05 AM My bonnie++ results are found in this message: http://archives.postgresql.org/pgsql-performance/2006-07/msg00164.php Apologies if I've already said this, but those bonnie++ results are very disappointing. The sequential transfer rates between 20MB/s and 57MB/s are slower than a single SATA disk, and your SCSI disks might even do 80MB/s sequential transfer rate each. Random access is also very poor, though perhaps equal to 5 disk drives at 500/second. By comparison, we routinely get 950MB/s sequential transfer rate using 16 SATA disks and 3Ware 9550SX SATA RAID adapters on Linux. On Solaris ZFS on an X4500, we recently got this bonnie++ result on 36 SATA disk drives in RAID10 (single thread first): Version 1.03 --Sequential Output----Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 120453 99 467814 98 290391 58 109371 99 993344 94 1801 4 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ 30850 99 + +++ + +++ Bumping up the number of concurrent processes to 2, we get about 1.5x speed reads of RAID10 with a concurrent workload (you have to add the rates together): Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 111441 95 212536 54 171798 51 106184 98 719472 88 1233 2 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26085 90 + +++ 5700 98 21448 97 + +++ 4381 97 Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 116355 99 212509 54 171647 50 106112 98 715030 87 1274 3 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26082 99 + +++ 5588 98 21399 88 + +++ 4272 97 So that's 2500 seeks per second, 1440MB/s sequential block read, 212MB/s per character sequential read. - Luke ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Luke, Yeah, I read those results, and I'm very disappointed with my results from the MSA1500. I would however be interested in other people's bonnie++ and benchmarksql results using a similar machine (2 cpu dual core opteron) with other off the shelf storage systems (EMC/Netapp/Xyratex/../). Could you run benchmarksql against that machine with the 16 SATA disk and 3Ware 9550SX SATA RAID adapters? It would be *very* interesting to see how the I/O performance correlates to benchmarksql (postgres) transaction throughout. /Mikael -Original Message- From: Luke Lonergan [mailto:[EMAIL PROTECTED] Sent: den 28 juli 2006 11:17 To: Mikael Carneholm; Kjell Tore Fossbakk; pgsql-performance@postgresql.org Subject: RE: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig Mikael, -Original Message- From: Mikael Carneholm [mailto:[EMAIL PROTECTED] Sent: Friday, July 28, 2006 2:05 AM My bonnie++ results are found in this message: http://archives.postgresql.org/pgsql-performance/2006-07/msg00164.php Apologies if I've already said this, but those bonnie++ results are very disappointing. The sequential transfer rates between 20MB/s and 57MB/s are slower than a single SATA disk, and your SCSI disks might even do 80MB/s sequential transfer rate each. Random access is also very poor, though perhaps equal to 5 disk drives at 500/second. By comparison, we routinely get 950MB/s sequential transfer rate using 16 SATA disks and 3Ware 9550SX SATA RAID adapters on Linux. On Solaris ZFS on an X4500, we recently got this bonnie++ result on 36 SATA disk drives in RAID10 (single thread first): Version 1.03 --Sequential Output----Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 120453 99 467814 98 290391 58 109371 99 993344 94 1801 4 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ 30850 99 + +++ + +++ Bumping up the number of concurrent processes to 2, we get about 1.5x speed reads of RAID10 with a concurrent workload (you have to add the rates together): Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 111441 95 212536 54 171798 51 106184 98 719472 88 1233 2 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26085 90 + +++ 5700 98 21448 97 + +++ 4381 97 Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 116355 99 212509 54 171647 50 106112 98 715030 87 1274 3 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26082 99 + +++ 5588 98 21399 88 + +++ 4272 97 So that's 2500 seeks per second, 1440MB/s sequential block read, 212MB/s per character sequential read. - Luke ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Hello. Unfortunately, I'm leaving for my vacation now, gone 3 weeks. When I'm back I'll run benchmarksql and bonnie++ and give the results here. The spec I will be using: Prolite DL585 2 x AMD/Opteron 64-bit 2,6GHZ 8G DDR PC3200 4 x 150G SCSI in SmartArray 5i Running Gentoo 2006.0 AMD_64 Hardened kernel Then I will remove the SmartArray 5i, and use a simple nonRAID SCSI controller and implement Linux software RAID, and re-run the tests. I'll give signal in 3 weeks - Kjell Tore.On 7/28/06, Mikael Carneholm [EMAIL PROTECTED] wrote: Luke,Yeah, I read those results, and I'm very disappointed with my resultsfrom the MSA1500. I would however be interested in other people's bonnie++ and benchmarksql results using a similar machine (2 cpu dualcore opteron) with other off the shelf storage systems(EMC/Netapp/Xyratex/../). Could you run benchmarksql against thatmachine with the 16 SATA disk and 3Ware 9550SX SATA RAID adapters? It would be *very* interesting to see how the I/O performance correlates tobenchmarksql (postgres) transaction throughout./Mikael-Original Message-From: Luke Lonergan [mailto: [EMAIL PROTECTED]]Sent: den 28 juli 2006 11:17To: Mikael Carneholm; Kjell Tore Fossbakk;pgsql-performance@postgresql.orgSubject: RE: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gigMikael, -Original Message- From: Mikael Carneholm [mailto:[EMAIL PROTECTED]] Sent: Friday, July 28, 2006 2:05 AM My bonnie++ results are found in this message: http://archives.postgresql.org/pgsql-performance/2006-07/msg00164.php Apologies if I've already said this, but those bonnie++ results are verydisappointing.The sequential transfer rates between 20MB/s and 57MB/sare slower than a single SATA disk, and your SCSI disks might even do 80MB/s sequential transfer rate each.Random access is also very poor, though perhaps equal to 5 disk drivesat 500/second.By comparison, we routinely get 950MB/s sequential transfer rate using16 SATA disks and 3Ware 9550SX SATA RAID adapters on Linux. On Solaris ZFS on an X4500, we recently got this bonnie++ result on 36SATA disk drives in RAID10 (single thread first):Version1.03 --Sequential OutputSequential Input---Random- -Per Chr---Block---Rewrite--Per Chr---BlockSeeks--MachineSize K/sec%CP K/sec%CP K/sec%CP K/sec%CP K/sec%CP /sec %CPthumperdw-i-1 32G 12045399 46781498 29039158 10937199 993344 94 1801 4--Sequential Create-- RandomCreate-Create-- --Read--- -Delete-- -Create-- --ReadDelete--files/sec %CP/sec %CP/sec %CP/sec %CP/sec %CP /sec %CP 16 + +++ + +++ + +++ 3085099 + +++Bumping up the number of concurrent processes to 2, we get about 1.5xspeed reads of RAID10 with a concurrent workload (you have to add the rates together):Version1.03 --Sequential Output-- --Sequential Input---Random--Per Chr- --Block---Rewrite--Per Chr---BlockSeeks--MachineSize K/sec%CP K/sec%CP K/sec%CP K/sec%CP K/sec %CP/sec %CPthumperdw-i-1 32G 11144195 21253654 17179851 10618498 719472881233 2--Sequential Create-- RandomCreate-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--files/sec %CP/sec %CP/sec %CP/sec %CP/sec %CP/sec %CP 16 2608590 + +++570098 2144897 + +++438197Version1.03 --Sequential Output-- --Sequential Input- --Random--Per Chr---Block---Rewrite--Per Chr---Block-- --Seeks--MachineSize K/sec%CP K/sec%CP K/sec%CP K/sec%CP K/sec%CP/sec %CPthumperdw-i-1 32G 11635599 21250954 17164750 10611298 715030 871274 3--Sequential Create-- RandomCreate-Create-- --Read--- -Delete-- -Create-- --ReadDelete--files/sec %CP/sec %CP/sec %CP/sec %CP/sec %CP /sec %CP 16 2608299 + +++558898 2139988 + +++427297So that's 2500 seeks per second, 1440MB/s sequential block read, 212MB/sper character sequential read.- Luke -- Be nice to people on your way up because you meet them on your way down.
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
I too have a DL385 with a single DC Opteron 270. It claims to have a smart array 6i controller and over the last couple of days I've been runnign some tests on it, which have been yielding some suprising results. I've got 6 10k U320 disks in it. 2 are in a mirror set. We'll not pay any attention to them. The remaining 4 disks I've been toying with to see what config works best, using hardware raid and software raid. system info: dl dl385 - 1 opteron 270 - 5GB ram - smart array 6i cciss0: HP Smart Array 6i Controller Firmware Version: 2.58 Linux db03 2.6.17-1.2157_FC5 #1 SMP Tue Jul 11 22:53:56 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux using xfs Each drive can sustain 80MB/sec read (dd, straight off device) So here are the results I have so far. (averaged) hardware raid 5: dd - write 20GB file - 48MB/sec dd - read 20GB file - 247MB/sec [ didn't do a bonnie run on this yet ] pretty terrible write performance. good read. hardware raid 10 dd - write 20GB - 104MB/sec dd - read 20GB - 196MB/sec bonnie++ Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- -- Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % CP /sec %CP db03 9592M 45830 97 129501 31 62981 14 48524 99 185818 19 949.0 1 software raid 5 dd - write 20gb - 85MB/sec dd - read 20gb - 135MB/sec I was very suprised at those results. I was sort of expecting it to smoke the hardware. I repeated the test many times, and kept getting these numbers. bonnie++: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- -- Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % CP /sec %CP db03 9592M 44110 97 81481 23 34604 10 44495 95 157063 28 919.3 1 software 10: dd - write - 20GB - 108MB/sec dd - read - 20GB - 86MB/sec( WTF? - this is repeatable!!) bonnie++ Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- -- Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % CP /sec %CP db03 9592M 44539 98 105444 20 34127 8 39830 83 100374 10 1072 1 so I'm going to be going with hw r5, which went against what I thought going in - read perf is more important for my usage than write. I'm still not sure about that software 10 read number. something is not right there... -- Jeff Trout [EMAIL PROTECTED] http://www.dellsmartexitin.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Mikael Carneholm wrote: I would be interested in what numbers you would get out of bonnie++ (http://www.coker.com.au/bonnie++) and BenchmarkSQL (http://sourceforge.net/projects/benchmarksql) on that hardware, for comparison with our DL385 (2xOpteron 280, 16Gb ram) and MSA1500. If you need help building benchmarksql, I can assist you with that. Actually, I would be interested if everyone who's reading this that has a similar machine (2 cpu, dual core opteron) with different storage systems could send me their bonnie + benchmarksql results! Here's the bonnie++ results from our Sun Fire V40z (2x Opteron 250, 4GB RAM) with 6 15krpm 73GB drives connected to an LSI MegaRAID 320-2X controller with 512MB cache. It's running Linux, and I'm using what seems to be a fairly typical 6-drive setup: 2 drives in RAID-1 for OS and WAL, and 4 drives in RAID-10 for data. This is from the 4-drive RAID-10 array: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP gaz 8G 56692 88 73061 12 33048 6 44994 64 132571 14 474.0 0 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 19448 88 + +++ 18611 72 19952 90 + +++ 15167 65 This system is actually in production currently, and while it's a rather quiet time at the moment, it still wasn't _entirely_ inactive when those numbers were run, so the real performance is probably a little higher. I'll see if I can run some BenchmarkSQL numbers as well. Thanks Leigh /Mikael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Luke Lonergan Sent: den 28 juli 2006 08:55 To: Kjell Tore Fossbakk; pgsql-performance@postgresql.org Subject: Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig Kjell, I got 4 150GIG SCSI disks in a Smart Array 5i 1+0 RAID. The Smart Array 5i is a terrible performer on Linux. I would be surprised if you exceed the performance of a single hard drive with this controller when doing I/O from disk. Since your database working set is larger than memory on the machine, I would recommend you use a simple non-RAID U320 SCSI controller, like those from LSI Logic (which HP resells) and implement Linux software RAID. You should see a nearly 10x increase in performance as compared to the SmartArray 5i. If you have a good relationship with HP, please ask them for some documentation of RAID performance on Linux with the SmartArray 5i. I predict they will tell you what they've told me and others: the 5i is only useful for booting the OS. Alternately they could say: we have world record performance with our RAID controllers, in which case you should ask them if that was with the 5i on Linux or whether it was the 6-series on Windows. - Luke ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
This isn't all that surprising. The main weaknesses of RAID-5 are poor write performance and stupid hardware controllers that make the write performance even worse than it needs to be. Your numbers bear that out. Reads off RAID-5 are usually pretty good. Your 'dd' test is going to be a little misleading though. Most DB access isn't usually purely sequential; while it's easy to see why HW RAID-5 might outperform HW-RAID-10 in large sequential reads (the RAID controller would need to be smarter than most to make RAID-10 as fast as RAID-5), I would expect that HW RAID-5 and RAID-10 random reads would be about equal or else maybe give a slight edge to RAID-10. -- Mark Lewis On Fri, 2006-07-28 at 13:31 -0400, Jeff Trout wrote: I too have a DL385 with a single DC Opteron 270. It claims to have a smart array 6i controller and over the last couple of days I've been runnign some tests on it, which have been yielding some suprising results. I've got 6 10k U320 disks in it. 2 are in a mirror set. We'll not pay any attention to them. The remaining 4 disks I've been toying with to see what config works best, using hardware raid and software raid. system info: dl dl385 - 1 opteron 270 - 5GB ram - smart array 6i cciss0: HP Smart Array 6i Controller Firmware Version: 2.58 Linux db03 2.6.17-1.2157_FC5 #1 SMP Tue Jul 11 22:53:56 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux using xfs Each drive can sustain 80MB/sec read (dd, straight off device) So here are the results I have so far. (averaged) hardware raid 5: dd - write 20GB file - 48MB/sec dd - read 20GB file - 247MB/sec [ didn't do a bonnie run on this yet ] pretty terrible write performance. good read. hardware raid 10 dd - write 20GB - 104MB/sec dd - read 20GB - 196MB/sec bonnie++ Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- -- Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % CP /sec %CP db03 9592M 45830 97 129501 31 62981 14 48524 99 185818 19 949.0 1 software raid 5 dd - write 20gb - 85MB/sec dd - read 20gb - 135MB/sec I was very suprised at those results. I was sort of expecting it to smoke the hardware. I repeated the test many times, and kept getting these numbers. bonnie++: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- -- Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % CP /sec %CP db03 9592M 44110 97 81481 23 34604 10 44495 95 157063 28 919.3 1 software 10: dd - write - 20GB - 108MB/sec dd - read - 20GB - 86MB/sec( WTF? - this is repeatable!!) bonnie++ Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- -- Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec % CP /sec %CP db03 9592M 44539 98 105444 20 34127 8 39830 83 100374 10 1072 1 so I'm going to be going with hw r5, which went against what I thought going in - read perf is more important for my usage than write. I'm still not sure about that software 10 read number. something is not right there... -- Jeff Trout [EMAIL PROTECTED] http://www.dellsmartexitin.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] Performance with 2 AMD/Opteron 2.6Ghz and 8gig
Jeff, On 7/28/06 10:31 AM, Jeff Trout [EMAIL PROTECTED] wrote: I'm still not sure about that software 10 read number. something is not right there... It's very consistent with what we've seen before - the hardware RAID controller doesn't do JBOD with SCSI command queuing like a simple SCSI controller would do. The Smart Array 6402 makes a very bad SCSI controller for software RAID. The hardware results look very good - seems like the 2.6.17 linux kernel has a drastically improved CCISS driver as compared to what I've previously seen. - Luke ---(end of broadcast)--- TIP 6: explain analyze is your friend
