Re: [PERFORM] availability of SATA vendors
On Wed, Nov 22, 2006 at 09:02:04AM -0800, Jeff Frost wrote: A valid question. Does the caching raid controller negate the desire to separate pg_xlog from PGDATA? Theoretically, yes. But I don't think I've seen any hard numbers from testing. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PERFORM] availability of SATA vendors
On Wed, Nov 22, 2006 at 04:35:37PM -0500, Bucky Jordan wrote: While I'm at it, if I have time I'll run pgbench with pg_log on a separate RAID1, and one with it on a RAID10x6, but I don't know how useful those results will be. Very, but only if the controller has write-caching enabled. For testing purposes it won't batter if it's actually got a BBU so long as the write cache works (of course you wouldn't run in production like that...) -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] availability of SATA vendors
Hi Luke, I forgot about that article, thanks for that link. That's indeed a nice overview of (in august) recent controllers. The Areca 1280 in that test (and the results I linked to earlier) is a pre-production model, so it might actually perform even better than in that test. We've been getting samples from AMCC in the past, so a 96xx should be possible. I've pointed it out to the author of the previous raid-articles. Thanks for pointing that out to me. Best regards, Arjen On 22-11-2006 22:47 Luke Lonergan wrote: Arjen, As usual, your articles are excellent! Your results show again that the 3Ware 9550SX is really poor at random I/O with RAID5 and all of the Arecas are really good. 3Ware/AMCC have designed the 96xx to do much better for RAID5, but I've not seen results - can you get a card and test it? We now run the 3Ware controllers in RAID10 with 8 disks each and they have been excellent. Here (on your site) are results that bear this out: http://tweakers.net/reviews/639/9 - Luke On 11/22/06 11:07 AM, Arjen van der Meijden [EMAIL PROTECTED] wrote: Jeff, You can find some (Dutch) results here on our website: http://tweakers.net/reviews/647/5 You'll find the AMCC/3ware 9550SX-12 with up to 12 disks, Areca 1280 and 1160 with up to 14 disks and a Promise and LSI sata-raid controller with each up to 8 disks. Btw, that Dell Perc5 (sas) is afaik not the same card as the LSI MegaRAID SATA 300-8X, but I have no idea whether they share the same controllerchip. In most of the graphs you also see a Areca 1160 with 1GB in stead of its default 256MB. Hover over the labels to see only that specific line, that makes the graphs quite readable. You'll also see a Dell Perc5/e in the results, but that was done using Fujitsu SAS 15k rpm drives, not the WD Raptor 10k rpm's If you dive deeper in our (still Dutch) benchmark database you may find some results of several disk-configurations on several controllers in various storage related tests, like here: http://tweakers.net/benchdb/test/193 If you want to filter some results, look for Resultaatfilter tabelgenerator and press on the Toon filteropties-tekst. I think you'll be able to understand the selection-overview there, even if you don't understand Dutch ;) Filter resultaten below means the same as in English (filter [the] results) Best regards, Arjen On 22-11-2006 17:36 Jeff Frost wrote: On Wed, 22 Nov 2006, Bucky Jordan wrote: Dells (at least the 1950 and 2950) come with the Perc5, which is basically just the LSI MegaRAID. The units I have come with a 256MB BBU, I'm not sure if it's upgradeable, but it looks like a standard DIMM in there... I posted some dd and bonnie++ benchmarks of a 6-disk setup a while back on a 2950, so you might search the archive for those numbers if you're interested- you should be able to get the same or better from a similarly equipped LSI setup. I don't recall if I posted pgbench numbers, but I can if that's of interest. I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] availability of SATA vendors
Dells (at least the 1950 and 2950) come with the Perc5, which is basically just the LSI MegaRAID. The units I have come with a 256MB BBU, I'm not sure if it's upgradeable, but it looks like a standard DIMM in there... I posted some dd and bonnie++ benchmarks of a 6-disk setup a while back on a 2950, so you might search the archive for those numbers if you're interested- you should be able to get the same or better from a similarly equipped LSI setup. I don't recall if I posted pgbench numbers, but I can if that's of interest. Dell's probably not the best performance you can get for the money, and if you're not running a supported Linux distro, you might have to do a little cajoling to get support from Dell (though in my experience, it's not too difficult to get them to fix failed hw regardless of the OS), but other than that I haven't had too many issues. HTH, Bucky -Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-performance- [EMAIL PROTECTED] On Behalf Of Joshua D. Drake Sent: Tuesday, November 21, 2006 9:38 PM To: Jeff Frost Cc: Luke Lonergan; pgsql-performance@postgresql.org Subject: Re: [PERFORM] availability of SATA vendors Don't count out LSI either. They make a great SATA controller based off their very well respected SCSI controller. Interesting. Does it perform as well as the ARECAs I don't know if it performs as well as the ARECAs but I can say, I have never had a complaint. and how much BBU cache can you put in it? Yes it support BBU and the max cache depends on the card. 4 drive model, comes with a static 64 megs 8 drive model, comes with a static 128 megs I don't know if they are expandable but keep. Oh, does it use the good ole megaraid_mbox driver as well? Yeah it uses the long standing megaraid, stable as all get out and fast driver :) Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] availability of SATA vendors
On Wed, 22 Nov 2006, Bucky Jordan wrote: Dells (at least the 1950 and 2950) come with the Perc5, which is basically just the LSI MegaRAID. The units I have come with a 256MB BBU, I'm not sure if it's upgradeable, but it looks like a standard DIMM in there... I posted some dd and bonnie++ benchmarks of a 6-disk setup a while back on a 2950, so you might search the archive for those numbers if you're interested- you should be able to get the same or better from a similarly equipped LSI setup. I don't recall if I posted pgbench numbers, but I can if that's of interest. I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? -- Jeff Frost, Owner [EMAIL PROTECTED] Frost Consulting, LLC http://www.frostconsultingllc.com/ Phone: 650-780-7908 FAX: 650-649-1954 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] availability of SATA vendors
On Wed, 2006-11-22 at 08:36 -0800, Jeff Frost wrote: On Wed, 22 Nov 2006, Bucky Jordan wrote: Dells (at least the 1950 and 2950) come with the Perc5, which is basically just the LSI MegaRAID. The units I have come with a 256MB BBU, I'm not sure if it's upgradeable, but it looks like a standard DIMM in there... I posted some dd and bonnie++ benchmarks of a 6-disk setup a while back on a 2950, so you might search the archive for those numbers if you're interested- you should be able to get the same or better from a similarly equipped LSI setup. I don't recall if I posted pgbench numbers, but I can if that's of interest. I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? Why not 6 drive raid 10? IIRC you need 4 to start RAID 10 but only pairs after that. Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] availability of SATA vendors
On Wed, 22 Nov 2006, Joshua D. Drake wrote: I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? Why not 6 drive raid 10? IIRC you need 4 to start RAID 10 but only pairs after that. A valid question. Does the caching raid controller negate the desire to separate pg_xlog from PGDATA? -- Jeff Frost, Owner [EMAIL PROTECTED] Frost Consulting, LLC http://www.frostconsultingllc.com/ Phone: 650-780-7908 FAX: 650-649-1954 ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] availability of SATA vendors
Jeff, You can find some (Dutch) results here on our website: http://tweakers.net/reviews/647/5 You'll find the AMCC/3ware 9550SX-12 with up to 12 disks, Areca 1280 and 1160 with up to 14 disks and a Promise and LSI sata-raid controller with each up to 8 disks. Btw, that Dell Perc5 (sas) is afaik not the same card as the LSI MegaRAID SATA 300-8X, but I have no idea whether they share the same controllerchip. In most of the graphs you also see a Areca 1160 with 1GB in stead of its default 256MB. Hover over the labels to see only that specific line, that makes the graphs quite readable. You'll also see a Dell Perc5/e in the results, but that was done using Fujitsu SAS 15k rpm drives, not the WD Raptor 10k rpm's If you dive deeper in our (still Dutch) benchmark database you may find some results of several disk-configurations on several controllers in various storage related tests, like here: http://tweakers.net/benchdb/test/193 If you want to filter some results, look for Resultaatfilter tabelgenerator and press on the Toon filteropties-tekst. I think you'll be able to understand the selection-overview there, even if you don't understand Dutch ;) Filter resultaten below means the same as in English (filter [the] results) Best regards, Arjen On 22-11-2006 17:36 Jeff Frost wrote: On Wed, 22 Nov 2006, Bucky Jordan wrote: Dells (at least the 1950 and 2950) come with the Perc5, which is basically just the LSI MegaRAID. The units I have come with a 256MB BBU, I'm not sure if it's upgradeable, but it looks like a standard DIMM in there... I posted some dd and bonnie++ benchmarks of a 6-disk setup a while back on a 2950, so you might search the archive for those numbers if you're interested- you should be able to get the same or better from a similarly equipped LSI setup. I don't recall if I posted pgbench numbers, but I can if that's of interest. I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] availability of SATA vendors
On Wed, 2006-11-22 at 09:02 -0800, Jeff Frost wrote: On Wed, 22 Nov 2006, Joshua D. Drake wrote: I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? Why not 6 drive raid 10? IIRC you need 4 to start RAID 10 but only pairs after that. A valid question. Does the caching raid controller negate the desire to separate pg_xlog from PGDATA? There is a point where the seperate pg_xlog does not help with RAID 10. Where that point is, entirely depends on your database usage :) Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] availability of SATA vendors
On Wed, 2006-11-22 at 11:02, Jeff Frost wrote: On Wed, 22 Nov 2006, Joshua D. Drake wrote: I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? Why not 6 drive raid 10? IIRC you need 4 to start RAID 10 but only pairs after that. A valid question. Does the caching raid controller negate the desire to separate pg_xlog from PGDATA? I remember seeing something on the list a while back that having separate file systems was as important as having separate disks / arrays for pg_xlog and PGDATA. Something about the linux on the machine under test being better at ordering of writes if they were to two separate file systems. Of course, the weird thing is how counter intuitive that is, knowing that the heads will have to move from one partition to another on a single disk. but on a multi-disk RAID10, it starts to make sense that the writes to pg_xlog and the writes to data would likely be happening to different drives at once, and so having them be on separate file systems would make it faster if the kernel was better at handling the ordering that way. It's worth looking into at least. Oh, and another vote of confidence for the LSI based controllers. I've had good luck with both the genuine article from LSI and the Dell aftermarket ones. Avoid the Dell - Adaptec controllers like the plague. If you're lucky being slow is the only problem you'll have with those. I'm really hoping to spec out a data warehouse machine here in the next year with lots of drives and an Areca or LSI controller in it... This thread (and all the ones that have come before it) has been most useful and will be archived. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] availability of SATA vendors
I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? Unfortunately most of the tests were run with bonnie 1.9 since they were before I realized that people didn't use the latest version. If I get the chance to run additional tests, I'll post bonnie++ 1.03 numbers. We ended up going with 6xRaid5 since we need as much storage as we can get at the moment. While I'm at it, if I have time I'll run pgbench with pg_log on a separate RAID1, and one with it on a RAID10x6, but I don't know how useful those results will be. Thanks, Bucky ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] availability of SATA vendors
Arjen, As usual, your articles are excellent! Your results show again that the 3Ware 9550SX is really poor at random I/O with RAID5 and all of the Arecas are really good. 3Ware/AMCC have designed the 96xx to do much better for RAID5, but I've not seen results - can you get a card and test it? We now run the 3Ware controllers in RAID10 with 8 disks each and they have been excellent. Here (on your site) are results that bear this out: http://tweakers.net/reviews/639/9 - Luke On 11/22/06 11:07 AM, Arjen van der Meijden [EMAIL PROTECTED] wrote: Jeff, You can find some (Dutch) results here on our website: http://tweakers.net/reviews/647/5 You'll find the AMCC/3ware 9550SX-12 with up to 12 disks, Areca 1280 and 1160 with up to 14 disks and a Promise and LSI sata-raid controller with each up to 8 disks. Btw, that Dell Perc5 (sas) is afaik not the same card as the LSI MegaRAID SATA 300-8X, but I have no idea whether they share the same controllerchip. In most of the graphs you also see a Areca 1160 with 1GB in stead of its default 256MB. Hover over the labels to see only that specific line, that makes the graphs quite readable. You'll also see a Dell Perc5/e in the results, but that was done using Fujitsu SAS 15k rpm drives, not the WD Raptor 10k rpm's If you dive deeper in our (still Dutch) benchmark database you may find some results of several disk-configurations on several controllers in various storage related tests, like here: http://tweakers.net/benchdb/test/193 If you want to filter some results, look for Resultaatfilter tabelgenerator and press on the Toon filteropties-tekst. I think you'll be able to understand the selection-overview there, even if you don't understand Dutch ;) Filter resultaten below means the same as in English (filter [the] results) Best regards, Arjen On 22-11-2006 17:36 Jeff Frost wrote: On Wed, 22 Nov 2006, Bucky Jordan wrote: Dells (at least the 1950 and 2950) come with the Perc5, which is basically just the LSI MegaRAID. The units I have come with a 256MB BBU, I'm not sure if it's upgradeable, but it looks like a standard DIMM in there... I posted some dd and bonnie++ benchmarks of a 6-disk setup a while back on a 2950, so you might search the archive for those numbers if you're interested- you should be able to get the same or better from a similarly equipped LSI setup. I don't recall if I posted pgbench numbers, but I can if that's of interest. I could only find the 6 disk RAID5 numbers in the archives that were run with bonnie++1.03. Have you run the RAID10 tests since? Did you settle on 6 disk RAID5 or 2xRAID1 + 4XRAID10? ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] availability of SATA vendors
On Fri, 17 Nov 2006, Luke Lonergan wrote: Currently, I'm looking at Penguin, HP and Sun (though Sun's store isn't working for me at the moment). Maybe I just need to order a Penguin and then buy the controller separately, but was hoping to get support from a single entity. Rackable or Asacomputers sell and support systems with the 3Ware or Areca controllers. Luke, ASAcomputers has been the most helpful of all the vendors so far, so thanks for point me at them. I know you've been posting results with the Areca and 3ware controllers, do you have a preference for one over the other? It seems that you can only get 256MB cache with the 3ware 9550SX and you can get 512MB with the 9650SE, but only the Areca cards go up to 1GB. I'm curious how big a performance gain we would see going from 256MB cache to 512MB to 1GB. This is for a web site backend DB which is mostly read intensive, but occassionally has large burts of write activity due to new user signups generated by the marketing engine. -- Jeff Frost, Owner [EMAIL PROTECTED] Frost Consulting, LLC http://www.frostconsultingllc.com/ Phone: 650-780-7908 FAX: 650-649-1954 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] availability of SATA vendors
ASAcomputers has been the most helpful of all the vendors so far, so thanks for point me at them. I know you've been posting results with the Areca and 3ware controllers, do you have a preference for one over the other? It seems that you can only get 256MB cache with the 3ware 9550SX and you can get 512MB with the 9650SE, but only the Areca cards go up to 1GB. Don't count out LSI either. They make a great SATA controller based off their very well respected SCSI controller. Sincerely, Joshua D. Drake I'm curious how big a performance gain we would see going from 256MB cache to 512MB to 1GB. This is for a web site backend DB which is mostly read intensive, but occassionally has large burts of write activity due to new user signups generated by the marketing engine. -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] availability of SATA vendors
On Tue, 21 Nov 2006, Joshua D. Drake wrote: ASAcomputers has been the most helpful of all the vendors so far, so thanks for point me at them. I know you've been posting results with the Areca and 3ware controllers, do you have a preference for one over the other? It seems that you can only get 256MB cache with the 3ware 9550SX and you can get 512MB with the 9650SE, but only the Areca cards go up to 1GB. Don't count out LSI either. They make a great SATA controller based off their very well respected SCSI controller. Interesting. Does it perform as well as the ARECAs and how much BBU cache can you put in it? Oh, does it use the good ole megaraid_mbox driver as well? -- Jeff Frost, Owner [EMAIL PROTECTED] Frost Consulting, LLC http://www.frostconsultingllc.com/ Phone: 650-780-7908 FAX: 650-649-1954 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] availability of SATA vendors
On 17-11-2006 18:45 Jeff Frost wrote: I see many of you folks singing the praises of the Areca and 3ware SATA controllers, but I've been trying to price some systems and am having trouble finding a vendor who ships these controllers with their systems. Are you rolling your own white boxes or am I just looking in the wrong places? In Holland it are indeed the smaller companies who supply such cards. But luckily there is a very simple solution, all those big suppliers do supply SAS-controllers. And as you may know, SATA disks can be used without any problem on a SAS controller. Of course they are less advanced and normally slower than a SAS disk. So you can have a nice SAS raid card and insert SATA disks in it. And than you can shop at any major server vendor I know off. Good luck, Arjen ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] availability of SATA vendors
Contact Pogo Linux, www.pogolinux.com. I believe they OEM and VAR both Areca and 3ware in their systems. 3ware was bought out by AMCC, www.amcc.com Areca cards are distributed in NA by Tekram, www.tekram.com, and are available from them as solo items as well as in OEM storage systems. Areca main offices are in Taiwan. Ron At 12:45 PM 11/17/2006, Jeff Frost wrote: I see many of you folks singing the praises of the Areca and 3ware SATA controllers, but I've been trying to price some systems and am having trouble finding a vendor who ships these controllers with their systems. Are you rolling your own white boxes or am I just looking in the wrong places? Currently, I'm looking at Penguin, HP and Sun (though Sun's store isn't working for me at the moment). Maybe I just need to order a Penguin and then buy the controller separately, but was hoping to get support from a single entity. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] availability of SATA vendors
On Nov 17, 2006, at 9:45 AM, Jeff Frost wrote: I see many of you folks singing the praises of the Areca and 3ware SATA controllers, but I've been trying to price some systems and am having trouble finding a vendor who ships these controllers with their systems. Are you rolling your own white boxes or am I just looking in the wrong places? Currently, I'm looking at Penguin, HP and Sun (though Sun's store isn't working for me at the moment). Maybe I just need to order a Penguin and then buy the controller separately, but was hoping to get support from a single entity. I bought my last system pre-built from asacomputers.com, with the 3ware controller in it. I don't recall if the controller was listed on the webpage or not - I wrote their sales address and asked for a quote on one like that, with a 9550sx controller. (one like that was their 5U, Opteron, 24 bay box.) Cheers, Steve ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] availability of SATA vendors
Jeff, On 11/17/06 11:45 AM, Jeff Frost [EMAIL PROTECTED] wrote: I see many of you folks singing the praises of the Areca and 3ware SATA controllers, but I've been trying to price some systems and am having trouble finding a vendor who ships these controllers with their systems. Are you rolling your own white boxes or am I just looking in the wrong places? Currently, I'm looking at Penguin, HP and Sun (though Sun's store isn't working for me at the moment). Maybe I just need to order a Penguin and then buy the controller separately, but was hoping to get support from a single entity. Rackable or Asacomputers sell and support systems with the 3Ware or Areca controllers. - Luke ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
