Re: Optimal File System config for 2.5TB RAID5
The 3ware 9690SA outperforms gmirror and can be had in 4 port with the did you made tests comparing it with gmirror with the same config? battery for $600 or so. 8 port with a battery is closer to $1000 Hardware RAID gets you boot support from stripes, already said what should be done. email alerts for RAID smartmontools... of course - if someone is happy spending money for such things - it's OK, their manufacturers are happy too. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
What you're asking for is too much -- and this conversation is starting to delve into freebsd-hardware, not freebsd-questions. the simple answer is that software RAID on todays computers vastly outperforms ANY hardware raid solution, maybe except the ones for 1$ or more. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
Ok, I have to pickup gVinum where I left it 4 years ago. Hopefully, the software is stable now. AFAIK it's not at least when i tried it in 6.* ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Wojciech Puchar wrote: | | What you're asking for is too much -- and this conversation is | starting to delve into freebsd-hardware, not freebsd-questions. | | the simple answer is that software RAID on todays computers vastly | outperforms ANY hardware raid solution, maybe except the ones for 1$ | or more. You're basically correct, but I think you're overestimating the price of a good RAID controller. The big win with hardware RAID controllers comes when they are fitted with a BBU. Typically this turns a £400 item into a £600 item, so most sales weasels will artfully forget to include it[+]. When you've got a BBU, then it lets you do the good stuff like disable write cache on the drives but *enable* it on the controller -- the presence of the battery means that data cached in RAM is safe and in the event of an unscheduled reboot, it can be flushed to disk as the system comes up again -- so the RAID can justifiably report to the OS that the IO transaction is complete without having to wait for the bits to actually hit the disk platter[*]. I've occasionally wondered why there isn't a simple device commonly available which consists of a few hundred MB of battery backed (or otherwise persistent in the face of power loss) RAM that can plug into a PCI slot and fulfil that function generically for any disks in a machine. Solid state hard drives are getting there, but they're still too expensive and not really fast enough yet. Cheers, Matthew [+] And I don't know how the manufacturers justify that price tag, as the battery tech used is based on the same off-the-shelf components that go into any mobile phone [*] Unlike the normal hw.ata.wc enable, which reports this unjustifiably... - -- Dr Matthew J Seaman MA, D.Phil. Flat 3 ~ 7 Priory Courtyard PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate ~ Kent, CT11 9PW, UK -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREDAAYFAkjjPMYACgkQ3jDkPpsZ+VYBGwCgmIndkiis5+OfA8ahXCbTasxO pbkAn2y69JagZweBpD62TnctqtQdt+mF =yHzW -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
On Wed, Oct 01, 2008 at 10:03:02AM +0100, Matthew Seaman wrote: I've occasionally wondered why there isn't a simple device commonly available which consists of a few hundred MB of battery backed (or otherwise persistent in the face of power loss) RAM that can plug into a PCI slot and fulfil that function generically for any disks in a machine. Solid state hard drives are getting there, but they're still too expensive and not really fast enough yet. You mean this? http://en.wikipedia.org/wiki/I-RAM -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
| | the simple answer is that software RAID on todays computers vastly | outperforms ANY hardware raid solution, maybe except the ones for 1$ | or more. You're basically correct, but I think you're overestimating the price of a good RAID controller. no. please give me example of any RAID hardware below 1$ that WILL be faster than properly configure software RAID solution under FreeBSD with same amount of same disks. i mean faster under normal unix-like load, which is lots of parallel accesses to same or different things, not simple tests. there are NONE. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Jeremy Chadwick wrote: | On Wed, Oct 01, 2008 at 10:03:02AM +0100, Matthew Seaman wrote: | I've occasionally wondered why there isn't a simple device commonly available | which consists of a few hundred MB of battery backed (or otherwise persistent | in the face of power loss) RAM that can plug into a PCI slot and fulfil that | function generically for any disks in a machine. Solid state hard drives are | getting there, but they're still too expensive and not really fast enough yet. | | You mean this? | | http://en.wikipedia.org/wiki/I-RAM | Yeah -- pretty much that. Although I'd prefer something that doesn't try and pretend to be a hard drive -- after all, there's no reason to limit the thing to the performance envelope of a SATA bus. If you want solid state drives, nowadays there are devices based on compact flash that fit in a normal 3.5 drive bay, speak SATA and don't need standby power. Latency is much better than a normal HDD, but probably not so good as the I-RAM you pointed out. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. Flat 3 ~ 7 Priory Courtyard PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate ~ Kent, CT11 9PW, UK -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREDAAYFAkjjS0oACgkQ3jDkPpsZ+VaS5ACePK+3KUc+Kiu612I0JTqWJz9h WgoAmgNqC3/tNCmcfwogmFJZqs6gP4zF =nj5y -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Wojciech Puchar wrote: | | the simple answer is that software RAID on todays computers vastly | outperforms ANY hardware raid solution, maybe except the ones for 1$ | or more. You're basically correct, but I think you're overestimating the price of a good RAID controller. no. please give me example of any RAID hardware below 1$ that WILL be faster than properly configure software RAID solution under FreeBSD with same amount of same disks. i mean faster under normal unix-like load, which is lots of parallel accesses to same or different things, not simple tests. there are NONE. I have a number of systems running postgresql + a python web application that see fairly heavy concurrent access. The 3ware 9690SA outperforms gmirror and can be had in 4 port with the battery for $600 or so. 8 port with a battery is closer to $1000 Hardware RAID gets you boot support from stripes, email alerts for RAID events in many cases, and with a battery the option to turn write caching on on the controller. Unfortunately there are a number of bad and/or poorly supported RAID controllers out there, especially on FreeBSD. I'd never suggest to anyone that my highpoint 2300 or LSI 3041R-E are high performance, but on the other hand, at real world tasks like a database that backs up a webserver doing millions of hits a day, the LSI 320-2E does RAID 10 faster than gvinum, and my 3ware 9690SA's are faster than gmirror at RAID 1, plus offer the option for a warm spare. Software RAID has advantages, namely hardware independance. And it can be faster than low end hardware RAID. But you don't have to spend much to get a hardware solution that will smoke software RAID at real world applications. - -- Thanks, Josh Paetzel PGP: 8A48 EF36 5E9F 4EDA 5ABC 11B4 26F9 01F1 27AF AECB -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) iD8DBQFI5EbNJvkB8SevrssRAvdLAJ96CLUVK3M2YLKNmAxmIPlxoqp+fgCgnLp6 H1OgEHevOKqDJ/FRg5+fHpU= =MQFo -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Optimal File System config for 2.5TB RAID5
Hello, I am building a 6x500GB SATA HARDWARE RAID5 storage server to - Store large files, 10BM~1GB/file - Handling 500+ concurrent connections - Transfer rate around 100~200Mbit/s I am thinking of using the patch from Wojciech Puchar to reduce hard drive data seek in order to handle large number of concurrent connections whilst outputting 100~200Mbit/s. patch /usr/src/sys/sys/param.h #ifndef DFLTPHYS #define DFLTPHYS(1024 * 1024) /* default max raw I/O transfer size */ #endif #ifndef MAXPHYS #define MAXPHYS (1024 * 1024) /* max raw I/O transfer size */ #endif #ifndef MAXDUMPPGS To store files greater than 10MB, I come up with the following proposal for my File System: - UFS2 - Soft Update Enable - block-size 1,048,576 I am not completely sure what advantage I got from this configuration but I am pretty sure that FSCK is much quicker with 1M file system block-size. Is there any other thing I need to consider in term of performance and reliability? I hope that this system will perform much better than my current 6x300GB SCSI 10K RPM system. Appreciate any advice, Danny ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Danny Do wrote: Hello, I am building a 6x500GB SATA HARDWARE RAID5 storage server to - Store large files, 10BM~1GB/file - Handling 500+ concurrent connections - Transfer rate around 100~200Mbit/s I am thinking of using the patch from Wojciech Puchar to reduce hard drive data seek in order to handle large number of concurrent connections whilst outputting 100~200Mbit/s. patch /usr/src/sys/sys/param.h #ifndef DFLTPHYS #define DFLTPHYS(1024 * 1024) /* default max raw I/O transfer size */ #endif #ifndef MAXPHYS #define MAXPHYS (1024 * 1024) /* max raw I/O transfer size */ #endif #ifndef MAXDUMPPGS To store files greater than 10MB, I come up with the following proposal for my File System: - UFS2 - Soft Update Enable - block-size 1,048,576 I am not completely sure what advantage I got from this configuration but I am pretty sure that FSCK is much quicker with 1M file system block-size. Is there any other thing I need to consider in term of performance and reliability? I hope that this system will perform much better than my current 6x300GB SCSI 10K RPM system. Appreciate any advice, Danny Why do you think slower drives using an interface that has known problems handling concurrent connections will be faster than faster drives using an interface designed for concurrency? Based on my experiences with SATA vs. U160/U320 SCSI or SAS your likely outcome is to see a marked decrease in performance. I'd be interested to hear your results. - -- Thanks, Josh Paetzel PGP: 8A48 EF36 5E9F 4EDA 5ABC 11B4 26F9 01F1 27AF AECB -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) iD8DBQFI4hxmJvkB8SevrssRAqErAJ0Tt9WPT25RhkUfGVLxEzSykEMvtwCeKXRV jdgJ/whLeeAQ3E97i7FkB4w= =UyD6 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
On Tue, Sep 30, 2008 at 7:32 AM, Josh Paetzel [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Danny Do wrote: Hello, I am building a 6x500GB SATA HARDWARE RAID5 storage server to - Store large files, 10BM~1GB/file - Handling 500+ concurrent connections - Transfer rate around 100~200Mbit/s I am thinking of using the patch from Wojciech Puchar to reduce hard drive data seek in order to handle large number of concurrent connections whilst outputting 100~200Mbit/s. patch /usr/src/sys/sys/param.h #ifndef DFLTPHYS #define DFLTPHYS(1024 * 1024) /* default max raw I/O transfer size */ #endif #ifndef MAXPHYS #define MAXPHYS (1024 * 1024) /* max raw I/O transfer size */ #endif #ifndef MAXDUMPPGS To store files greater than 10MB, I come up with the following proposal for my File System: - UFS2 - Soft Update Enable - block-size 1,048,576 I am not completely sure what advantage I got from this configuration but I am pretty sure that FSCK is much quicker with 1M file system block-size. Is there any other thing I need to consider in term of performance and reliability? I hope that this system will perform much better than my current 6x300GB SCSI 10K RPM system. Appreciate any advice, Danny Why do you think slower drives using an interface that has known problems handling concurrent connections will be faster than faster drives using an interface designed for concurrency? Based on my experiences with SATA vs. U160/U320 SCSI or SAS your likely outcome is to see a marked decrease in performance. I'd be interested to hear your results. - -- Thanks, Josh Paetzel PGP: 8A48 EF36 5E9F 4EDA 5ABC 11B4 26F9 01F1 27AF AECB -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) iD8DBQFI4hxmJvkB8SevrssRAqErAJ0Tt9WPT25RhkUfGVLxEzSykEMvtwCeKXRV jdgJ/whLeeAQ3E97i7FkB4w= =UyD6 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] Interface concurrent connection problems?, do you have a link or something? actually i recommend again the RAID 10, if you want performance for heavy I/O (multiple reading,not only one file lineal reading) for storage intensive apps its the way to go. -- mmm, interesante. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
Why do you think slower drives using an interface that has known problems handling concurrent connections will be faster than faster drives using an interface designed for concurrency? My current 6x300GB SCSI system using the FreeBSD default max raw I/O transfer size (64KB). Assume that all reads are random. In order to read 1MB from the hard drive, it would cost: - 1024/64 * (seek time + time to read 64K) - 16 * (8ms + 1ms) [average seek time on SATA 7200RPM is 8ms, make it 0ms for read time] - 128ms to read 1MB If I change the default max raw I/O transfer size to 1MB it would only cost (8ms seek time + 2.6ms read 1MB using SATA300). So, the time to read 1MB is only about 10.6ms. As we can see here reading 1MB from the hard disk is at least 10 times better if we increase the default max raw I/O transfer size to 1MB. This is mainly because the main cost for reading random data from hard disk is seek time. I think the drawback from such configuration is that our server will consume at least: - n concurrent connections * default max raw I/O transfer size of memory just for reading the data from hard disk. RAM quite cheap these days, I think it's ok. Based on my experiences with SATA vs. U160/U320 SCSI or SAS your likely outcome is to see a marked decrease in performance. I'd be interested to hear your results. If both SATA and SCSI system using the same configuration, the end result should be obvious. However, If SCSI system using 64K IO transfer size whilst SATA using 1MB IO transfer size, I don't know! I think the SATA system will outperform the SCSI system. I'll let you know when I get the new SATA system from my ISP. Cheers, Danny ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
SATA using 1MB IO transfer size, I don't know! I think the SATA system will SATA drives aren't much slower than SCSI. simply make this 1MB IO transfer size. as you still want hardware RAID5 it looks you simply read maybe every second word from my mails we exchanged privately. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
Hi Wojciech Puchar, The reason I want to use hardware RAID is because I got so much problem with software RAID5 4 years ago on FreeBSD 5.4. I still remember those nightmares. Furthermore, hardware RAID5 doesn't require much knowledge and management. But you could be right, the CPU speed is triple now, software RAID gets smarter and more stable, it could perform better than hardware RAID because it's more flexible. But again, I still prefer hardware because it's easy to use and easy to manage. Thanks all the tips Wojciech Puchar, Danny -Original Message- From: Wojciech Puchar [mailto:[EMAIL PROTECTED] Sent: Tuesday, 30 September 2008 9:16 PM To: Danny Do Cc: 'Josh Paetzel'; freebsd-questions@freebsd.org Subject: RE: Optimal File System config for 2.5TB RAID5 SATA using 1MB IO transfer size, I don't know! I think the SATA system will SATA drives aren't much slower than SCSI. simply make this 1MB IO transfer size. as you still want hardware RAID5 it looks you simply read maybe every second word from my mails we exchanged privately. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
The reason I want to use hardware RAID is because I got so much problem with software RAID5 4 years ago on FreeBSD 5.4. I still remember those nightmares. Furthermore, hardware RAID5 doesn't require much knowledge and management. But you could be right, the CPU speed is triple now, software RAID gets smarter and more stable, it could perform better than hardware RAID because it's more flexible. But again, I still prefer hardware because it's easy to it's someone more than that, still - you don't read my mails carefully. you will get better performance with my patch, but still it will be crappy with your hardware RAIDs compared to what it should be ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
Hi Wojciech Puchar, I got Perc 4E-DI Embedded Raid Adapter (256MB) from DELL for my current SCSI system. They said it's the enterprise class. I don't know much about the performance between software RAID and hardware RAID. Could you please tell me if this type of hardware RAID controller could match the software RAID you were talking about? I don't want to pay for it if I don't really need it. Thanks, Danny -Original Message- From: Wojciech Puchar [mailto:[EMAIL PROTECTED] Sent: Wednesday, 1 October 2008 1:56 AM To: Danny Do Cc: freebsd-questions@freebsd.org Subject: RE: Optimal File System config for 2.5TB RAID5 The reason I want to use hardware RAID is because I got so much problem with software RAID5 4 years ago on FreeBSD 5.4. I still remember those nightmares. Furthermore, hardware RAID5 doesn't require much knowledge and management. But you could be right, the CPU speed is triple now, software RAID gets smarter and more stable, it could perform better than hardware RAID because it's more flexible. But again, I still prefer hardware because it's easy to it's someone more than that, still - you don't read my mails carefully. you will get better performance with my patch, but still it will be crappy with your hardware RAIDs compared to what it should be ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
On Wed, Oct 01, 2008 at 04:49:27AM +0700, Danny Do wrote: I got Perc 4E-DI Embedded Raid Adapter (256MB) from DELL for my current SCSI system. They said it's the enterprise class. I don't know much about the performance between software RAID and hardware RAID. I'm not familiar with PERC (LSI) controllers, just for the record. Could you please tell me if this type of hardware RAID controller could match the software RAID you were talking about? What you're asking for is too much -- and this conversation is starting to delve into freebsd-hardware, not freebsd-questions. Unless someone out there has done full benchmarks comparing FreeBSD ZFS or FreeBSD gvinum to a PERC 4E-DI, with all kinds of test cases (what sort of server it is, what it's doing disk-wise, etc.), I doubt you'll be able to get a conclusive answer here. Such benchmarking would require weeks of effort by someone. Heck, I'm not even sure FreeBSD supports the PERC 4E-DI. That said, if you go with that controller, you should be aware of the following things: there are many problems with hardware RAID. 1) If the controller goes bad after the lifetime of the controller has expired, there is very little chance the vendor will give you a replacement controller that understands the metadata of the previous/bad controller. You are flat out stuck with that model of controller for the rest of your life, unless the vendor can *guarantee* backwards compatibility when providing a newer controller. And I'm willing to bet money that general technical support has no idea what metadata is, or any technical details; they just know what they're told (controller X is no longer available, give them controller Y) 2) Driver support is often iffy with such controllers, at least under FreeBSD. FreeBSD SCSI CAM is quite reliable, so that's not the problem. Here's some past evidence of mfi(4) and mpt(4) having problems administrating arrays, or experiencing horrible performance, requiring tuning be done and much troubleshooting: http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues 3) You are at the whim of the hardware RAID controller's BIOS. Performance can be affected by bugs in the BIOS, or BIOS bugs can cause you trouble down the road. You have to ask yourself how much you ultimately trust the technical support people at Dell vs. the FreeBSD community. 4) Driver regressions may hurt you. There may be a day when you go to upgrade to FreeBSD 8.0 (when it becomes stable), only to find that your controller isn't recognised, or has odd problems. (I myself just ran into this situation with -CURRENT last week, where my SATA controller isn't detected, while works perfectly in RELENG_7). You're then stuck on an older FreeBSD until those problems can be worked out. The only hardware RAID controller I've seen praise for, under FreeBSD, are Areca controllers. I'm told the performance (on a purely general level) is absolutely incredible/blazing fast. I don't know what those people are comparing against, though. Be aware that many developers, including folks like Matt Dillon (of DragonflyBSD) and Ade Lovett (very familiar with filers and disk storage) recommend you *completely avoid* hardware RAID controllers or on-motherboard RAID (e.g. Intel MatrixRAID), and go with OS-based RAID (ZFS, gvinum, or standalone UFS2+SU filesystems). If you reach a point where disk I/O on that server is becoming so heavy that you feel you need a hardware RAID controller, that would be when you should come back to the list (freebsd-stable, freebsd-hardware, or freebsd-isp) to discuss the problems you're having with performance. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Optimal File System config for 2.5TB RAID5
On Tuesday, September 30, 2008, at 02:44PM, Jeremy Chadwick [EMAIL PROTECTED] wrote: The only hardware RAID controller I've seen praise for, under FreeBSD, are Areca controllers. 3ware has provided very good FreeBSD support as well. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Optimal File System config for 2.5TB RAID5
Thanks for the concrete example of the pitfall of hardware RAID Jeremy. I never had any problem with hardware driver, that's why I never thought of it. But you are quite right! I should be avoiding hardware RAID whenever possible. I get much more support and quicker response here than from hardware vendor. Ok, I have to pickup gVinum where I left it 4 years ago. Hopefully, the software is stable now. Thanks again Jeremy Chadwick, Wojciech Puchar and this wonderful community, Danny -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jeremy Chadwick Sent: Wednesday, 1 October 2008 5:45 AM To: Danny Do Cc: 'Wojciech Puchar'; freebsd-questions@freebsd.org Subject: Re: Optimal File System config for 2.5TB RAID5 On Wed, Oct 01, 2008 at 04:49:27AM +0700, Danny Do wrote: I got Perc 4E-DI Embedded Raid Adapter (256MB) from DELL for my current SCSI system. They said it's the enterprise class. I don't know much about the performance between software RAID and hardware RAID. I'm not familiar with PERC (LSI) controllers, just for the record. Could you please tell me if this type of hardware RAID controller could match the software RAID you were talking about? What you're asking for is too much -- and this conversation is starting to delve into freebsd-hardware, not freebsd-questions. Unless someone out there has done full benchmarks comparing FreeBSD ZFS or FreeBSD gvinum to a PERC 4E-DI, with all kinds of test cases (what sort of server it is, what it's doing disk-wise, etc.), I doubt you'll be able to get a conclusive answer here. Such benchmarking would require weeks of effort by someone. Heck, I'm not even sure FreeBSD supports the PERC 4E-DI. That said, if you go with that controller, you should be aware of the following things: there are many problems with hardware RAID. 1) If the controller goes bad after the lifetime of the controller has expired, there is very little chance the vendor will give you a replacement controller that understands the metadata of the previous/bad controller. You are flat out stuck with that model of controller for the rest of your life, unless the vendor can *guarantee* backwards compatibility when providing a newer controller. And I'm willing to bet money that general technical support has no idea what metadata is, or any technical details; they just know what they're told (controller X is no longer available, give them controller Y) 2) Driver support is often iffy with such controllers, at least under FreeBSD. FreeBSD SCSI CAM is quite reliable, so that's not the problem. Here's some past evidence of mfi(4) and mpt(4) having problems administrating arrays, or experiencing horrible performance, requiring tuning be done and much troubleshooting: http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues 3) You are at the whim of the hardware RAID controller's BIOS. Performance can be affected by bugs in the BIOS, or BIOS bugs can cause you trouble down the road. You have to ask yourself how much you ultimately trust the technical support people at Dell vs. the FreeBSD community. 4) Driver regressions may hurt you. There may be a day when you go to upgrade to FreeBSD 8.0 (when it becomes stable), only to find that your controller isn't recognised, or has odd problems. (I myself just ran into this situation with -CURRENT last week, where my SATA controller isn't detected, while works perfectly in RELENG_7). You're then stuck on an older FreeBSD until those problems can be worked out. The only hardware RAID controller I've seen praise for, under FreeBSD, are Areca controllers. I'm told the performance (on a purely general level) is absolutely incredible/blazing fast. I don't know what those people are comparing against, though. Be aware that many developers, including folks like Matt Dillon (of DragonflyBSD) and Ade Lovett (very familiar with filers and disk storage) recommend you *completely avoid* hardware RAID controllers or on-motherboard RAID (e.g. Intel MatrixRAID), and go with OS-based RAID (ZFS, gvinum, or standalone UFS2+SU filesystems). If you reach a point where disk I/O on that server is becoming so heavy that you feel you need a hardware RAID controller, that would be when you should come back to the list (freebsd-stable, freebsd-hardware, or freebsd-isp) to discuss the problems you're having with performance. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http
