Re: [PERFORM] Hardware vs Software RAID

2008-06-27 Thread Merlin Moncure
On Fri, Jun 27, 2008 at 7:00 AM, Matthew Wakeling <[EMAIL PROTECTED]> wrote: > On Thu, 26 Jun 2008, Merlin Moncure wrote: >> >> In addition there are many different types of flash (MLC/SLC) and the >> flash cells themselves can be organized in particular ways involving various >> trade-offs. > > Ye

Re: [PERFORM] Hardware vs Software RAID

2008-06-27 Thread Matthew Wakeling
On Thu, 26 Jun 2008, Merlin Moncure wrote: In addition there are many different types of flash (MLC/SLC) and the flash cells themselves can be organized in particular ways involving various trade-offs. Yeah, I wouldn't go for MLC, given it has a tenth the lifespan of SLC. The main issue is lo

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Robert Treat
On Wednesday 25 June 2008 11:24:23 Greg Smith wrote: > What I often do is get a hardware RAID controller, just to accelerate disk > writes, but configure it in JBOD mode and use Linux or other software RAID > on that platform. > JBOD + RAIDZ2 FTW ;-) -- Robert Treat Build A Brighter LAMP :: Linu

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread Greg Smith
On Thu, 26 Jun 2008, Peter T. Breuer wrote: Double buffering is a killer. No, it isn't; it's a completely trivial bit of overhead. It only exists during the time when blocks are queued to write but haven't been written yet. On any database system, in those cases I/O congestion at the disk

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread david
On Thu, 26 Jun 2008, Peter T. Breuer wrote: "Also sprach Merlin Moncure:" The linux software raid algorithms are highly optimized, and run on a I can confidently tell you that that's balderdash both as a Linux author and as a software RAID linux author (check the attributions in the kernel so

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2008 at 1:03 AM, Peter T. Breuer <[EMAIL PROTECTED]> wrote: > "Also sprach Merlin Moncure:" >> write back: raid controller can lie to host o/s. when o/s asks > > This is not what the linux software raid controller does, then. It > does not queue requests internally at all, nor ack r

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread Peter T. Breuer
"Also sprach Merlin Moncure:" > write back: raid controller can lie to host o/s. when o/s asks This is not what the linux software raid controller does, then. It does not queue requests internally at all, nor ack requests that have not already been acked by the components (modulo the fact that on

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2008 at 12:14 PM, Matthew Wakeling <[EMAIL PROTECTED]> wrote: >> None of these manufacturers rates these drives for massive amounts of >> writes. They're sold as suitable for laptop/desktop use, which normally is >> not a heavy wear and tear operation like a DB. Once they claim sui

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2008 at 9:49 AM, Peter T. Breuer <[EMAIL PROTECTED]> wrote: > "Also sprach Merlin Moncure:" >> As discussed down thread, software raid still gets benefits of >> write-back caching on the raid controller...but there are a couple of > > (I wish I knew what write-back caching was!) ha

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Scott Marlowe
On Thu, Jun 26, 2008 at 10:14 AM, Matthew Wakeling <[EMAIL PROTECTED]> wrote: > On Thu, 26 Jun 2008, Vivek Khera wrote: >>> >>> Anyone have experience with IDE, SATA, or SAS-connected flash devices >>> like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a >>> transfer rate of 100M

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Matthew Wakeling
On Thu, 26 Jun 2008, Vivek Khera wrote: Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a transfer rate of 100MB/s, and doesn't degrade much in performance when writing small random blocks. But what's it

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Vivek Khera
On Jun 25, 2008, at 11:35 AM, Matthew Wakeling wrote: On Wed, 25 Jun 2008, Greg Smith wrote: A firewire-attached log device is an extremely bad idea. Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, a

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Greg Smith
On Wed, 25 Jun 2008, Andrew Sullivan wrote: the key thing to do is to ensure you have good testing infrastructure in place to check that things will work before you deploy to production. This is true whether you're using Linux or completely closed source software. There are two main differen

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Merlin Moncure wrote: So, based on this and other experiences, i'm starting to be more partial to linux distributions with faster moving kernels, mainly because i trust the kernel drivers more than the vendor provided drivers. Depends on how fast. I find it takes a minim

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Peter T. Breuer wrote: I refrained from saying in my reply that I would set up a firewire-based link to ram in a spare old portable (which comes with a battery) if I wanted to do this cheaply. Maybe, but this is kind of a weird setup. Not many people are going to run a p

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
On Wed, Jun 25, 2008 at 01:07:25PM -0500, Kevin Grittner wrote: > > It doesn't have to be free software to cut that way. I've actually > found the free software to waste less of my time. No question. But one of the unfortunate facts of the no-charge-for-licenses world is that many people exp

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Kevin Grittner
>>> Andrew Sullivan <[EMAIL PROTECTED]> wrote: > this is a blade that cuts both ways, and the key thing to do is > to ensure you have good testing infrastructure in place to check that > things will work before you deploy to production. (The other way to > say that, of course, is "Linux is only

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
On Wed, Jun 25, 2008 at 01:35:49PM -0400, Merlin Moncure wrote: > experiences, i'm starting to be more partial to linux distributions > with faster moving kernels, mainly because i trust the kernel drivers > more than the vendor provided drivers. While I have some experience that agrees with this,

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 9:03 AM, Matthew Wakeling <[EMAIL PROTECTED]> wrote: > On Wed, 25 Jun 2008, Merlin Moncure wrote: >>> >>> Has anyone done some benchmarks between hardware RAID vs Linux MD >>> software >>> RAID? >> >> I have here: >> >> http://merlinmoncure.blogspot.com/2007/08/following-are

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 11:55 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: > On Wed, 2008-06-25 at 09:53 -0600, Scott Marlowe wrote: >> On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey >> <[EMAIL PROTECTED]> wrote: > >> I'm currently having a problem with a "well known very large >> servermanufactu

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Peter T. Breuer
"Also sprach Greg Smith:" > On Wed, 25 Jun 2008, Peter T. Breuer wrote: > > > You can put the log/bitmap wherever you want in software raid, including > > on a battery-backed local ram disk if you feel so inclined. So there is > > no intrinsic advantage to be gained there at all. > > You are t

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Joshua D. Drake
On Wed, 2008-06-25 at 09:53 -0600, Scott Marlowe wrote: > On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey > <[EMAIL PROTECTED]> wrote: > > Hi > I'm currently having a problem with a "well known very large > servermanufacturer who shall remain unnamed" and their semi-custom > RAID controller firmw

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Scott Marlowe
On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey <[EMAIL PROTECTED]> wrote: > Hi > > Has anyone done some benchmarks between hardware RAID vs Linux MD software > RAID? > > I'm keen to know the result. I've had good performance from sw RAID-10 in later kernels, especially if it was handling a mostly

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Greg Smith wrote: A firewire-attached log device is an extremely bad idea. Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a transfer rate of 100MB/s, and doesn't degrade much in p

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Joshua D. Drake
On Wed, 2008-06-25 at 11:30 -0400, Jonah H. Harris wrote: > On Wed, Jun 25, 2008 at 11:24 AM, Greg Smith <[EMAIL PROTECTED]> wrote: > > SMART doesn't detect 100% of drive failures in advance, but you'd be silly > > to setup a database system where you don't get to take advantage of the > > ~50% i

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Jonah H. Harris
On Wed, Jun 25, 2008 at 11:24 AM, Greg Smith <[EMAIL PROTECTED]> wrote: > SMART doesn't detect 100% of drive failures in advance, but you'd be silly > to setup a database system where you don't get to take advantage of the > ~50% it does catch before you lose any data. Can't argue with that one.

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Peter T. Breuer wrote: You can put the log/bitmap wherever you want in software raid, including on a battery-backed local ram disk if you feel so inclined. So there is no intrinsic advantage to be gained there at all. You are technically correct but this is irrelevant.

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Peter T. Breuer
"Also sprach Matthew Wakeling:" > >> Has anyone done some benchmarks between hardware RAID vs Linux MD software > >> RAID? ... > > The upshot is I don't really see a difference in performance. > > The main difference is that you can get hardware RAID with > battery-backed-up cache, which means

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Merlin Moncure wrote: Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? I have here: http://merlinmoncure.blogspot.com/2007/08/following-are-results-of-our-testing-of.html The upshot is I don't really see a difference in performance. The ma

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 7:05 AM, Adrian Moisey <[EMAIL PROTECTED]> wrote: > Has anyone done some benchmarks between hardware RAID vs Linux MD software > RAID? > > I'm keen to know the result. I have here: http://merlinmoncure.blogspot.com/2007/08/following-are-results-of-our-testing-of.html I als

[PERFORM] Hardware vs Software RAID

2008-06-25 Thread Adrian Moisey
Hi Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? I'm keen to know the result. -- Adrian Moisey Systems Administrator | CareerJunction | Your Future Starts Here. Web: www.careerjunction.co.za | Email: [EMAIL PROTECTED] Phone: +27 21 686 6820 | Mobile: +27 82