Re: [PERFORM] Hardware vs Software RAID

2008-06-27 Thread Matthew Wakeling
On Thu, 26 Jun 2008, Merlin Moncure wrote: In addition there are many different types of flash (MLC/SLC) and the flash cells themselves can be organized in particular ways involving various trade-offs. Yeah, I wouldn't go for MLC, given it has a tenth the lifespan of SLC. The main issue is

Re: [PERFORM] Hardware vs Software RAID

2008-06-27 Thread Merlin Moncure
On Fri, Jun 27, 2008 at 7:00 AM, Matthew Wakeling [EMAIL PROTECTED] wrote: On Thu, 26 Jun 2008, Merlin Moncure wrote: In addition there are many different types of flash (MLC/SLC) and the flash cells themselves can be organized in particular ways involving various trade-offs. Yeah, I

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Greg Smith
On Wed, 25 Jun 2008, Andrew Sullivan wrote: the key thing to do is to ensure you have good testing infrastructure in place to check that things will work before you deploy to production. This is true whether you're using Linux or completely closed source software. There are two main

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Vivek Khera
On Jun 25, 2008, at 11:35 AM, Matthew Wakeling wrote: On Wed, 25 Jun 2008, Greg Smith wrote: A firewire-attached log device is an extremely bad idea. Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB,

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Matthew Wakeling
On Thu, 26 Jun 2008, Vivek Khera wrote: Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a transfer rate of 100MB/s, and doesn't degrade much in performance when writing small random blocks. But what's

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Scott Marlowe
On Thu, Jun 26, 2008 at 10:14 AM, Matthew Wakeling [EMAIL PROTECTED] wrote: On Thu, 26 Jun 2008, Vivek Khera wrote: Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a transfer rate of 100MB/s, and

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2008 at 9:49 AM, Peter T. Breuer [EMAIL PROTECTED] wrote: Also sprach Merlin Moncure: As discussed down thread, software raid still gets benefits of write-back caching on the raid controller...but there are a couple of (I wish I knew what write-back caching was!) hardware

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2008 at 12:14 PM, Matthew Wakeling [EMAIL PROTECTED] wrote: None of these manufacturers rates these drives for massive amounts of writes. They're sold as suitable for laptop/desktop use, which normally is not a heavy wear and tear operation like a DB. Once they claim

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread Peter T. Breuer
Also sprach Merlin Moncure: write back: raid controller can lie to host o/s. when o/s asks This is not what the linux software raid controller does, then. It does not queue requests internally at all, nor ack requests that have not already been acked by the components (modulo the fact that one

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread Merlin Moncure
On Thu, Jun 26, 2008 at 1:03 AM, Peter T. Breuer [EMAIL PROTECTED] wrote: Also sprach Merlin Moncure: write back: raid controller can lie to host o/s. when o/s asks This is not what the linux software raid controller does, then. It does not queue requests internally at all, nor ack requests

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread david
On Thu, 26 Jun 2008, Peter T. Breuer wrote: Also sprach Merlin Moncure: The linux software raid algorithms are highly optimized, and run on a I can confidently tell you that that's balderdash both as a Linux author and as a software RAID linux author (check the attributions in the kernel

Re: [PERFORM] Hardware vs Software Raid

2008-06-26 Thread Greg Smith
On Thu, 26 Jun 2008, Peter T. Breuer wrote: Double buffering is a killer. No, it isn't; it's a completely trivial bit of overhead. It only exists during the time when blocks are queued to write but haven't been written yet. On any database system, in those cases I/O congestion at the disk

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Robert Treat
On Wednesday 25 June 2008 11:24:23 Greg Smith wrote: What I often do is get a hardware RAID controller, just to accelerate disk writes, but configure it in JBOD mode and use Linux or other software RAID on that platform. JBOD + RAIDZ2 FTW ;-) -- Robert Treat Build A Brighter LAMP :: Linux

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Merlin Moncure wrote: Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? I have here: http://merlinmoncure.blogspot.com/2007/08/following-are-results-of-our-testing-of.html The upshot is I don't really see a difference in performance. The

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Peter T. Breuer
Also sprach Matthew Wakeling: Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? ... The upshot is I don't really see a difference in performance. The main difference is that you can get hardware RAID with battery-backed-up cache, which means small writes

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Peter T. Breuer wrote: You can put the log/bitmap wherever you want in software raid, including on a battery-backed local ram disk if you feel so inclined. So there is no intrinsic advantage to be gained there at all. You are technically correct but this is irrelevant.

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Jonah H. Harris
On Wed, Jun 25, 2008 at 11:24 AM, Greg Smith [EMAIL PROTECTED] wrote: SMART doesn't detect 100% of drive failures in advance, but you'd be silly to setup a database system where you don't get to take advantage of the ~50% it does catch before you lose any data. Can't argue with that one. --

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Joshua D. Drake
On Wed, 2008-06-25 at 11:30 -0400, Jonah H. Harris wrote: On Wed, Jun 25, 2008 at 11:24 AM, Greg Smith [EMAIL PROTECTED] wrote: SMART doesn't detect 100% of drive failures in advance, but you'd be silly to setup a database system where you don't get to take advantage of the ~50% it does

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Greg Smith wrote: A firewire-attached log device is an extremely bad idea. Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a transfer rate of 100MB/s, and doesn't degrade much in

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Joshua D. Drake
On Wed, 2008-06-25 at 09:53 -0600, Scott Marlowe wrote: On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey [EMAIL PROTECTED] wrote: Hi I'm currently having a problem with a well known very large servermanufacturer who shall remain unnamed and their semi-custom RAID controller firmware not

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 11:55 AM, Joshua D. Drake [EMAIL PROTECTED] wrote: On Wed, 2008-06-25 at 09:53 -0600, Scott Marlowe wrote: On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey [EMAIL PROTECTED] wrote: I'm currently having a problem with a well known very large servermanufacturer who shall

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 9:03 AM, Matthew Wakeling [EMAIL PROTECTED] wrote: On Wed, 25 Jun 2008, Merlin Moncure wrote: Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? I have here:

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
On Wed, Jun 25, 2008 at 01:35:49PM -0400, Merlin Moncure wrote: experiences, i'm starting to be more partial to linux distributions with faster moving kernels, mainly because i trust the kernel drivers more than the vendor provided drivers. While I have some experience that agrees with this,

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
On Wed, Jun 25, 2008 at 01:07:25PM -0500, Kevin Grittner wrote: It doesn't have to be free software to cut that way. I've actually found the free software to waste less of my time. No question. But one of the unfortunate facts of the no-charge-for-licenses world is that many people

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Peter T. Breuer wrote: I refrained from saying in my reply that I would set up a firewire-based link to ram in a spare old portable (which comes with a battery) if I wanted to do this cheaply. Maybe, but this is kind of a weird setup. Not many people are going to run a

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Merlin Moncure wrote: So, based on this and other experiences, i'm starting to be more partial to linux distributions with faster moving kernels, mainly because i trust the kernel drivers more than the vendor provided drivers. Depends on how fast. I find it takes a