Re: Help: very slow software RAID 5.
Dean S. Messing wrote: Michael Tokarev writes: : Dean S. Messing wrote: : [] : [] That's what : attracted me to RAID 0 --- which seems to have no downside EXCEPT : safety :-). : : So I'm not sure I'll ever figure out the right tuning. I'm at the : point of abandoning RAID entirely and just putting the three disks : together as a big LV and being done with it. (I don't have quite the : moxy to define a RAID 0 array underneath it. :-) : : Putting three disks together as a big LV - that's exactly what : linear md module. : It's almost as unsafe as raid0, but with : linear read/write speed equal to speed of single drive... I understand I only get the speed of a single drive was I was not aware of the safety factor. I had intended to use snapshotting off to a cheap USB drive each evening. Will that not keep me safe within a day's worth of data change? I only learned about snapshots yesterday. I'm utterly new to the disk array/LVM game. But your read speed need not be limited if you tune the readahead. There's also the question of how much transfer speed you actually *need*. If your application is CPU-bound faster will not be the same as runs in less time, and random access is limited by the seek speed of your drives, although some RAID tuning does apply to random writes. For that matter why not run a RAID-0 + LVM across two of the three drives and snapshot to the third? : Note also that the more drives you add to raid0-like config, : the more chances of failure you'll have - because raid0 fails : when ANY drive fails. Ditto - for certain extent - for linear : md module and for one big LV which is basically the same thing. I understand the probability increases for additional drives. : By the way, before abandoming R in RAID, I'd check whenever : the resulting speed with raid5 (after at least read-ahead tuning) : is acceptable, and use that if yes. My problem is not quite knowing what acceptable is. I bought a Dell Precision 490 with two relatively fast SATA II drives. With RAID 0 I attain speeds of nearly 140 MB/s (using 2 drives) for reads and writes and the system is very snappy for everything, from processing 4Kx2K video to building a 'locate' datebase, to searching my very large mail archives for technical info. When you process video and monitor the system with vmstat, do you see significant iowait time? No, neither do I, with a modest readahead I am totally CPU limited. If you are searching your mail database, if you just use a text tool which reads everything, that's pure sequential access. And unless you actually *use* the locate command, building that database is just a way to beat your disks (and it's more sequential than you would expect). You can turn it off and do your bit to avoid global warming. When I see the speed loss of software RAID 5 (writes are at 55MB/s and random reads are at 54 MB/s) for everything but seq. reads (and that only if I increase read-ahead from 512 to 16384 to get read speeds of about 110 MB/s I lose heart, esp. since I don't know the other consequences of increasing read-ahead by so much. Assuming that your have enough memory, there would be a small slowdown in random reading a lot of small records. You should know what your application would do, but that access is typical of looking things up in a database or processing small records, like a DNS or mail server. Numbers from bonnie or similar benchmarks are nice, but they show details of various performance area, and if you don't match what you do to what works best you make bad choices. In other words if your application can only read 10MB/s the benchmark is telling you your disk is fast enough to keep up with the CPU. : If no, maybe raid10 over : the same 3 drives will give better results. Does RAID10 work on three drives? I though one needed 4 drives, with striping across a pair of mirrored pairs. No, that's 0+1, RAID-10 works across any number of drives. Have you actually take 10-15 minutes to read man md and get the overview of how RAID works, or are you reading bits and pieces about individual features? -- bill davidsen [EMAIL PROTECTED] CTO TMR Associates, Inc Doing interesting things with small computers since 1979 - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
syncing remote homes.
Hello all, I hope this isn't a complete off-topic question (or a stupid one either). We want to setup two desktop servers at two different office branches. users will open sessions via nx. We dont' have are problem keeping the server instalations in sync and, in case of disaster at one office, we can route sessions to the remaining server at the other office with DNS (that's the plan anayway). But we're not sure about how to keep the homes in sync between the two servers. We've thought about nightly rsync but what else could be done? One suggestion was to use md with one local disc and another remote disc via iSCSI. But I don't think that would work because our network connection is not fast. Is that correct? Any advise please? Thanks. Chris. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2.6.23-rc7 1/3] async_tx: usage documentation and developer notes
Dan Williams wrote: Signed-off-by: Dan Williams [EMAIL PROTECTED] --- Documentation/crypto/async-tx-api.txt | 217 + 1 files changed, 217 insertions(+), 0 deletions(-) diff --git a/Documentation/crypto/async-tx-api.txt b/Documentation/crypto/async-tx-api.txt new file mode 100644 index 000..48d685a --- /dev/null +++ b/Documentation/crypto/async-tx-api.txt @@ -0,0 +1,217 @@ +Asynchronous Transfers/Transforms API + +1 INTRODUCTION + +2 GENEALOGY + +3 USAGE +3.1 General format of the API +3.2 Supported operations +3.2 Descriptor management +3.3 When does the operation execute? +3.4 When does the operation complete? +3.5 Constraints +3.6 Example + This is very readable, and appears extensible to any new forthcoming technology I'm aware of. Great job! -- bill davidsen [EMAIL PROTECTED] CTO TMR Associates, Inc Doing interesting things with small computers since 1979 - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2.6.23-rc7 1/3] async_tx: usage documentation and developernotes
From: Williams, Dan J Sent: Thursday, September 20, 2007 6:28 PM To: [EMAIL PROTECTED]; linux-raid@vger.kernel.org; [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; Nelson, Shannon Subject: [PATCH 2.6.23-rc7 1/3] async_tx: usage documentation and developernotes Signed-off-by: Dan Williams [EMAIL PROTECTED] --- Documentation/crypto/async-tx-api.txt | 217 + 1 files changed, 217 insertions(+), 0 deletions(-) Thanks, Dan. In addition to Randy's most excellent nit-picking, here are a couple more minor nits. [...] +4.2 My application needs finer control of hardware channels +This requirement seems to arise from cases where a DMA engine driver is [...] +dma_async_client_chan_request triggers dmaeninge to notify the client of s/dmaeninge/dmaengine/ [...] +5 SOURCE +drivers/dma/dmaengine.c: offload engine channel management routines +drivers/dma/: location for offload engine drivers +crypto/async_tx/async_tx.c: async_tx interface to dmaengine and common code +crypto/async_tx/async_memcpy.c: copy offload +crypto/async_tx/async_memset.c: memory fill offload +crypto/async_tx/async_xor.c: xor offload Also include/linux/dmaengine.h sln -- == Mr. Shannon Nelson LAN Access Division, Intel Corp. [EMAIL PROTECTED]I don't speak for Intel (503) 712-7659Parents can't afford to be squeamish. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
raid5 post mortem analisys
Hello, I had a raid 5 array with 3 drives. (on a USB 2.0 bus :)). After some time, on drive failed. After some more time another drive failed and the array stopped running. I know that the usage pattern from the first failure to the second was read-only, i.e as a user only reads were performed. I also know that the cause of the drive's failures was that they just dissapeared from the USB bus (probably from a bug in the hard drive's enclosure's USB to IDE bridge) I trashed the array anyway, but since I am new to linux md devices, I was wishing that you could help me understand if there was any possibility of getting it back up assuming that there was no data corruption. I run kernel 2.6.17 on a debian system and use mdadm for controlling the array from user space. Daniel Santos - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] [mdadm] config.c include dirent.h instead sys/dir.h
sys/dir.h is an old BSD'ism, include dirent directly. small step for better klibc support. compile tested against glibc. Signed-off-by: maximilian attems [EMAIL PROTECTED] --- config.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/config.c b/config.c index 73031b7..68c1507 100644 --- a/config.c +++ b/config.c @@ -29,7 +29,7 @@ #include mdadm.h #include dlink.h -#include sys/dir.h +#include dirent.h #include glob.h #include fnmatch.h #include ctype.h -- 1.5.2.4 - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2.6.23-rc7 0/3] async_tx and md-accel fixes for 2.6.23
On Thu, 20 Sep 2007 18:27:35 -0700 Dan Williams [EMAIL PROTECTED] wrote: Fix a couple bugs and provide documentation for the async_tx api. Neil, please 'ack' patch #3. git://lost.foo-projects.org/~dwillia2/git/iop async-tx-fixes-for-linus Well it looks like Neil is on vacation or is hiding from us or something. In which case our options would be to go ahead and merge your #3 with our fingers crossed, or wait and do it in 2.6.23.1 or .2. How serious is this bug which you're fixing? Will it affect users of legacy MD setups, or only those who have enabled fancy dma-engine features? Either way, we need to get #2 (at least) in to Linus. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Help: very slow software RAID 5.
Bill Davidsen wrote: Dean Messing wrote: : I understand I only get the speed of a single drive was I was not : aware of the safety factor. I had intended to use snapshotting off : to a cheap USB drive each evening. Will that not keep me safe within a : day's worth of data change? I only learned about snapshots yesterday. : I'm utterly new to the disk array/LVM game. : : : But your read speed need not be limited if you tune the readahead. : There's also the question of how much transfer speed you actually : *need*. If your application is CPU-bound faster will not be the same as : runs in less time, and random access is limited by the seek speed of : your drives, although some RAID tuning does apply to random writes. This is my main workstation. I do many things on it, not just an application. I live on the machine. It will run XP on top of VMware. It will transcode video. It will run map software, emacs, mail, mplayer, re-compile large applications like ImageMagick, Transcode, and Scilab, and much more. The video/image processing (whch can be random access) is just an example. Increasing readahead does increase the sequential read speed for RAID5. But the random read speed suffers and the write speed suffers a loss of 20% over single disk writes (according to the bonnie++ numbers). RAID0 on the other machine spoiled me, I'm afraid. Question: From what I've read, when reading in RAID5, parity is not read. Why then does read speed suffer so badly with the default Readahead parameter? : When you process video and monitor the system with vmstat, do you see : significant iowait time? No, neither do I, with a modest readahead I am : totally CPU limited. If you are searching your mail database, if you : just use a text tool which reads everything, that's pure sequential : access. And unless you actually *use* the locate command, building that : database is just a way to beat your disks (and it's more sequential than : you would expect). You can turn it off and do your bit to avoid global : warming. I have not used vmstat, but I am dealing with 4Kx2K 24bit uncompressed video frames. mplayer, for one, is quite disk i/o limited. So are several spatio-temporal adaptive interpolation algorithms we work on for upscaling video (part of my research). I can see this from the gkrellm disk meter both during video i/o and when swap space is getting used. Funny you should mention locate. I use it heavily to find stuff. I typically generate results on any of 4 different machines, and then `scp' the results around between the machines. So I rebuild the database relatively often. On another machine that I bought from Dell, configured with RAID0, everything is very snappy. Rebuilding the locate db simply flew. It is that uniform snappiness I was hoping (against hope?) to duplicate on this current workstation with a third drive and RAID5. : : If no, maybe raid10 over : : the same 3 drives will give better results. : : Does RAID10 work on three drives? I though one needed 4 drives, : with striping across a pair of mirrored pairs. : : No, that's 0+1, RAID-10 works across any number of drives. : : Have you actually take 10-15 minutes to read man md and get the : overview of how RAID works, or are you reading bits and pieces about : individual features? I confess I have not read the md man page. (I shall, right after this.) I have read the mdadm page pretty thoroughly. And I've read parts of lots of other stuff in the last few days. It has all uniformly said that RAID-10 is striped mirrors and requires 4 drives. One such example (which I just googled) is: http://www.pcguide.com/ref/hdd/perf/raid/levels/multLevel01-c.html http://www.pcguide.com/ref/hdd/perf/raid/levels/multXY-c.html I suppose Linux Software Raid is more general and allows 3 drives in a RAID-10 config. I'll find out in a few minutes. In my last note I asked: : : Putting three disks together as a big LV - that's exactly what : linear md module. : It's almost as unsafe as raid0, but with : linear read/write speed equal to speed of single drive... I understand I only get the speed of a single drive but I was not aware of the safety factor. I had intended to use snapshotting off to a cheap USB drive each evening. Will that not keep me safe within a day's worth of data change? I only learned about snapshots yesterday. I'm utterly new to the disk array/LVM game. snip For that matter why not run a RAID-0 + LVM across two of the three drives and snapshot to the third? What do you think about the RAID-0 + LVM plus daily snapshots? I am not running a server. In the (fairly remote) chance that I do have a RAID 0 failure, I can tolerate the couple of hours it will take to rebuild the file system and be back up and running (in ordinary non-RAID mode). Dean - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo
Re: Help: very slow software RAID 5.
This is to both Bill Davidsen and Michael Tokarev. I just realised in re-reading previous messages that I badly screwed up my attributions in my just-sent message. I attributed to Bill some technical remarks by Michael. I apologise to both of you! Dean - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 2.6.23-rc7 0/3] async_tx and md-accel fixes for 2.6.23
From: Andrew Morton [mailto:[EMAIL PROTECTED] Fix a couple bugs and provide documentation for the async_tx api. Neil, please 'ack' patch #3. git://lost.foo-projects.org/~dwillia2/git/iop async-tx-fixes-for-linus Well it looks like Neil is on vacation or is hiding from us or something. In which case our options would be to go ahead and merge your #3 with our fingers crossed, or wait and do it in 2.6.23.1 or .2. How serious is this bug which you're fixing? Will it affect users of legacy MD setups, or only those who have enabled fancy dma-engine features? The symptom is raid array that fails to complete a read request. It has only been triggered by Yuri when he tried a dma-engine configuration on his PPC platform with PAGE_SIZE=64K, this opens a large window for the bug to hit. The window is much smaller but not 100% closed in the !dma-engine case, although no one else has hit it. 2.6.22 returned reads from handle_stripe under the stripe lock so this patch can be seen as 'reverting a botched micro-optimization' versus 'introducing a new way to handle reads'. Either way, we need to get #2 (at least) in to Linus. #2 is less critical and cannot hit with any of the current drivers in the kernel, so it is ok if this one misses 2.6.23. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2.6.23-rc7 0/3] async_tx and md-accel fixes for 2.6.23
On Friday September 21, [EMAIL PROTECTED] wrote: On Thu, 20 Sep 2007 18:27:35 -0700 Dan Williams [EMAIL PROTECTED] wrote: Fix a couple bugs and provide documentation for the async_tx api. Neil, please 'ack' patch #3. git://lost.foo-projects.org/~dwillia2/git/iop async-tx-fixes-for-linus Well it looks like Neil is on vacation or is hiding from us or something. Neil is just not coping well with jet-lag Patch #3 looks good and necessary Acked-By: NeilBrown [EMAIL PROTECTED] I know that should probably be a reviewed-by I was a bit surprised that the handle_completed_read_requests call was so early in handle_stripe5 - I don't think the code was originally that early. But it is probably right. Hopefully my brain will have cleared by Monday and I'll review it again then. NeilBrown - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
