Re: Raid5 reads and cpu
On Monday August 28, [EMAIL PROTECTED] wrote: This might be a dumb question, but what causes md to use a large amount of cpu resources when reading a large amount of data from a raid1 array? I assume you meant raid5 there. md/raid5 shouldn't use that much CPU when reading. It does use more than raid0 as it reads data in the stripe-cache and then copies the data from the stripe cache into the read-buffer. But I wouldn't expect that to come anywhere near 50%. Are you really seeing 'raid5d' using 50% of CPU in 'top' or similar? NeilBrown Sorry for the long response time -- email got lost. top - 16:45:21 up 10 days, 17:41, 2 users, load average: 0.58, 0.17, 0.05 Tasks: 113 total, 2 running, 111 sleeping, 0 stopped, 0 zombie Cpu(s): 1.7% us, 87.7% sy, 6.3% ni, 0.0% id, 0.0% wa, 0.0% hi, 4.3% si Mem: 2061564k total, 2044784k used,16780k free, 1193384k buffers Swap: 4257016k total, 552k used, 4256464k free,24348k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 945 root 10 -5 000 S 44.2 0.0 7:27.73 md11_raid5 Examples are on a 2.4GHz AMD64, 2GB, 2.6.15.1 (I realize there are md enhancements to later versions; I had some other unrelated issues and rolled back to one I've run on for several months). A given 7-disk raid0 array can read 450MB/s (using cat null) and use virtually no CPU resources. (Although cat and kswapd use quite a bit [60%] munching on the data) A raid5 array on the same drive set pulls in at 250MB/s, but md uses roughly 50% of the CPU (the other 50% is spent dealing with the data, saturating the processor). A consistency check on the raid5 array uses roughly 3% of the cpu. It is otherwise ~97% idle. md11 : active raid5 sdi2[5] sdh2[4] sdf2[3] sde2[2] sdd2[1] sdc2[6] sdb2[0] 248974848 blocks level 5, 256k chunk, algorithm 2 [7/7] [UUU] [==..] resync = 72.2% (29976960/41495808) finish=3.7min speed=51460K/sec (~350MB/s aggregate throughput, 50MB/s on each device) Just a friendly question as to why CPU utilization is significantly different between a check and a real-world read on raid5? I feel like if there was vm overhead getting the data into userland, the slowdown would be present in raid0 as well. I assume parity calculations aren't done on a read of the array, which leaves me at my question. Thanks, Rob - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Raid5 reads and cpu
Rob Bray wrote: This might be a dumb question, but what causes md to use a large amount of cpu resources when reading a large amount of data from a raid1 array? Examples are on a 2.4GHz AMD64, 2GB, 2.6.15.1 (I realize there are md enhancements to later versions; I had some other unrelated issues and rolled back to one I've run on for several months). A given 7-disk raid0 array can read 450MB/s (using cat null) and use virtually no CPU resources. (Although cat and kswapd use quite a bit [60%] munching on the data) A raid5 array on the same drive set pulls in at 250MB/s, but md uses roughly 50% of the CPU (the other 50% is spent dealing with the data, saturating the processor). A consistency check on the raid5 array uses roughly 3% of the cpu. It is otherwise ~97% idle. md11 : active raid5 sdi2[5] sdh2[4] sdf2[3] sde2[2] sdd2[1] sdc2[6] sdb2[0] 248974848 blocks level 5, 256k chunk, algorithm 2 [7/7] [UUU] [==..] resync = 72.2% (29976960/41495808) finish=3.7min speed=51460K/sec (~350MB/s aggregate throughput, 50MB/s on each device) Just a friendly question as to why CPU utilization is significantly different between a check and a real-world read on raid5? I feel like if there was vm overhead getting the data into userland, the slowdown would be present in raid0 as well. I assume parity calculations aren't done on a read of the array, which leaves me at my question. What are you stripe and cache sizes? -- bill davidsen [EMAIL PROTECTED] CTO TMR Associates, Inc Doing interesting things with small computers since 1979 - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html md11 : active raid5 sdi2[5] sdh2[4] sdf2[3] sde2[2] sdd2[1] sdc2[6] sdb2[0] 248974848 blocks level 5, 256k chunk, algorithm 2 [7/7] [UUU] stripe_cache_size = 256 I've tried increasing it with the same result - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Raid5 reads and cpu
On Monday August 28, [EMAIL PROTECTED] wrote: This might be a dumb question, but what causes md to use a large amount of cpu resources when reading a large amount of data from a raid1 array? I assume you meant raid5 there. md/raid5 shouldn't use that much CPU when reading. It does use more than raid0 as it reads data in the stripe-cache and then copies the data from the stripe cache into the read-buffer. But I wouldn't expect that to come anywhere near 50%. Are you really seeing 'raid5d' using 50% of CPU in 'top' or similar? NeilBrown Examples are on a 2.4GHz AMD64, 2GB, 2.6.15.1 (I realize there are md enhancements to later versions; I had some other unrelated issues and rolled back to one I've run on for several months). A given 7-disk raid0 array can read 450MB/s (using cat null) and use virtually no CPU resources. (Although cat and kswapd use quite a bit [60%] munching on the data) A raid5 array on the same drive set pulls in at 250MB/s, but md uses roughly 50% of the CPU (the other 50% is spent dealing with the data, saturating the processor). A consistency check on the raid5 array uses roughly 3% of the cpu. It is otherwise ~97% idle. md11 : active raid5 sdi2[5] sdh2[4] sdf2[3] sde2[2] sdd2[1] sdc2[6] sdb2[0] 248974848 blocks level 5, 256k chunk, algorithm 2 [7/7] [UUU] [==..] resync = 72.2% (29976960/41495808) finish=3.7min speed=51460K/sec (~350MB/s aggregate throughput, 50MB/s on each device) Just a friendly question as to why CPU utilization is significantly different between a check and a real-world read on raid5? I feel like if there was vm overhead getting the data into userland, the slowdown would be present in raid0 as well. I assume parity calculations aren't done on a read of the array, which leaves me at my question. Thanks, Rob - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html -- VGER BF report: U 0.5 - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Raid5 reads and cpu
Rob Bray wrote: This might be a dumb question, but what causes md to use a large amount of cpu resources when reading a large amount of data from a raid1 array? Examples are on a 2.4GHz AMD64, 2GB, 2.6.15.1 (I realize there are md enhancements to later versions; I had some other unrelated issues and rolled back to one I've run on for several months). A given 7-disk raid0 array can read 450MB/s (using cat null) and use virtually no CPU resources. (Although cat and kswapd use quite a bit [60%] munching on the data) A raid5 array on the same drive set pulls in at 250MB/s, but md uses roughly 50% of the CPU (the other 50% is spent dealing with the data, saturating the processor). A consistency check on the raid5 array uses roughly 3% of the cpu. It is otherwise ~97% idle. md11 : active raid5 sdi2[5] sdh2[4] sdf2[3] sde2[2] sdd2[1] sdc2[6] sdb2[0] 248974848 blocks level 5, 256k chunk, algorithm 2 [7/7] [UUU] [==..] resync = 72.2% (29976960/41495808) finish=3.7min speed=51460K/sec (~350MB/s aggregate throughput, 50MB/s on each device) Just a friendly question as to why CPU utilization is significantly different between a check and a real-world read on raid5? I feel like if there was vm overhead getting the data into userland, the slowdown would be present in raid0 as well. I assume parity calculations aren't done on a read of the array, which leaves me at my question. What are you stripe and cache sizes? -- bill davidsen [EMAIL PROTECTED] CTO TMR Associates, Inc Doing interesting things with small computers since 1979 - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
