Re: raid and 2.4 kernels
On Mon, Jul 31, 2000 at 01:43:49PM +1000, Neil Brown wrote: raid0 will only get close to 'n' times a single disc when you have a number of separate threads accessing the device, otherwise there are fewer opportunities for multiple drives to be accessed at once. I believe that bonnie is single-threaded, so it is unlikely to drive a raid0 array optimally. Here are the numbers of tiobench 0.3.1 (sorry, raid0 is gone now, the machine is in active use now) File Block Num Seq ReadRand Read Seq Write Rand Write DirSize Size Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%) --- -- --- --- --- --- --- --- fujitsu 50032768 1 20.07 10.1% 3.286 1.65% 17.33 15.2% 2.900 2.20% fujitsu 50032768 2 11.57 5.72% 3.281 1.60% 16.75 15.7% 2.960 2.26% fujitsu 50032768 4 10.07 5.16% 3.251 1.58% 15.15 14.2% 3.032 2.36% fujitsu 50032768 8 9.534 5.08% 3.269 1.56% 14.50 13.7% 3.047 2.54% raid5-6 50032768 1 20.55 15.1% 5.165 4.33% 2.712 4.22% 4.129 4.88% raid5-6 50032768 2 20.96 15.5% 6.231 5.10% 2.690 4.18% 4.069 4.89% raid5-6 50032768 4 21.09 15.9% 7.269 5.93% 2.693 4.21% 4.112 5.05% raid5-6 50032768 8 19.59 15.4% 8.145 6.89% 2.654 4.19% 4.122 5.47% Fujitsu is a single disk, raid5-6 is the raid5 array of 6 IBM disks. Interestingly, the read rate doesn't increase with raid5 over a single disk, but isn't decreasing significantly either when the number of threads rise. Sequential write is extremly disappointing. Nils -- Quotes from the net, featuring John Lapeyre[L] and Christopher F. Miller [M]: M I'm not sure what the right words are to describe Upside. Last month they M mentioned the sendmail **web server** as an example of the failure of the M open source process L Well, sendmail does a lousy job of serving webpages. PGP signature
Re: raid and 2.4 kernels
On Thursday July 27, [EMAIL PROTECTED] wrote: On Wed, Jul 26, 2000 at 08:12:47PM -0700, Gregory Leblanc wrote: Given the code at the moment, I am highly confident that linear, raid0 and raid1 should be just as fast in 2.4 as in 2.2. There are some issues with raid5 that I am looking into. I don't know that they affect speed much, though they might. I doubt it, see below I'm ready to be proved wrong ... you may well have suceeded :-) Could you be a little more specific? Speed comparisons on disk access? Then you can't compare RAID with no RAID effectively. You could compare the speed of 2.2/2.4 RAID, and 2.2/2.4 no RAID, but comparisons across would seem to be meaningless. Later, Ok, so here is what I did. Machine (you might have seen this before, sorry for repetitions) AMD Athlon 650, 256MB RAM, Abit KA7 Mainboard, VIA Chipset, system on Fujitsu 20GB IDE disk, 3 Promise PDC20262 UMDA Controller, 6 IBM DTLA-307045 46GB disks for data. It runs kernel 2.4.0-test1 (first 3 tests) and 2.4.0-test5-pre3 (rest) with Andre Hedricks IDE patch and the latest reiserfs. These are bonnie results: ---Sequential Output ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU fujitsu 500 2243 39.7 3400 9.4 1422 54.9 1945 75.4 2795 66.3 182.7 29.7 single500 5025 95.7 20218 81.1 11182 40.5 5214 90.9 26707 48.9 213.2 18.7 raid0, 3 500 4804 83.3 17275 42.7 11067 50.7 5393 92.6 37242 65.2 361.4 28.1 raid0, 4 500 4941 81.8 20920 48.7 13068 66.4 5283 91.0 35318 55.3 394.8 27.6 raid5, 6 500 3009 52.0 3004 4.6 2589 3.6 4411 70.4 18982 13.9 360.5 22.7 where - "fujitsu" is the system disk - "single" is a filesystem on a single IBM disk - "raid0, 3" is a RAID0 array of 3 IBM disks - "raid0, 4" is a RAID0 array of 4 IBM disks - "raid5, 6" is a RAID5 array of 6 IBM disks In between the disks have been freshly partitioned (1 big primary only) mkraid'ed and given a fresh reiserfs. It could of course be a combination of reiserfs, Andre Hedrick's IDE patches and raid. I expected the raid 0 with n disks to get a bit less than n times the block read performance of 1 disk, and raid5 to have block write performance of a bit less than single disk, block read performance much better than for a single disk. Are these expectations unrealistic? I think your expectations of raid0 may be a little over optimistic. raid0 will only get close to 'n' times a single disc when you have a number of separate threads accessing the device, otherwise there are fewer opportunities for multiple drives to be accessed at once. I believe that bonnie is single-threaded, so it is unlikely to drive a raid0 array optimally. Possibly you could try running 3 bonnies in parallel, and compare that to three paralel bonnies on three separate drives. So your read timings look believable - raid0 is faster but not stunningly faster. The extra speed probably comes from read-ahead, which adds an element of multi-threading. Your write times are a bit dissapointing though. The write-behind that the kernel does should provide plenty of opportunity to get all the drives writing at once. Cluster size can affect this a bit, but I wouldn't expect it that much. I might do some similar tests my self... For raid-5, your numbers are disappointing, but not actually very surprising now that I think about it. The way the raid5 code is at the moment, only one request can be outstanding for each 4k wide stripe. Also, request are processed immediately, but are queue, and have to wait for the raid5d thread to wake-up, which is probably a context-switch away at least. I hope to fix this one day, but whether it will be before 2.4.0-final comes out or not, I don't know. Thanks for the numbers. NeilBrown
RE: raid and 2.4 kernels
On Wednesday July 26, [EMAIL PROTECTED] wrote: Could you be a little more specific? Speed comparisons on disk access? Then you can't compare RAID with no RAID effectively. You could compare the speed of 2.2/2.4 RAID, and 2.2/2.4 no RAID, but comparisons across would seem to be meaningless. Later, Grego Suppose we are interested in raid1. We start with 2.2 We have two similar drives. We make a file system on just one, and run bonnie (or whatever). We then make a raid1 set of the two drives, make a file system on that, and run bonnie on that. Then we boot 2.4 and repeat the process, and look at the numbers. If raid on 2.4 is fast than raid in 2.2, we say "great". If it is slower, we look at the no-raid numbers. If no-raid on 2.4 is slow than no-raid on 2.2, we say "oh dear, the disc subsystem is slower on 2.4", and point the finger appropriately. If no-raid on 2.2 is fast than no-raid on 2.4, then we say "Hmm, must be a problem with raid" and point the finger there. Does that make sense? To put it another way - I don't want to compare speeds between no-raid and raid, I want to compare "changes-between-os-version" between no-raid and raid. NeilBrown
RE: raid and 2.4 kernels
On Thu, 27 Jul 2000, Neil Brown wrote: If raid on 2.4 is fast than raid in 2.2, we say "great". If it is slower, we look at the no-raid numbers. If no-raid on 2.4 is slow than no-raid on 2.2, we say "oh dear, the disc subsystem is slower on 2.4", and point the finger appropriately. If no-raid on 2.2 is fast than no-raid on 2.4, then we say "Hmm, must be a problem with raid" and point the finger there. Does that make sense? In a way, yes. But raid could depend on other parts of the kernel more heavily then no-raid disk access and thus could be more affected by errors/problems in those parts. D.
Re: raid and 2.4 kernels
On Wed, Jul 26, 2000 at 08:12:47PM -0700, Gregory Leblanc wrote: Given the code at the moment, I am highly confident that linear, raid0 and raid1 should be just as fast in 2.4 as in 2.2. There are some issues with raid5 that I am looking into. I don't know that they affect speed much, though they might. I doubt it, see below Could you be a little more specific? Speed comparisons on disk access? Then you can't compare RAID with no RAID effectively. You could compare the speed of 2.2/2.4 RAID, and 2.2/2.4 no RAID, but comparisons across would seem to be meaningless. Later, Ok, so here is what I did. Machine (you might have seen this before, sorry for repetitions) AMD Athlon 650, 256MB RAM, Abit KA7 Mainboard, VIA Chipset, system on Fujitsu 20GB IDE disk, 3 Promise PDC20262 UMDA Controller, 6 IBM DTLA-307045 46GB disks for data. It runs kernel 2.4.0-test1 (first 3 tests) and 2.4.0-test5-pre3 (rest) with Andre Hedricks IDE patch and the latest reiserfs. These are bonnie results: ---Sequential Output ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU fujitsu 500 2243 39.7 3400 9.4 1422 54.9 1945 75.4 2795 66.3 182.7 29.7 single500 5025 95.7 20218 81.1 11182 40.5 5214 90.9 26707 48.9 213.2 18.7 raid0, 3 500 4804 83.3 17275 42.7 11067 50.7 5393 92.6 37242 65.2 361.4 28.1 raid0, 4 500 4941 81.8 20920 48.7 13068 66.4 5283 91.0 35318 55.3 394.8 27.6 raid5, 6 500 3009 52.0 3004 4.6 2589 3.6 4411 70.4 18982 13.9 360.5 22.7 where - "fujitsu" is the system disk - "single" is a filesystem on a single IBM disk - "raid0, 3" is a RAID0 array of 3 IBM disks - "raid0, 4" is a RAID0 array of 4 IBM disks - "raid5, 6" is a RAID5 array of 6 IBM disks In between the disks have been freshly partitioned (1 big primary only) mkraid'ed and given a fresh reiserfs. It could of course be a combination of reiserfs, Andre Hedrick's IDE patches and raid. I expected the raid 0 with n disks to get a bit less than n times the block read performance of 1 disk, and raid5 to have block write performance of a bit less than single disk, block read performance much better than for a single disk. Are these expectations unrealistic? Nils -- Plug-and-Play is really nice, unfortunately it only works 50% of the time. To be specific the "Plug" almost always works.--unknown source PGP signature
RE: raid and 2.4 kernels
-Original Message- From: Danilo Godec [mailto:[EMAIL PROTECTED]] Sent: Thursday, July 27, 2000 12:22 AM To: Neil Brown Cc: [EMAIL PROTECTED] Subject: RE: raid and 2.4 kernels On Thu, 27 Jul 2000, Neil Brown wrote: If raid on 2.4 is fast than raid in 2.2, we say "great". If it is slower, we look at the no-raid numbers. If no-raid on 2.4 is slow than no-raid on 2.2, we say "oh dear, the disc subsystem is slower on 2.4", and point the finger appropriately. If no-raid on 2.2 is fast than no-raid on 2.4, then we say "Hmm, must be a problem with raid" and point the finger there. Does that make sense? In a way, yes. But raid could depend on other parts of the kernel more heavily then no-raid disk access and thus could be more affected by errors/problems in those parts. Not really, if you structure the test correctly. Take a FAST system, with a FAST bus, and a couple of disks that can't come close to saturating the bus (either memory, disk, or interface (probably PCI)), then you could perform a realistic test. I haven't got the time or the drives to do that right now, but it certainly would be nice. I'll take a look at the other posted results a bit later, and see if I can't make a few suggestions. It would be nice to see a similar test done with tiobench as well as with Bonnie++... Later, Grego
Re: raid and 2.4 kernels
On Wed, 26 Jul 2000, Anton wrote:
do the kernel developers responsible for RAID read this list? I would be
interested in seeing some constructive discussion about the reports of
degraded RAID performance in the 2.4 kernels. It is particularly
disappointing given that SMP appears to be a lot better in 2.4 vs 2.2
2.4 is having VM problems, therefore especially write performance can be
poor. This is not directly RAID related, although maybe RAID will amplify
the problem.
There's a lot of work going on in trying to make the VM perform more reasonably,
and I'm confident that RAID performance will be absolutely *smoking* once those
other issues are sorted out.
The block I/O layer has seen some optimizations and especially the buffer/cache
merger should improve throughput significantly (especially on low memory-bandwidth
machines).
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
Re: raid and 2.4 kernels
On Wednesday July 26, [EMAIL PROTECTED] wrote: do the kernel developers responsible for RAID read this list? I would be interested in seeing some constructive discussion about the reports of degraded RAID performance in the 2.4 kernels. It is particularly disappointing given that SMP appears to be a lot better in 2.4 vs 2.2 As Jakob mentioned, the slow down is quite possibly related to other parts of the kernel. I would really like to see speed comparisons for 2.2 no raid 2.2 raid 2.4 no raid 2.4 raid where 2.3+raid didn't fit the pattern, before I looked too deeply. Given the code at the moment, I am highly confident that linear, raid0 and raid1 should be just as fast in 2.4 as in 2.2. There are some issues with raid5 that I am looking into. I don't know that they affect speed much, though they might. NeilBrown
RE: raid and 2.4 kernels
-Original Message- From: Neil Brown [mailto:[EMAIL PROTECTED]] Sent: Wednesday, July 26, 2000 7:41 PM To: Anton Cc: [EMAIL PROTECTED] Subject: Re: raid and 2.4 kernels On Wednesday July 26, [EMAIL PROTECTED] wrote: do the kernel developers responsible for RAID read this list? I would be interested in seeing some constructive discussion about the reports of degraded RAID performance in the 2.4 kernels. It is particularly disappointing given that SMP appears to be a lot better in 2.4 vs 2.2 As Jakob mentioned, the slow down is quite possibly related to other parts of the kernel. I would really like to see speed comparisons for 2.2 no raid 2.2 raid 2.4 no raid 2.4 raid where 2.3+raid didn't fit the pattern, before I looked too deeply. Given the code at the moment, I am highly confident that linear, raid0 and raid1 should be just as fast in 2.4 as in 2.2. There are some issues with raid5 that I am looking into. I don't know that they affect speed much, though they might. Could you be a little more specific? Speed comparisons on disk access? Then you can't compare RAID with no RAID effectively. You could compare the speed of 2.2/2.4 RAID, and 2.2/2.4 no RAID, but comparisons across would seem to be meaningless. Later, Grego
