Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
On Wed, Sep 27, 2000 at 04:10:38PM +1000, Robert Cohen wrote: > > [EMAIL PROTECTED] wrote > > I doubt this has any relevance whatsoever, but when I try this on a > 2.2.16 > kernel running on top of a Pentium Pro 200 w/96megs of mem w/ a SCSI 2 > disk, I get some funny numbers: > > matt@zeus:~/cwork/personal$ ./elv_test 8 30 > files created, 240 megs written at 4.32 megs/sec > finished writing 240 megs written at 1794.23 megs per sec > finished reading, 240 megs read at 1675.813817 megs/sec > > Any ideas why I would be getting these numbers? > - > > That will teach me to play fast and loose with checking error returns > from system calls. I wasnt specifying a mode when creating the files. > So they were being created without user write permission.. > I dont know why it worked under 2.4.0. Maybe a kernel bug :-). > > Anyway, a new version is available now at > http://tltsu.anu.edu.au/~robert/elv_test.c Ah, thanks. I have much more likely results now: matt@zeus:~/cwork/personal$ ./elv_test2 8 30 files created, 240 megs written at 3.91 megs/sec finished writing 240 megs written at 2.28 megs per sec finished reading, 240 megs read at 0.909702 megs/sec BTW, why would the first incorrectly function in kernel 2.2? A bug feature? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
[EMAIL PROTECTED] wrote I doubt this has any relevance whatsoever, but when I try this on a 2.2.16 kernel running on top of a Pentium Pro 200 w/96megs of mem w/ a SCSI 2 disk, I get some funny numbers: matt@zeus:~/cwork/personal$ ./elv_test 8 30 files created, 240 megs written at 4.32 megs/sec finished writing 240 megs written at 1794.23 megs per sec finished reading, 240 megs read at 1675.813817 megs/sec Any ideas why I would be getting these numbers? - That will teach me to play fast and loose with checking error returns from system calls. I wasnt specifying a mode when creating the files. So they were being created without user write permission.. I dont know why it worked under 2.4.0. Maybe a kernel bug :-). Anyway, a new version is available now at http://tltsu.anu.edu.au/~robert/elv_test.c Robert Cohen TLTSU, Unix support Australian National University
Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
On Mon, Sep 25, 2000 at 04:00:51PM +0200, Jens Axboe wrote: > On Mon, Sep 25 2000, Robert Cohen wrote: > > With kernel version 2.4.0-test9pre6 the results are as follows. > > The test machine has 128 Megs of memory. The tests accesses 240 Megs of > > files so that it can't fit in cache. > > > > If I run it with 8 files of size 30 Megs: > > > > [robert@test25 src]$ ./elv_test 8 30 > > files created, 240 megs written at 8.96 megs/sec > > finished writing 240 megs written at 1.05 megs per sec << > > finished reading, 240 megs read at 5.848833 megs/sec > > > > If I do the same with a single file of size 240 Megs > > > > [robert@test25 src]$ ./elv_test 1 240 > > files created, 240 megs written at 11.12 megs/sec > > finished writing 240 megs written at 11.08 megs per sec > > finished reading, 240 megs read at 12.580521 megs/sec > > axboe@burns:/opt/software/testing > ./elv_test 8 30 > files created, 240 megs written at 21.64 megs/sec > finished writing 240 megs written at 21.12 megs per sec > > This is my current tree on 2.4.0-test9-pre5. Thanks for the test program, > Andrea and I are working on getting a polished patch ready for inclusion > that (apparently) also fixes this problem. I doubt this has any relevance whatsoever, but when I try this on a 2.2.16 kernel running on top of a Pentium Pro 200 w/96megs of mem w/ a SCSI 2 disk, I get some funny numbers: matt@zeus:~/cwork/personal$ ./elv_test 8 30 files created, 240 megs written at 4.32 megs/sec finished writing 240 megs written at 1794.23 megs per sec finished reading, 240 megs read at 1675.813817 megs/sec Any ideas why I would be getting these numbers? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
On Mon, Sep 25, 2000 at 04:00:51PM +0200, Jens Axboe wrote: On Mon, Sep 25 2000, Robert Cohen wrote: With kernel version 2.4.0-test9pre6 the results are as follows. The test machine has 128 Megs of memory. The tests accesses 240 Megs of files so that it can't fit in cache. If I run it with 8 files of size 30 Megs: [robert@test25 src]$ ./elv_test 8 30 files created, 240 megs written at 8.96 megs/sec finished writing 240 megs written at 1.05 megs per sec finished reading, 240 megs read at 5.848833 megs/sec If I do the same with a single file of size 240 Megs [robert@test25 src]$ ./elv_test 1 240 files created, 240 megs written at 11.12 megs/sec finished writing 240 megs written at 11.08 megs per sec finished reading, 240 megs read at 12.580521 megs/sec axboe@burns:/opt/software/testing ./elv_test 8 30 files created, 240 megs written at 21.64 megs/sec finished writing 240 megs written at 21.12 megs per sec This is my current tree on 2.4.0-test9-pre5. Thanks for the test program, Andrea and I are working on getting a polished patch ready for inclusion that (apparently) also fixes this problem. I doubt this has any relevance whatsoever, but when I try this on a 2.2.16 kernel running on top of a Pentium Pro 200 w/96megs of mem w/ a SCSI 2 disk, I get some funny numbers: matt@zeus:~/cwork/personal$ ./elv_test 8 30 files created, 240 megs written at 4.32 megs/sec finished writing 240 megs written at 1794.23 megs per sec finished reading, 240 megs read at 1675.813817 megs/sec Any ideas why I would be getting these numbers? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
[EMAIL PROTECTED] wrote I doubt this has any relevance whatsoever, but when I try this on a 2.2.16 kernel running on top of a Pentium Pro 200 w/96megs of mem w/ a SCSI 2 disk, I get some funny numbers: matt@zeus:~/cwork/personal$ ./elv_test 8 30 files created, 240 megs written at 4.32 megs/sec finished writing 240 megs written at 1794.23 megs per sec finished reading, 240 megs read at 1675.813817 megs/sec Any ideas why I would be getting these numbers? - That will teach me to play fast and loose with checking error returns from system calls. I wasnt specifying a mode when creating the files. So they were being created without user write permission.. I dont know why it worked under 2.4.0. Maybe a kernel bug :-). Anyway, a new version is available now at http://tltsu.anu.edu.au/~robert/elv_test.c Robert Cohen TLTSU, Unix support Australian National University
Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
On Mon, Sep 25 2000, Robert Cohen wrote: > With kernel version 2.4.0-test9pre6 the results are as follows. > The test machine has 128 Megs of memory. The tests accesses 240 Megs of > files so that it can't fit in cache. > > If I run it with 8 files of size 30 Megs: > > [robert@test25 src]$ ./elv_test 8 30 > files created, 240 megs written at 8.96 megs/sec > finished writing 240 megs written at 1.05 megs per sec<< > finished reading, 240 megs read at 5.848833 megs/sec > > If I do the same with a single file of size 240 Megs > > [robert@test25 src]$ ./elv_test 1 240 > files created, 240 megs written at 11.12 megs/sec > finished writing 240 megs written at 11.08 megs per sec > finished reading, 240 megs read at 12.580521 megs/sec axboe@burns:/opt/software/testing > ./elv_test 8 30 files created, 240 megs written at 21.64 megs/sec finished writing 240 megs written at 21.12 megs per sec This is my current tree on 2.4.0-test9-pre5. Thanks for the test program, Andrea and I are working on getting a polished patch ready for inclusion that (apparently) also fixes this problem. -- * Jens Axboe <[EMAIL PROTECTED]> * SuSE Labs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
[Demo program]: Poor elevator performance in 2.4.0-test9pre6
Ive written a small program to demonstrate the performance problems Ive been seeing in recent Linux kernels. The benchmark is a single process which writes and read 8k blocks round-robin from a number of files. It is written as a single process so the ordering of the operations is known and perfectly interleaved. I include the source at the end of this message. The files are initially created using sequential writes so that the files are laid out nicely on disk. With kernel version 2.4.0-test9pre6 the results are as follows. The test machine has 128 Megs of memory. The tests accesses 240 Megs of files so that it can't fit in cache. If I run it with 8 files of size 30 Megs: [robert@test25 src]$ ./elv_test 8 30 files created, 240 megs written at 8.96 megs/sec finished writing 240 megs written at 1.05 megs per sec << finished reading, 240 megs read at 5.848833 megs/sec If I do the same with a single file of size 240 Megs [robert@test25 src]$ ./elv_test 1 240 files created, 240 megs written at 11.12 megs/sec finished writing 240 megs written at 11.08 megs per sec finished reading, 240 megs read at 12.580521 megs/sec Comparing this to a similar tiotest run [robert@test25 src]$ tiotest -f 30 -b 8192 -t 8 -r 0 Tiotest results for 8 concurrent io threads: ,--. | Item | Time | Rate | Usr CPU | Sys CPU | +---+--+--+--+-+ | Write 240 MBs | 25.5 s | 9.410 MB/s | 0.1 % | 10.0 % | | Read 240 MBs | 20.4 s | 11.755 MB/s | 0.0 % | 8.8 % | `--' As the tests demonstrate, we get terrible write performance when a single processes is writing round robin to a number of files. There are two possible explanations for this, the single threaded nature of the program is slowing things down. Or the fact that the files are being written round robin is slowing us down. Since I see exactly the same kind of behaviour with the netatalk benchmark I have been using and the netatalk benchmark isnt single threaded, I believe that its the round robin interleaving of writes that leading to the performance problems. As a comparison, heres the results of the test program in kernel version 2.4.0-test1-ac22. [robert@testmac25 src]$ ./elv_test 8 30 files created, 240 megs written at 8.24 megs/sec finished writing 240 megs written at 8.99 megs per sec finished reading, 240 megs read at 5.849072 megs/sec Here the write performance is fine. This definitely indicates that its not the single threaded benchmark thats slowing things down. As I understand it, the elevator should be dealing with the interleaved nature of the writes. This seems to be working ok for reads, but it doesnt seem to be working properly for writes. The source can be found at http://tltsu.anu.edu.au/~robert/elv_test.c -- Robert Cohen TLTSU, Unix support Australian National University - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
[Demo program]: Poor elevator performance in 2.4.0-test9pre6
Ive written a small program to demonstrate the performance problems Ive been seeing in recent Linux kernels. The benchmark is a single process which writes and read 8k blocks round-robin from a number of files. It is written as a single process so the ordering of the operations is known and perfectly interleaved. I include the source at the end of this message. The files are initially created using sequential writes so that the files are laid out nicely on disk. With kernel version 2.4.0-test9pre6 the results are as follows. The test machine has 128 Megs of memory. The tests accesses 240 Megs of files so that it can't fit in cache. If I run it with 8 files of size 30 Megs: [robert@test25 src]$ ./elv_test 8 30 files created, 240 megs written at 8.96 megs/sec finished writing 240 megs written at 1.05 megs per sec finished reading, 240 megs read at 5.848833 megs/sec If I do the same with a single file of size 240 Megs [robert@test25 src]$ ./elv_test 1 240 files created, 240 megs written at 11.12 megs/sec finished writing 240 megs written at 11.08 megs per sec finished reading, 240 megs read at 12.580521 megs/sec Comparing this to a similar tiotest run [robert@test25 src]$ tiotest -f 30 -b 8192 -t 8 -r 0 Tiotest results for 8 concurrent io threads: ,--. | Item | Time | Rate | Usr CPU | Sys CPU | +---+--+--+--+-+ | Write 240 MBs | 25.5 s | 9.410 MB/s | 0.1 % | 10.0 % | | Read 240 MBs | 20.4 s | 11.755 MB/s | 0.0 % | 8.8 % | `--' As the tests demonstrate, we get terrible write performance when a single processes is writing round robin to a number of files. There are two possible explanations for this, the single threaded nature of the program is slowing things down. Or the fact that the files are being written round robin is slowing us down. Since I see exactly the same kind of behaviour with the netatalk benchmark I have been using and the netatalk benchmark isnt single threaded, I believe that its the round robin interleaving of writes that leading to the performance problems. As a comparison, heres the results of the test program in kernel version 2.4.0-test1-ac22. [robert@testmac25 src]$ ./elv_test 8 30 files created, 240 megs written at 8.24 megs/sec finished writing 240 megs written at 8.99 megs per sec finished reading, 240 megs read at 5.849072 megs/sec Here the write performance is fine. This definitely indicates that its not the single threaded benchmark thats slowing things down. As I understand it, the elevator should be dealing with the interleaved nature of the writes. This seems to be working ok for reads, but it doesnt seem to be working properly for writes. The source can be found at http://tltsu.anu.edu.au/~robert/elv_test.c -- Robert Cohen TLTSU, Unix support Australian National University - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [Demo program]: Poor elevator performance in 2.4.0-test9pre6
On Mon, Sep 25 2000, Robert Cohen wrote: With kernel version 2.4.0-test9pre6 the results are as follows. The test machine has 128 Megs of memory. The tests accesses 240 Megs of files so that it can't fit in cache. If I run it with 8 files of size 30 Megs: [robert@test25 src]$ ./elv_test 8 30 files created, 240 megs written at 8.96 megs/sec finished writing 240 megs written at 1.05 megs per sec finished reading, 240 megs read at 5.848833 megs/sec If I do the same with a single file of size 240 Megs [robert@test25 src]$ ./elv_test 1 240 files created, 240 megs written at 11.12 megs/sec finished writing 240 megs written at 11.08 megs per sec finished reading, 240 megs read at 12.580521 megs/sec axboe@burns:/opt/software/testing ./elv_test 8 30 files created, 240 megs written at 21.64 megs/sec finished writing 240 megs written at 21.12 megs per sec This is my current tree on 2.4.0-test9-pre5. Thanks for the test program, Andrea and I are working on getting a polished patch ready for inclusion that (apparently) also fixes this problem. -- * Jens Axboe [EMAIL PROTECTED] * SuSE Labs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
