Re: [gentoo-user] Performance problem at writing big files and Multitasking
Hi Daniel, on Saturday, 2007-02-10 at 12:49:14, you wrote: I will give short overview what i have tried so far. 1. Trying different I/O Scheduler ( cfq anticipatory and deadline) 2. Enabling Low latency kernel and Preemptible kernel 3. Setting 1000 HZ for timer frequency 4. Tried the new kernel 2.6.19-gentoo-r6 and even the testing version 2.6.20-gentoo with core 2 enabled in processor type Oh, so it is a multicore CPU---sorry if you mentioned it already, I had deleted the start of the thread already when I read Benno's advice. In that case, try 100 Hz scheduling period as well. I've had very bad experiences with I/O and 250 Hz or higher on a dual Xeon. My guess is that it was a cache effect and therefore shouldn't happen on the Core2Duo, but it might still be worth a try. As i am using Xfce i installed the diskperf-plugin which monitors disk I/O. The monitoring is divided in disk-read and disk-write. I recognized that every time when reading stops writing starts. So is this staggering of writing to disk normal as the programs have to read data they want to write to disk? On my previous machine i didn't recognize such a behaviour. So you're reading and writing from/to the same disk? I'd expect that behavior then, because the I/O scheduler tries to satisfy requests with as little thrashing as possible. So if there are enough write requests queued up it may keep the HD busy writing for a while before reading the next chunk from somewhere else. cheers! Matthias -- I prefer encrypted and signed messages. KeyID: FAC37665 Fingerprint: 8C16 3F0A A6FC DF0D 19B0 8DEF 48D9 1700 FAC3 7665 pgp1cmbiVv67p.pgp Description: PGP signature
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Sent via BlackBerry® from Vodafone -Original Message- From: Matthias Bethke [EMAIL PROTECTED] Date: Mon, 26 Feb 2007 17:28:35 To:gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Performance problem at writing big files and Multitasking Hi Daniel, on Saturday, 2007-02-10 at 12:49:14, you wrote: I will give short overview what i have tried so far. 1. Trying different I/O Scheduler ( cfq anticipatory and deadline) 2. Enabling Low latency kernel and Preemptible kernel 3. Setting 1000 HZ for timer frequency 4. Tried the new kernel 2.6.19-gentoo-r6 and even the testing version 2.6.20-gentoo with core 2 enabled in processor type Oh, so it is a multicore CPU---sorry if you mentioned it already, I had deleted the start of the thread already when I read Benno's advice. In that case, try 100 Hz scheduling period as well. I've had very bad experiences with I/O and 250 Hz or higher on a dual Xeon. My guess is that it was a cache effect and therefore shouldn't happen on the Core2Duo, but it might still be worth a try. As i am using Xfce i installed the diskperf-plugin which monitors disk I/O. The monitoring is divided in disk-read and disk-write. I recognized that every time when reading stops writing starts. So is this staggering of writing to disk normal as the programs have to read data they want to write to disk? On my previous machine i didn't recognize such a behaviour. So you're reading and writing from/to the same disk? I'd expect that behavior then, because the I/O scheduler tries to satisfy requests with as little thrashing as possible. So if there are enough write requests queued up it may keep the HD busy writing for a while before reading the next chunk from somewhere else. cheers! Matthias -- I prefer encrypted and signed messages. KeyID: FAC37665 Fingerprint: 8C16 3F0A A6FC DF0D 19B0 8DEF 48D9 1700 FAC3 7665 �éí¢‹¬z¸ žÚ(¢¸j)bž b²
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Hi i am back again! Unfortunately my problem is still bugging me! I will give short overview what i have tried so far. 1. Trying different I/O Scheduler ( cfq anticipatory and deadline) 2. Enabling Low latency kernel and Preemptible kernel 3. Setting 1000 HZ for timer frequency 4. Tried the new kernel 2.6.19-gentoo-r6 and even the testing version 2.6.20-gentoo with core 2 enabled in processor type Nothing seems to help. Hdparm also shows me that my disk has IO_support = 0 (16-bit) set instead of IO_support = 1 (32-bit) I can't set it to 32 bit as hdparm can only display settings for sata disks but not alter it. I tried sdparm and blktool but there seems to be no option for it. As i am using Xfce i installed the diskperf-plugin which monitors disk I/O. The monitoring is divided in disk-read and disk-write. I recognized that every time when reading stops writing starts. So is this staggering of writing to disk normal as the programs have to read data they want to write to disk? On my previous machine i didn't recognize such a behaviour. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Le mardi 06 février 2007 à 22:28 +0100, Daniel Pielmeier a écrit : What I/O scheduler are you using? Did you try to experiment with the deadline or the cfq I/O schedulers? If you have them enabled in your kernel config, read Documentation/block/switching-sched.txt and see if things get better after changing the I/O scheduler for your hard disk. Of course, your problem might be caused by something else altogether. At the moment i use the CFQ-scheduler others are not compiled in the kernel, I will compile one and tell you if this gives any improvements. Did you try to play with ionice to assign priority. It has helped me a lot with io-intensive background taskes. Fred -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Did you try to play with ionice to assign priority. It has helped me a lot with io-intensive background taskes. Where can i get it. It seems to be included in util-linux but i have util-linux installed and ionice is missing! -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
On Thursday 8 February 2007 19:34, Daniel Pielmeier wrote: Where can i get it. It seems to be included in util-linux but i have util-linux installed and ionice is missing! It's explained here: http://forums.gentoo.org/viewtopic-t-462230.html -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
On Thu, 2007-02-08 at 11:42 +0100, Frédéric Grosshans wrote: Did you try to play with ionice to assign priority. It has helped me a lot with io-intensive background taskes. I am trying to compile it to overcome some heavy-disk access performance issues (ext3), but I can't! Not compatible with recent headers or something :( See my thread on _syscallX isn't in linux-headers-2.6.20 if you can help!! thanks, -- Iain Buchanan iaindb at netspace dot net dot au Style may not be the answer, but at least it's a workable alternative. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
What I/O scheduler are you using? Did you try to experiment with the deadline or the cfq I/O schedulers? If you have them enabled in your kernel config, read Documentation/block/switching-sched.txt and see if things get better after changing the I/O scheduler for your hard disk. Of course, your problem might be caused by something else altogether. No other suggestions? -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Daniel Pielmeier wrote: What I/O scheduler are you using? Did you try to experiment with the deadline or the cfq I/O schedulers? If you have them enabled in your kernel config, read Documentation/block/switching-sched.txt and see if things get better after changing the I/O scheduler for your hard disk. Of course, your problem might be caused by something else altogether. No other suggestions? The hdparm optimizations, the CFQ scheduler and maybe low latency desktop in the kernel are the best way to make your desktop useable under heavy disk usage.These are my hdparm settings: /dev/hda: multcount= 16 (on) IO_support = 1 (32-bit) unmaskirq= 1 (on) using_dma= 1 (on) keepsettings = 0 (off) readonly = 0 (off) readahead= 256 (on) geometry = 26310/16/63, sectors = 26520480, start = 0 cat /sys/block/hda/queue/scheduler noop [cfq] Linux c-68-85-77-239 2.6.20-gentoo #4 PREEMPT Tue Feb 6 17:03:56 EST 2007 i686 AMD Athlon(tm) XP 2000+ AuthenticAMD GNU/Linux -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
The hdparm optimizations, the CFQ scheduler and maybe low latency desktop in the kernel are the best way to make your desktop useable under heavy disk usage.These are my hdparm settings: [snip] Thanks for your suggestions. As i have an SATA harddisk, can i use hdparm with it , i thought it could only be used with PATA devices and for SATA there is sdparm. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Mike wrote: Daniel Pielmeier wrote: No other suggestions? The hdparm optimizations, the CFQ scheduler and maybe low latency desktop in the kernel are the best way to make your desktop useable under heavy disk usage. And check you have HZ set to 1000. $ grep HZ /usr/src/linux/.config # CONFIG_HZ_100 is not set # CONFIG_HZ_250 is not set CONFIG_HZ_1000=y CONFIG_HZ=1000 Benno -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
On 2/7/07, Daniel Pielmeier [EMAIL PROTECTED] wrote: No other suggestions? -- gentoo-user@gentoo.org mailing list Could your problem be similar to this: http://marc.theaimsgroup.com/?l=gentoo-userm=114252338111258w=2 Tom
Re: [gentoo-user] Performance problem at writing big files and Multitasking
The hdparm optimizations, the CFQ scheduler and maybe low latency desktop in the kernel are the best way to make your desktop useable under heavy disk usage.These are my hdparm settings: /dev/hda: multcount= 16 (on) IO_support = 1 (32-bit) unmaskirq= 1 (on) using_dma= 1 (on) keepsettings = 0 (off) readonly = 0 (off) readahead= 256 (on) geometry = 26310/16/63, sectors = 26520480, start = 0 cat /sys/block/hda/queue/scheduler noop [cfq] Linux c-68-85-77-239 2.6.20-gentoo #4 PREEMPT Tue Feb 6 17:03:56 EST 2007 i686 AMD Athlon(tm) XP 2000+ AuthenticAMD GNU/Linux Now i am in front of my PC. I already use the cfq scheduler and have low latency desktop enabled. I looked at the output of hdparm -I /dev/sda and all seems to be ok by default. And check you have HZ set to 1000. $ grep HZ /usr/src/linux/.config # CONFIG_HZ_100 is not set # CONFIG_HZ_250 is not set CONFIG_HZ_1000=y CONFIG_HZ=1000 Thanks i will check if this helps, i have just set 250 HZ for timer frequency. Could your problem be similar to this: http://marc.theaimsgroup.com/?l=gentoo-userm=114252338111258w=2 Tom Maybe, today the gentoo-sources-2.6.19-r5 become stable so i will try the new kernel. I am also waiting for kernel 2.6.20 as it has a setting for the core 2 duo in processor settings. Maybe i try the testing version of it. Also the new gcc compilers will support the core 2 duo directly, but i think it will take some time till 4.2 or even 4.3 will become stable. I show up here again and tell you if something improves the performance of my system. Regards Daniel -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
On Tuesday 6 February 2007 21:31, Daniel Pielmeier wrote: On the other i recently recognized when i doing some video editing i.e. demultiplex the movie with projectx which shows the speed of writing video and audio to the disk. It writes about 200MB with about 25MB/s then it halts for about 2-3 seconds and starts again writing data to the disk, after another 200MB it halts again and starts over again and so on. I also recognize the same behavior when i multiplex the video and audio together with mplex. What could cause this problems What I/O scheduler are you using? Did you try to experiment with the deadline or the cfq I/O schedulers? If you have them enabled in your kernel config, read Documentation/block/switching-sched.txt and see if things get better after changing the I/O scheduler for your hard disk. Of course, your problem might be caused by something else altogether. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Hi, Am Dienstag, 6. Februar 2007 schrieb Daniel Pielmeier: Hi all, i have bought components to assemble a new PC by myself. This are the main components. that's very interesting. You seem to have exactly the same problem I posted one minute before you :) My machine is an amd64 3000, with via based board, gentoo compiled for ~x86. Any chance you are using xfs? Regards, Michael -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
Michael Schreckenbauer schrieb: Hi, Am Dienstag, 6. Februar 2007 schrieb Daniel Pielmeier: Hi all, i have bought components to assemble a new PC by myself. This are the main components. that's very interesting. You seem to have exactly the same problem I posted one minute before you :) My machine is an amd64 3000, with via based board, gentoo compiled for ~x86. Any chance you are using xfs? Regards, Michael No i use ext2 for boot and ext3 for all other filesystems I don't know if it also happens when my system is in an idle state, i have to investigate on this! Regards Daniel -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
What I/O scheduler are you using? Did you try to experiment with the deadline or the cfq I/O schedulers? If you have them enabled in your kernel config, read Documentation/block/switching-sched.txt and see if things get better after changing the I/O scheduler for your hard disk. Of course, your problem might be caused by something else altogether. At the moment i use the CFQ-scheduler others are not compiled in the kernel, I will compile one and tell you if this gives any improvements. Regards Daniel -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Performance problem at writing big files and Multitasking
What I/O scheduler are you using? Did you try to experiment with the deadline or the cfq I/O schedulers? If you have them enabled in your kernel config, read Documentation/block/switching-sched.txt and see if things get better after changing the I/O scheduler for your hard disk. Of course, your problem might be caused by something else altogether. I have tried the three available schedulers but unfortunately this has no effect. -- gentoo-user@gentoo.org mailing list
