Re: [Fwd: Re: [SLUG] IOWait definition]
iostat can be a bit special.I -think- under linux its the amount of time spent waiting for pending disk IO to complete. Now, some chipsets and their drivers seem to spend a lot of time in IOWAIT compared to others. The traditional difference was polled vs dma'ed disk IO - with polled IO, the driver would submit a request and then sit there and wait for it to complete. That time spent waiting was IOWAIT. DMA'ing controllers would spend less time in IOWAIT because they'd submit a request, kick it off, it'd happen in the background, and then the kernel would be notified when it completed. Almost no time was spent in IOWAIT - just the time scheduling the DMA commands and handling the response. Now, I've seen some low-ened SATA chipsets on some reasonably speccy hardware (eg some of the nvidia-driver Sun workstations) with large amounts of IOWAIT time. I don't know if its a driver thing or a hardware thing (or both), but there are definitely issues. Now, you could go groveling through the kernel to try and figure out whether IOWAIT includes nfs activity (which I believe would be the disk IO related to NFS, but it could be other things I guess) and first see if doing userspace disk IO does a lot of IOWAIT. Grab bonnie or some other hard disk throughput testing thing, run it, and see how much time is spent in IOWAIT. If you've got almost no IOWAIT time when doing it locally but IOWAIT time when doing it over NFS, you could be right. If you have IOWAIT time on both, I'd poke the disk/controller/ driver combo. 2d, Adrian On Thu, Oct 09, 2008, David Kempe wrote: Grant Street wrote: I have a machine with a good proportion of IOWait 20-30%. It does have local disks and it performs operations on NFS mounts. I just wanted to be sure if IOWait includes NFS activity or not. I also want a way if it is NFS to be able to say for sure if it is a bottleneck on the nfs client or server. NFS is not a linux machine so visibility is not allways the best. dstat might help you correlate stuff. http://dag.wieers.com/home-made/dstat/ dave -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA - -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [Fwd: Re: [SLUG] IOWait definition]
On Fri, Oct 10, 2008, Adrian Chadd wrote: Now, you could go groveling through the kernel to try and figure out whether IOWAIT includes nfs activity (which I believe would be the disk IO related to NFS, but it could be other things I guess) and first see if doing userspace disk IO does a lot of IOWAIT. Grab bonnie or some other hard disk throughput testing thing, run it, and see how much time is spent in IOWAIT. If you've got almost no IOWAIT time when doing it locally but IOWAIT time when doing it over NFS, you could be right. If you have IOWAIT time on both, I'd poke the disk/controller/ driver combo. I just re-re-read the OP and stuff and realise my assumptions don't match your request. I assumed you wanted to know if IOWAIT included disk IO time -and- nfs server time. But you're asking if IOWAIT include NFS client time (ie, time spent on your box talking to the NFS server.) Anyway, doing a bonnie++ or such test will still tell you. Run it locally and run it over NFS. See if you get IOWAIT increases for both. That should give you some hint as to whats going on. I've never seen IOWAIT for NFS client traffic (ie, traffic from an NFS client talking to an NFS server) but who knows, this is linux.. adrian 2d, Adrian On Thu, Oct 09, 2008, David Kempe wrote: Grant Street wrote: I have a machine with a good proportion of IOWait 20-30%. It does have local disks and it performs operations on NFS mounts. I just wanted to be sure if IOWait includes NFS activity or not. I also want a way if it is NFS to be able to say for sure if it is a bottleneck on the nfs client or server. NFS is not a linux machine so visibility is not allways the best. dstat might help you correlate stuff. http://dag.wieers.com/home-made/dstat/ dave -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA - -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA - -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [Fwd: Re: [SLUG] IOWait definition]
On Fri, Oct 10, 2008 at 05:50:52AM +0800, Adrian Chadd wrote: I've never seen IOWAIT for NFS client traffic (ie, traffic from an NFS client talking to an NFS server) but who knows, this is linux.. I would say this doesn't count to iowait either; see fs/nfs/pagelist.c:nfs_wait_on_request() -- it appears to put itself out of action without doing anything required to update iowait times (see kernel/sched.c:io_schedule()). I rekon Shehjar would know though... :) -i -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[Fwd: Re: [SLUG] IOWait definition]
I have a machine with a good proportion of IOWait 20-30%. It does have local disks and it performs operations on NFS mounts. I just wanted to be sure if IOWait includes NFS activity or not. I also want a way if it is NFS to be able to say for sure if it is a bottleneck on the nfs client or server. NFS is not a linux machine so visibility is not allways the best. Grant Daniel Pittman wrote: Grant Street [EMAIL PROTECTED] writes: I was wondering if someone could point me in the right direction for some doco? I don't know of any, but ... I'm finding it hard to get a definition of what constitutes IOWait. I know that IOwait is CPU time waiting for IO to happen to physical local disks, but I'm unsure about the following scenarios and if they contribute to IOWait: Not quite. IOWait is a *software* state, indicating that a process or thread is blocked waiting for I/O to complete. This is different from CPU time waiting for ... in that it implies the software is making no progress, but *NOT* that your CPU is spending cycles working on it.[1] - CPU time waiting for an NFS read/write to occur Yes, along with more or less any other disk I/O that happens to be run over the network -- as long as it is synchronous, and something is waiting on it. - CPU time waiting for a network buffer to be read/written to. eg waiting for a full buffer to clear. Generally not. I am not certain about blocking on a full buffer condition for sending data, but not for blocking while reading. - Anything else?? Any other synchronous disk I/O, certainly. Probably certain other, related, conditions where the kernel developers feel that the process is blocked on I/O. PS. How do you set/query the network buffers in Linux? Via the socket fcntl / ioctl interface, or via the sysctls in /proc/sys/net, which are documented in the standard Linux kernel sysctl documentation. All that said, you might want to tell us why you are asking, not just what, since I suspect there is a question about why you have so much IOWait time on your system, or poor performance? Regards, Daniel Footnotes: [1] It does, technically, spend a few in terms of submitting and completing the I/O before it wakes up the blocked process, and various I/O devices need babysitting, but the principal is sound. ;) Animal Logic http://www.animallogic.com Please think of the environment before printing this email. This email and any attachments may be confidential and/or privileged. If you are not the intended recipient of this email, you must not disclose or use the information contained in it. Please notify the sender immediately and delete this document if you have received it in error. We do not guarantee this email is error or virus free. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [Fwd: Re: [SLUG] IOWait definition]
Grant Street wrote: I have a machine with a good proportion of IOWait 20-30%. It does have local disks and it performs operations on NFS mounts. I just wanted to be sure if IOWait includes NFS activity or not. I also want a way if it is NFS to be able to say for sure if it is a bottleneck on the nfs client or server. NFS is not a linux machine so visibility is not allways the best. dstat might help you correlate stuff. http://dag.wieers.com/home-made/dstat/ dave -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
