Re: [Qemu-devel] [patch] non-blocking disk IO
I've been using the non-blocking IO / DMA patch with a FreeBSD host and Windows 98 guest. It seems to work well -- I haven't done any benchmarking, but subjectively it seems to make a difference. However, I tried enabling DMA in the Windows 98 system settings for the C: drive, and that doesn't work -- when I reboot the emulator, it stalls fairly early in the boot process. Have others been able to get DMA working (with the patch) with Windows 98? Is there something I could to try to track down the problem (assuming it is supposed to work). ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Tue, Oct 04 2005, Troy Benjegerdes wrote: What we want is to be able to have the guest OS request some DMA I/O operation, and have qemu be able to use AIO so that the actual disk hardware can dump the data directly in the pages the userspace process on the guest OS ends up wanting it in, avoiding several expensive memcopy and context switch operations. That should be easy enough to do already, with or without the nonblocking patch. Just make sure to open the files O_DIRECT and align the io buffers and lengths. With a 2.6 host, you can usually get away with 512-b aligment, on 2.4 you may have to ensure 1k/4k alignment. -- Jens Axboe ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Whoops. I had the mistaken belief that Qemu already used pthreads.
So, the makefiles and configure script need to handle pthreads somehow
(especially on windows.) Pthreads-w32 is refusing to 'make install'
itself into standard lib and include directories. Unlike SDL, there is
no 'pthread-config' to tell us where it lives.
Suppose that on Windows, we tell people to unpack the pthread
distribution within the qemu tree. Qemu's configure script could find
it, build a static pthread library, and link that into Qemu.
This removes the need for the user to:
* read pthreads-w32 documents,
* issue a separate build command for pthreads-w32,
* get confused before realizing that pthreads-w32 doesn't really
support 'make install',
* hack at qemu until it points to the uninstalled pthreads dir,
* copy the pthreads DLL into c:\windows
I'll look at adding this to the configure script.
-john
Christian MICHON wrote:
I managed to make it work (qemu+non blocking IO on windows host),
with a rough estimation of 10% speed increase at the early stage of
windows setup. I expect more once windows installs itself in true
multitasking mode. :)
What you need to do is:
- download pthreads-w32-2-6-0-release.tar.gz and compile it from scratch
(make clean GC)
- you also need to add the following lines inside block.c (usleep is not
present in win32/mingw32)
#ifdef WIN32
void usleep(unsigned long usec)
{
Sleep(usec/1000);
}
#endif
- you need to update the link to include -lpthreadGC2 -lz, and
keep a copy of the pthreadGC2.dll where qemu.exe will be
(to pu it just in from of -lz is just my personal suggestion)
HIH,
Christian
On 10/3/05, Elefterios Stamatogiannakis [EMAIL PROTECTED] wrote:
With which pthreads library have you compiled it under windows?
I've tried it with the one here:
ftp://sources.redhat.com/pub/pthreads-win32/
but it didn't compile.
teris.
___
Qemu-devel mailing list
Qemu-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/qemu-devel
___
Qemu-devel mailing list
Qemu-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
I did not manage a static link on win32 yesterday. I needed the pthread dll in the same directory as qemu.exe. I'll look more into it. On 10/4/05, John Coiner [EMAIL PROTECTED] wrote: Suppose that on Windows, we tell people to unpack the pthread distribution within the qemu tree. Qemu's configure script could find it, build a static pthread library, and link that into Qemu. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Then all is good and I should not use (2) as a benchmark. Thanks for clarifying :) On 10/4/05, John Coiner [EMAIL PROTECTED] wrote: (1) is a parallel task -- 'tar' and 'bunzip2' run concurrently. You see a benefit here when one process can use the CPU while the other is blocked waiting for IO. (2) is a serial task. It can't benefit from non-blocking IO. 'make -j 2' should benefit, in cases where the host file cache is cold. -- Christian ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Mon, Oct 03, 2005 at 11:16:18PM -0400, John Coiner wrote: Troy Benjegerdes wrote: I am also haveing trouble getting a fresh win2k install under qemu to actually be able to run windows update. I had to download and install Win2k SP4, then Win2k SP4 Hotfixes, and also an IE6 upgrade, before windows update ran. Are you running with a tun/tap device, or -user-net ? I'm still unable to get windowsupdate to work. Getting it to work in vmware seems to work on the first try. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
I had to download and install Win2k SP4, then Win2k SP4 Hotfixes, and also an IE6 upgrade, before windows update ran. Are you running with a tun/tap device, or -user-net ? I'm still unable to get windowsupdate to work. Getting it to work in vmware seems to work on the first try. I've been running -user-net. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] [patch] non-blocking disk IO
Non-blocking disk IO now works for any type of disk image, not just raw format. There is no longer any format-specific code in the patch: http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html You might want this patch if: * you run a multitasking guest OS, * you access a disk sometimes, and * you wouldn't mind if QEMU ran a little faster. Why I have not got feedback in droves I do not understand ;) -John ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Magnus, I don't think the Windows 2000 install hack will ever be obsolete. The installer assumes that a hard disk will take nonzero time to read some data. QEMU always services a read in zero-guest-time. (With the nonblocking IO patch, zero-guest-time reads still occur, when the requested data is in the host's file cache.) I doubt the IDE spec allows Windows to make this assumption... but the assumption is there, and we work around it by adding a delay that's visible to the guest. -- John ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
I managed to make it work (qemu+non blocking IO on windows host),
with a rough estimation of 10% speed increase at the early stage of
windows setup. I expect more once windows installs itself in true
multitasking mode. :)
What you need to do is:
- download pthreads-w32-2-6-0-release.tar.gz and compile it from scratch
(make clean GC)
- you also need to add the following lines inside block.c (usleep is not
present in win32/mingw32)
#ifdef WIN32
void usleep(unsigned long usec)
{
Sleep(usec/1000);
}
#endif
- you need to update the link to include -lpthreadGC2 -lz, and
keep a copy of the pthreadGC2.dll where qemu.exe will be
(to pu it just in from of -lz is just my personal suggestion)
HIH,
Christian
On 10/3/05, Elefterios Stamatogiannakis [EMAIL PROTECTED] wrote:
With which pthreads library have you compiled it under windows?
I've tried it with the one here:
ftp://sources.redhat.com/pub/pthreads-win32/
but it didn't compile.
teris.
___
Qemu-devel mailing list
Qemu-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Mon, Oct 03 2005, John Coiner wrote: Non-blocking disk IO now works for any type of disk image, not just raw format. There is no longer any format-specific code in the patch: http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html You might want this patch if: * you run a multitasking guest OS, * you access a disk sometimes, and * you wouldn't mind if QEMU ran a little faster. Why I have not got feedback in droves I do not understand ;) Why not use aio for this instead, seems like a better fit than spawning a thread per block device? That would still require a thread for handling completions, but you could easily just use a single completion thread for all devices for this as it would not need to do any real work. -- Jens Axboe ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Thank you for the patch. I'll look at it ASAP. If there is no regression (in terms of bugs and performance) I will include it soon: I consider that non blocking I/O is now a critical feature in QEMU in order to get better performances. Fabrice. John Coiner wrote: Non-blocking disk IO now works for any type of disk image, not just raw format. There is no longer any format-specific code in the patch: http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html You might want this patch if: * you run a multitasking guest OS, * you access a disk sometimes, and * you wouldn't mind if QEMU ran a little faster. Why I have not got feedback in droves I do not understand ;) -John ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Jens Axboe wrote: Why not use aio for this instead, seems like a better fit than spawning a thread per block device? That would still require a thread for handling completions, but you could easily just use a single completion thread for all devices for this as it would not need to do any real work. Pthreads vs. AIO is an open question. The pthreads implementation works with all disk image formats, without modifying any format-specific code. To use AIO, each image format driver would have to be modified. QEMU already uses pthreads, therefore using pthreads does not add a new dependency on the host platform. (Or so I thought. The build on Windows broke anyway. Haha... my bad...) My questions are: * Which QEMU host platforms support Posix AIO? * Which host platform has the slowest pthreads library, and how many cycles will it waste on pthreads overhead? * What is the likely performance benefit of using AIO? When I have time, I'll hack AIO into the raw image driver, and take some benchmarks on a linux host. Thanks. -John ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Mon, Oct 03, 2005 at 08:41:46AM -0400, John Coiner wrote: Magnus, I don't think the Windows 2000 install hack will ever be obsolete. The installer assumes that a hard disk will take nonzero time to read some data. QEMU always services a read in zero-guest-time. (With the nonblocking IO patch, zero-guest-time reads still occur, when the requested data is in the host's file cache.) I doubt the IDE spec allows Windows to make this assumption... but the assumption is there, and we work around it by adding a delay that's visible to the guest. There are other bugs in the IDE emulation.. I need to try this latest patch, but with the previous DMA patch (without non-blocking support) having a disk image on an AFS filesystem just did not work at all. I am also haveing trouble getting a fresh win2k install under qemu to actually be able to run windows update. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Troy Benjegerdes wrote: I am also haveing trouble getting a fresh win2k install under qemu to actually be able to run windows update. I had to download and install Win2k SP4, then Win2k SP4 Hotfixes, and also an IE6 upgrade, before windows update ran. -john ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Mon, 3 Oct 2005, Troy Benjegerdes wrote: Why is there a posix async IO 'standard', but a different linux AIO standard? Because the Posix AIO sucks quite badly both to implement and use. But all Linuxes I know of have a Posix AIO library. Some even have a Posix AIO library using Linux AIO. Having something that wraps both Linux-native AIO and posix async I/O might make supporting it a little easier to swallow on cross-platform applications. If you anyway plan on having Posix AIO support then go for the Posix AIO interface. The performance reasons why Linux AIO exists is very unlikely to be an issue to qemu as you need to be quite I/O intensive to see any performance difference. Regards Henrik ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
Why is there a posix async IO 'standard', but a different linux AIO standard? Linux has a Posix AIO library that sits on top of linux's native AIO: http://www.bullopensource.org/posix/ It's pretty new, version 0.6. We might not want qemu to depend on it, because most distros probably do not bundle it yet, so it would be an extra dependency for all new users to chase down. -john ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
If you anyway plan on having Posix AIO support then go for the Posix AIO interface. The performance reasons why Linux AIO exists is very unlikely to be an issue to qemu as you need to be quite I/O intensive to see any performance difference. Ideally, we should be able to use a Posix AIO support libary that uses native Linux AIO underneath. What we want is to be able to have the guest OS request some DMA I/O operation, and have qemu be able to use AIO so that the actual disk hardware can dump the data directly in the pages the userspace process on the guest OS ends up wanting it in, avoiding several expensive memcopy and context switch operations. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
