Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 11:01:22PM +0800, Asias He wrote: On 06/16/2011 10:28 PM, Michael S. Tsirkin wrote: On Wed, Jun 15, 2011 at 06:53:34PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support. tools/kvm/ now has a highly scalable, largely lockless driver interface and the individual drivers are using finegrained locks. - TAP-based virtio networking Wanted to ask for a while: would it make sense to use vhost-net? Or maybe use that exclusively? Less hypervisor code to support would help the focus. Sure. We are planning to use vhost-net. Just out of time right now, we are currently working on simple user model network support which allows plain user to use network without root privilege. Yes, qemu does this by implementing NAT and the TCP stack in userspace. What always made me unhappy about this solution is that we have a perfectly fine NAT and TCP in kernel, we just lack APIs to make an unpriveledged user make use of it the way we want. I hope you can avoid this duplication. Another question is whether you want to implement a dhcp server. -- MST -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/16/2011 07:22 AM, Ingo Molnar wrote: * Christoph Hellwigh...@infradead.org wrote: On Thu, Jun 16, 2011 at 12:57:36PM +0300, Pekka Enberg wrote: Uh-oh. Someone needs to apply this patch to sync_file_range(): There actually are a few cases where using it makes sense. [...] Such as? I don't think apps can actually know whether disk blocks have been 'instantiated' by a particular filesystem or not, so the manpage: Some details None of these operations write out the file’s metadata. Therefore, unless the appli- cation is strictly performing overwrites of already-instantiated disk blocks, there are no guarantees that the data will be available after a crash. is rather misleading. This is a dangerous (and rather pointless) syscall and this should be made much clearer in the manpage. Not pointless at all -- see Linus's sync_file_range() examples in Re: Unexpected splice always copy behavior observed thread from May 2010. Apps like MythTV may use it for streaming data to disk, basically shoving the VM out of the way to give the app more fine-grained writeout control. Just don't mistake sync_file_range() for a data integrity syscall. Jeff -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 2011年06月15日 23:53, Pekka Enberg wrote: or alternatively, if you already have a kernel source tree: git remote add kvm-tool git://github.com/penberg/linux-kvm.git git remote update git checkout -b kvm-tool/master kvm-tool I tried this, but it do not work, there is something wrong when I executed the 3rd git command. I tried git checkout -b tools/kvm kvm-tool/master, it seems works fine. -- justin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Fri, 2011-06-17 at 06:00 +0100, Stefan Hajnoczi wrote: On Fri, Jun 17, 2011 at 2:03 AM, Sasha Levin levinsasha...@gmail.com wrote: On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for qemu-kvm which is why it performed so much worse. Here's a summary of qemu-kvm beating tools/kvm: https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt I'd ask for a brown paper bag if I wasn't so busy eating my hat at the moment. np, it happens. Is that still with QEMU with IDE emulation, cache=writethrough, and 128MB of guest memory? Does your raw driver support multiple parallel requests? It doesn't look like it does from how I read the code. At some point, I'd be happy to help ya'll do some benchmarking against QEMU. Each virtio-blk device can process requests regardless of other virtio-blk devices, which means that we can do parallel requests for devices. Within each device, we support parallel requests in the sense that we do vectored IO for each head (which may contain multiple blocks) in the vring, we don't do multiple heads because when I've tried adding AIO I've noticed that at most there are 2-3 possible heads - and since it points to the same device it doesn't really help running them in parallel. One thing that QEMU does but I'm a little suspicious of is request merging. virtio-blk will submit those 2-3 heads using bdrv_aio_multiwrite() if they become available in the same virtqueue notify. The requests will be merged if possible. My feeling is that we should already have merged requests coming through virtio-blk and there should be no need to do any merging - which could be a workaround for a poor virtio-blk vring configuration that prevented the guest from sending large requests. However, this feature did yield performance improvements with qcow2 image files when it was introduced, so that would be interesting to look at. Are you enabling indirect descriptors on the virtio-blk vring? That should allow more requests to be made available because you don't run out of vring descriptors so easily. No, but we're usually not getting close to running out of vring descriptors either. -- Sasha. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/16/2011 08:03 PM, Sasha Levin wrote: On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: Each virtio-blk device can process requests regardless of other virtio-blk devices, which means that we can do parallel requests for devices. Within each device, we support parallel requests in the sense that we do vectored IO for each head (which may contain multiple blocks) in the vring, we don't do multiple heads because when I've tried adding AIO A scatter/gather list isn't multiple requests, it's just one. So you handle one request at a time ATM. There's nothing with that, but there's no use in saying we support it in the sense... :-) I've noticed that at most there are 2-3 possible heads - and since it points to the same device it doesn't really help running them in parallel. Sure it does. If you use the host page cache (and you do), then if you have two requests, A and B, and request A requires a disk access and request B can be satisfied from the page cache, then being able to submit both requests means that you can return B almost immediately instead of stalling out to finish A before starting B. Not to mention that modern disks work better with multiple in flight requests because they have their own cache and reordering algorithms in the drives cache. With RAID and higher end storage devices, a single device may map to multiple spindles. The only way to have them all spin at once is to submit parallel requests. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 1:44 AM, Anthony Liguori anth...@codemonkey.ws wrote: That's probably why it's fast, it doesn't preserve data integrity :( Actually, I misread the code. It does unstable writes but it does do fsync() on FLUSH. On Thu, Jun 16, 2011 at 8:41 AM, Pekka Enberg penb...@kernel.org wrote: Yes. That's fine, right? Or did we misread how virtio block devices are supposed to work? And btw, we use sync_file_range() to make sure the metadata part of a QCOW2 image is never corrupted. The rational here is that if the guest doesn't do VIRTIO_BLK_T_FLUSH, you can corrupt your _guest filesystem_ but the _image_ will still work just fine and you can do fsck on it. Also, Prasad ran xfstests and did over-night stress tests to iron out corruption issues. Now we obviously can't promise that we'll never eat your data but I can assure you that we've done as much as we've been able to with the resources we have at the moment. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
* Pekka Enberg penb...@kernel.org wrote: On Thu, Jun 16, 2011 at 1:07 AM, Alexander Graf ag...@suse.de wrote: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? We don't use AIO at all. It's just normal read()/write() with a thread pool. I actually looked at AIO but didn't really see why we'd want to use it. We could certainly try kernel AIO, it would allow us to do all the virtio-blk logic from the vcpu thread, without single threading it - turning the QCOW2 logic into an AIO driven state machine in essence. Advantages: - we wouldnt do context-switching between the vcpu thread and the helper threads - it would potentially provide tighter caching and potentially would allow higher scalability. Disadvantages: - the kaio codepaths are actually *more* complex than the regular read()/write() IO codepaths - they keep track of an 'IO context', so part of the efficiency advantages are spent on AIO tracking. - executing AIO in the vcpu thread eats up precious vcpu execution time: combined QCOW2 throughput would be limited by a single core's performance, and any time spent on QCOW2 processing would not be spent running the guest CPU. (In such a model we certainly couldnt do more intelligent, CPU-intense storage solutions like on the fly compress/decompress of QCOW2 data.) - state machines are also fragile in the sense that any unintentional blocking of the vcpu context will kill the performance and latencies of *all* processing in certain circumstances. So we generally strive to keep the vcpu demux path obvious, simple and atomic. - more advanced security models go out the window as well: we couldnt isolate drivers from each other if all of them execute in the same vcpu context ... - state machines are also notoriously difficult to develop, debug and maintain. So careful performance, scalability, IO delay and maintainability measurements have to accompany such a model switch, because the disadvantages are numerous. I'd only consider KAIO it if it provides some *real* measurable performance advantage of at least 10% in some important usecase. A few percent probably wouldnt be worth it. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
Hi Ingo, On Thu, Jun 16, 2011 at 10:24 AM, Ingo Molnar mi...@elte.hu wrote: - executing AIO in the vcpu thread eats up precious vcpu execution time: combined QCOW2 throughput would be limited by a single core's performance, and any time spent on QCOW2 processing would not be spent running the guest CPU. (In such a model we certainly couldnt do more intelligent, CPU-intense storage solutions like on the fly compress/decompress of QCOW2 data.) Most image formats have optional on-the-fly compression/decompression so we'd need to keep the current I/O thread scheme anyway. I'd only consider KAIO it if it provides some *real* measurable performance advantage of at least 10% in some important usecase. A few percent probably wouldnt be worth it. I've only been following AIO kernel development from the sidelines but I really haven't seen any reports of significant gains over read()/write() from a thread pool. Are there any such reports? Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
* Pekka Enberg penb...@kernel.org wrote: Hi Ingo, On Thu, Jun 16, 2011 at 10:24 AM, Ingo Molnar mi...@elte.hu wrote: - executing AIO in the vcpu thread eats up precious vcpu execution time: combined QCOW2 throughput would be limited by a single core's performance, and any time spent on QCOW2 processing would not be spent running the guest CPU. (In such a model we certainly couldnt do more intelligent, CPU-intense storage solutions like on the fly compress/decompress of QCOW2 data.) Most image formats have optional on-the-fly compression/decompression so we'd need to keep the current I/O thread scheme anyway. Yeah - although high-performance setups will probably not use that. I'd only consider KAIO it if it provides some *real* measurable performance advantage of at least 10% in some important usecase. A few percent probably wouldnt be worth it. I've only been following AIO kernel development from the sidelines but I really haven't seen any reports of significant gains over read()/write() from a thread pool. Are there any such reports? I've measured such gains myself a couple of years ago, using an Oracle DB and a well-known OLTP benchmark, on a 64-way system. I also profiled+tuned the kernel-side AIO implementation to be more scalable so i'm reasonably certain that the gains exist, and they were above 10%. So the kaio gains existed back then but they needed sane userspace (POSIX AIO with signal notification sucks) and needed a well-tuned in-kernel implementation as well. (the current AIO code might have bitrotted) Also, synchronous read()/write() [and scheduler() :-)] scalability improvements have not stopped in the past few years so the performance picture might have shifted in favor of a thread pool. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 8:24 AM, Ingo Molnar mi...@elte.hu wrote: - executing AIO in the vcpu thread eats up precious vcpu execution time: combined QCOW2 throughput would be limited by a single core's performance, and any time spent on QCOW2 processing would not be spent running the guest CPU. (In such a model we certainly couldnt do more intelligent, CPU-intense storage solutions like on the fly compress/decompress of QCOW2 data.) This has been a problem in qemu-kvm. io_submit(2) steals time from the guest (I think it was around 20us on the system I measured last year). Add the fact that the guest kernel might be holding a spinlock and it becomes a scalability problem for SMP guests. Anything that takes noticable CPU time should be done outside the vcpu thread. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 09:21:03AM +0300, Pekka Enberg wrote: And btw, we use sync_file_range() Which doesn't help you at all. sync_file_range is just a hint for VM writeback, but never commits filesystem metadata nor the physical disk's write cache. In short it's a completely dangerous interface, and that is pretty well documented in the man page. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
Hi Christoph, On Thu, Jun 16, 2011 at 09:21:03AM +0300, Pekka Enberg wrote: And btw, we use sync_file_range() On Thu, Jun 16, 2011 at 12:24 PM, Christoph Hellwig h...@infradead.org wrote: Which doesn't help you at all. sync_file_range is just a hint for VM writeback, but never commits filesystem metadata nor the physical disk's write cache. In short it's a completely dangerous interface, and that is pretty well documented in the man page. Doh - I didn't read it carefully enough and got hung up with: Therefore, unless the application is strictly performing overwrites of already-instantiated disk blocks, there are no guarantees that the data will be available after a crash. without noticing that it obviously doesn't work with filesystems like btrfs that do copy-on-write. What's the right thing to do here? Is fdatasync() sufficient? Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 12:34:04PM +0300, Pekka Enberg wrote: Hi Christoph, On Thu, Jun 16, 2011 at 09:21:03AM +0300, Pekka Enberg wrote: And btw, we use sync_file_range() On Thu, Jun 16, 2011 at 12:24 PM, Christoph Hellwig h...@infradead.org wrote: Which doesn't help you at all. ?sync_file_range is just a hint for VM writeback, but never commits filesystem metadata nor the physical disk's write cache. ?In short it's a completely dangerous interface, and that is pretty well documented in the man page. Doh - I didn't read it carefully enough and got hung up with: Therefore, unless the application is strictly performing overwrites of already-instantiated disk blocks, there are no guarantees that the data will be available after a crash. without noticing that it obviously doesn't work with filesystems like btrfs that do copy-on-write. You also missed: This system call does not flush disk write caches and thus does not provide any data integrity on systems with volatile disk write caches. so it's not safe if you either have a cache, or are using btrfs, or are using a sparse image, or are using an image preallocated using fallocate/posix_fallocate. What's the right thing to do here? Is fdatasync() sufficient? Yes. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
* Christoph Hellwig h...@infradead.org wrote: What's the right thing to do here? Is fdatasync() sufficient? Yes. Prasad, Pekka, mind redoing the numbers with fdatasync()? I'd be surprised if they were significantly worse but it has to be done to have apples-to-apples numbers. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 12:48 PM, Christoph Hellwig h...@infradead.org wrote: You also missed: This system call does not flush disk write caches and thus does not provide any data integrity on systems with volatile disk write caches. so it's not safe if you either have a cache, or are using btrfs, or are using a sparse image, or are using an image preallocated using fallocate/posix_fallocate. Uh-oh. Someone needs to apply this patch to sync_file_range(): diff --git a/fs/sync.c b/fs/sync.c index ba76b96..32078aa 100644 --- a/fs/sync.c +++ b/fs/sync.c @@ -277,6 +277,8 @@ SYSCALL_DEFINE(sync_file_range)(int fd, loff_t offset, loff_t nbytes, int fput_needed; umode_t i_mode; + WARN_ONCE(1, when this breaks, you get to keep both pieces); + ret = -EINVAL; if (flags ~VALID_FLAGS) goto out; What's the right thing to do here? Is fdatasync() sufficient? Yes. We'll fix that up. Thanks Christoph! Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 12:57:36PM +0300, Pekka Enberg wrote: Uh-oh. Someone needs to apply this patch to sync_file_range(): There actually are a few cases where using it makes sense. It's just the minority. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
* Christoph Hellwig h...@infradead.org wrote: On Thu, Jun 16, 2011 at 12:57:36PM +0300, Pekka Enberg wrote: Uh-oh. Someone needs to apply this patch to sync_file_range(): There actually are a few cases where using it makes sense. [...] Such as? I don't think apps can actually know whether disk blocks have been 'instantiated' by a particular filesystem or not, so the manpage: Some details None of these operations write out the file’s metadata. Therefore, unless the appli- cation is strictly performing overwrites of already-instantiated disk blocks, there are no guarantees that the data will be available after a crash. is rather misleading. This is a dangerous (and rather pointless) syscall and this should be made much clearer in the manpage. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 01:22:30PM +0200, Ingo Molnar wrote: Such as? I don't think apps can actually know whether disk blocks have been 'instantiated' by a particular filesystem or not, so the manpage: In general they can't. The only good use case for sync_file_range is to paper over^H^H^H^H^H^H^H^H^Hcontrol write back behaviour. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
* Christoph Hellwig h...@infradead.org wrote: On Thu, Jun 16, 2011 at 01:22:30PM +0200, Ingo Molnar wrote: Such as? I don't think apps can actually know whether disk blocks have been 'instantiated' by a particular filesystem or not, so the manpage: In general they can't. The only good use case for sync_file_range is to paper over^H^H^H^H^H^H^H^H^Hcontrol write back behaviour. Well, if overwrite is fundamentally safe on a filesystem (which is most of them) then sync_file_range() would work - and it has the big advantage that it's a pretty simple facility. Filesystems that cannot guarantee that should map their sync_file_range() implementation to fdatasync() or so, right? Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 01:40:45PM +0200, Ingo Molnar wrote: Filesystems that cannot guarantee that should map their sync_file_range() implementation to fdatasync() or so, right? Filesystems aren't even told about sync_file_range, it's purely a VM thing, which is the root of the problem. In-kernel we have all the infrastructure for a real ranged fsync/fdatasync, and once we get a killer users for that can triviall export it at the syscall level. I don't think mapping sync_file_range with it's weird set of flags and confusing behaviour to it is a good idea, though. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, Jun 15, 2011 at 06:53:34PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support. tools/kvm/ now has a highly scalable, largely lockless driver interface and the individual drivers are using finegrained locks. - TAP-based virtio networking Wanted to ask for a while: would it make sense to use vhost-net? Or maybe use that exclusively? Less hypervisor code to support would help the focus. -- MST -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enberg penb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for qemu-kvm which is why it performed so much worse. Here's a summary of qemu-kvm beating tools/kvm: https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt I'd ask for a brown paper bag if I wasn't so busy eating my hat at the moment. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/16/2011 10:28 PM, Michael S. Tsirkin wrote: On Wed, Jun 15, 2011 at 06:53:34PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support. tools/kvm/ now has a highly scalable, largely lockless driver interface and the individual drivers are using finegrained locks. - TAP-based virtio networking Wanted to ask for a while: would it make sense to use vhost-net? Or maybe use that exclusively? Less hypervisor code to support would help the focus. Sure. We are planning to use vhost-net. Just out of time right now, we are currently working on simple user model network support which allows plain user to use network without root privilege. -- Best Regards, Asias He -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for qemu-kvm which is why it performed so much worse. Here's a summary of qemu-kvm beating tools/kvm: https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt I'd ask for a brown paper bag if I wasn't so busy eating my hat at the moment. np, it happens. Is that still with QEMU with IDE emulation, cache=writethrough, and 128MB of guest memory? Does your raw driver support multiple parallel requests? It doesn't look like it does from how I read the code. At some point, I'd be happy to help ya'll do some benchmarking against QEMU. It would be very useful to compare as we have some ugly things in QEMU that we've never quite been able to determine how much they affect performance. Having an alternative implementation to benchmark against would be quite helpful. Regards, Anthony Liguori Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for qemu-kvm which is why it performed so much worse. Here's a summary of qemu-kvm beating tools/kvm: https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt I'd ask for a brown paper bag if I wasn't so busy eating my hat at the moment. np, it happens. Is that still with QEMU with IDE emulation, cache=writethrough, and 128MB of guest memory? Does your raw driver support multiple parallel requests? It doesn't look like it does from how I read the code. At some point, I'd be happy to help ya'll do some benchmarking against QEMU. Each virtio-blk device can process requests regardless of other virtio-blk devices, which means that we can do parallel requests for devices. Within each device, we support parallel requests in the sense that we do vectored IO for each head (which may contain multiple blocks) in the vring, we don't do multiple heads because when I've tried adding AIO I've noticed that at most there are 2-3 possible heads - and since it points to the same device it doesn't really help running them in parallel. It would be very useful to compare as we have some ugly things in QEMU that we've never quite been able to determine how much they affect performance. Having an alternative implementation to benchmark against would be quite helpful. -- Sasha. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Fri, Jun 17, 2011 at 2:03 AM, Sasha Levin levinsasha...@gmail.com wrote: On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for qemu-kvm which is why it performed so much worse. Here's a summary of qemu-kvm beating tools/kvm: https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt I'd ask for a brown paper bag if I wasn't so busy eating my hat at the moment. np, it happens. Is that still with QEMU with IDE emulation, cache=writethrough, and 128MB of guest memory? Does your raw driver support multiple parallel requests? It doesn't look like it does from how I read the code. At some point, I'd be happy to help ya'll do some benchmarking against QEMU. Each virtio-blk device can process requests regardless of other virtio-blk devices, which means that we can do parallel requests for devices. Within each device, we support parallel requests in the sense that we do vectored IO for each head (which may contain multiple blocks) in the vring, we don't do multiple heads because when I've tried adding AIO I've noticed that at most there are 2-3 possible heads - and since it points to the same device it doesn't really help running them in parallel. One thing that QEMU does but I'm a little suspicious of is request merging. virtio-blk will submit those 2-3 heads using bdrv_aio_multiwrite() if they become available in the same virtqueue notify. The requests will be merged if possible. My feeling is that we should already have merged requests coming through virtio-blk and there should be no need to do any merging - which could be a workaround for a poor virtio-blk vring configuration that prevented the guest from sending large requests. However, this feature did yield performance improvements with qcow2 image files when it was introduced, so that would be interesting to look at. Are you enabling indirect descriptors on the virtio-blk vring? That should allow more requests to be made available because you don't run out of vring descriptors so easily. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 3:48 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enberg penb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for qemu-kvm which is why it performed so much worse. Here's a summary of qemu-kvm beating tools/kvm: https://raw.github.com/gist/1029359/9f9a714ecee64802c08a3455971e410d5029370b/gistfile1.txt Thanks for digging into the results so quickly and rerunning. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[ANNOUNCE] Native Linux KVM tool v2
Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support. tools/kvm/ now has a highly scalable, largely lockless driver interface and the individual drivers are using finegrained locks. - TAP-based virtio networking - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 - Virtio-9p support for host filesystem access in guests - Virtio Random Number Generator - Host block devices as in-memory copy-on-write guest images on 64-bit hosts 1. To try out the tool, clone the git repository: git clone git://github.com/penberg/linux-kvm.git or alternatively, if you already have a kernel source tree: git remote add kvm-tool git://github.com/penberg/linux-kvm.git git remote update git checkout -b kvm-tool/master kvm-tool 2. Compile the tool: cd tools/kvm make 3. Download a raw userspace image: Minimal: wget http://wiki.qemu.org/download/linux-0.2.img.bz2 bunzip2 linux-0.2.img.bz2 Debian Squeeze QCOW2 image: wget http://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2 4. Build a kernel with the following options: CONFIG_VIRTIO_BLK=y CONFIG_VIRTIO_NET=y CONFIG_VIRTIO_CONSOLE=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_HW_RANDOM_VIRTIO=y CONFIG_FB_VESA=y Note: also make sure you have CONFIG_EXT2_FS or CONFIG_EXT4_FS if you use the above images. 5. And finally, launch the hypervisor: ./kvm run -d linux-0.2.img or ./kvm run -d debian_squeeze_i386_standard.qcow2 or sudo ./kvm run -d linux-0.2.img -n virtio This release was brought to you by the following people: Sasha Levin Pekka Enberg Asias He Prasad Joshi Cyrill Gorcunov Ingo Molnar John Floren Amos Kong Giuseppe Calderaro Amerigo Wang Paul Bolle David Ahern Most of us developers are hanging out on #pvm channel at irc.freenode.net if you want to drop by for questions, comments, and bug reports. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? btw the dump above is a little hard to interpret. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. btw the dump above is a little hard to interpret. It's what fio reports. The relevant bits are: Qemu: Run status group 0 (all jobs): READ: io=204800KB, aggrb=61152KB/s, minb=15655KB/s, maxb=17845KB/s, mint=2938msec, maxt=3349msec WRITE: io=68544KB, aggrb=28045KB/s, minb=6831KB/s, maxb=7858KB/s, mint=2292msec, maxt=2444msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=61779KB/s, minb=15815KB/s, maxb=17189KB/s, mint=3050msec, maxt=3315msec WRITE: io=66576KB, aggrb=24165KB/s, minb=6205KB/s, maxb=7166KB/s, mint=2485msec, maxt=2755msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=6722KB/s, minb=1720KB/s, maxb=1737KB/s, mint=30178msec, maxt=30467msec WRITE: io=65424KB, aggrb=2156KB/s, minb=550KB/s, maxb=573KB/s, mint=29682msec, maxt=30342msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=6994KB/s, minb=1790KB/s, maxb=1834KB/s, mint=28574msec, maxt=29279msec WRITE: io=68192KB, aggrb=2382KB/s, minb=548KB/s, maxb=740KB/s, mint=27121msec, maxt=28625msec Disk stats (read/write): sdb: ios=60583/6652, merge=0/164, ticks=156340/672030, in_queue=828230, util=82.71% tools/kvm: Run status group 0 (all jobs): READ: io=204800KB, aggrb=149162KB/s, minb=38185KB/s, maxb=46030KB/s, mint=1139msec, maxt=1373msec WRITE: io=70528KB, aggrb=79156KB/s, minb=18903KB/s, maxb=23726KB/s, mint=804msec, maxt=891msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=188235KB/s, minb=48188KB/s, maxb=57932KB/s, mint=905msec, maxt=1088msec WRITE: io=64464KB, aggrb=84821KB/s, minb=21751KB/s, maxb=27392KB/s, mint=570msec, maxt=760msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=20005KB/s, minb=5121KB/s, maxb=5333KB/s, mint=9830msec, maxt=10237msec WRITE: io=66624KB, aggrb=6615KB/s, minb=1671KB/s, maxb=1781KB/s, mint=9558msec, maxt=10071msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=66149KB/s, minb=16934KB/s, maxb=17936KB/s, mint=2923msec, maxt=3096msec WRITE: io=69600KB, aggrb=26717KB/s, minb=6595KB/s, maxb=7342KB/s, mint=2530msec, maxt=2605msec Disk stats (read/write): vdb: ios=61002/6654, merge=0/183, ticks=27270/205780, in_queue=232220, util=69.46% -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hda disk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M FIO job file used for measuring the numbers was prasad@prasad-vm:~$ cat fio-mixed.job ; fio-mixed.job for autotest [global] name=fio-sync directory=/mnt rw=randrw rwmixread=67 rwmixwrite=33 bsrange=16K-256K direct=0 end_fsync=1 verify=crc32 ;ioscheduler=x numjobs=4 [file1] size=50M ioengine=sync mem=malloc [file2] stonewall size=50M ioengine=aio mem=shm iodepth=4 [file3] stonewall size=50M ioengine=mmap mem=mmap direct=1 [file4] stonewall size=50M ioengine=splice mem=malloc direct=1 - The test generates 16 file each of ~50MB, so in total ~800MB data was written. - The test.qcow2 was newly created before it was used with QEMU or KVM tool - The size of the QCOW2 image was 1.5GB. - The host machine had 2GB RAM. - The guest machine in both the cases was started with 1GB memory. Thanks and Regards, Prasad btw the dump above is a little hard to interpret. It's what fio reports. The relevant bits are: Qemu: Run status group 0 (all jobs): READ: io=204800KB, aggrb=61152KB/s, minb=15655KB/s, maxb=17845KB/s, mint=2938msec, maxt=3349msec WRITE: io=68544KB, aggrb=28045KB/s, minb=6831KB/s, maxb=7858KB/s, mint=2292msec, maxt=2444msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=61779KB/s, minb=15815KB/s, maxb=17189KB/s, mint=3050msec, maxt=3315msec WRITE: io=66576KB, aggrb=24165KB/s, minb=6205KB/s, maxb=7166KB/s, mint=2485msec, maxt=2755msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=6722KB/s, minb=1720KB/s, maxb=1737KB/s, mint=30178msec, maxt=30467msec WRITE: io=65424KB, aggrb=2156KB/s, minb=550KB/s, maxb=573KB/s, mint=29682msec, maxt=30342msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=6994KB/s, minb=1790KB/s, maxb=1834KB/s, mint=28574msec, maxt=29279msec WRITE: io=68192KB, aggrb=2382KB/s, minb=548KB/s, maxb=740KB/s, mint=27121msec, maxt=28625msec Disk stats (read/write): sdb: ios=60583/6652, merge=0/164, ticks=156340/672030, in_queue=828230, util=82.71% tools/kvm: Run status group 0 (all jobs): READ: io=204800KB, aggrb=149162KB/s, minb=38185KB/s, maxb=46030KB/s, mint=1139msec, maxt=1373msec WRITE: io=70528KB, aggrb=79156KB/s, minb=18903KB/s, maxb=23726KB/s, mint=804msec, maxt=891msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=188235KB/s, minb=48188KB/s, maxb=57932KB/s, mint=905msec, maxt=1088msec WRITE: io=64464KB, aggrb=84821KB/s, minb=21751KB/s, maxb=27392KB/s, mint=570msec, maxt=760msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=20005KB/s, minb=5121KB/s, maxb=5333KB/s, mint=9830msec, maxt=10237msec WRITE: io=66624KB, aggrb=6615KB/s, minb=1671KB/s, maxb=1781KB/s, mint=9558msec, maxt=10071msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=66149KB/s, minb=16934KB/s, maxb=17936KB/s, mint=2923msec, maxt=3096msec WRITE: io=69600KB, aggrb=26717KB/s, minb=6595KB/s, maxb=7342KB/s, mint=2530msec, maxt=2605msec Disk stats (read/write): vdb: ios=61002/6654, merge=0/183, ticks=27270/205780, in_queue=232220, util=69.46% -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, 2011-06-15 at 21:13 +0100, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hda disk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Prasad, Could you please run this test with '-drive file=/dev/shm/test.qcow2,if=virtio' instead of the '-hdb' thing? FIO job file used for measuring the numbers was prasad@prasad-vm:~$ cat fio-mixed.job ; fio-mixed.job for autotest [global] name=fio-sync directory=/mnt rw=randrw rwmixread=67 rwmixwrite=33 bsrange=16K-256K direct=0 end_fsync=1 verify=crc32 ;ioscheduler=x numjobs=4 [file1] size=50M ioengine=sync mem=malloc [file2] stonewall size=50M ioengine=aio mem=shm iodepth=4 [file3] stonewall size=50M ioengine=mmap mem=mmap direct=1 [file4] stonewall size=50M ioengine=splice mem=malloc direct=1 - The test generates 16 file each of ~50MB, so in total ~800MB data was written. - The test.qcow2 was newly created before it was used with QEMU or KVM tool - The size of the QCOW2 image was 1.5GB. - The host machine had 2GB RAM. - The guest machine in both the cases was started with 1GB memory. Thanks and Regards, Prasad btw the dump above is a little hard to interpret. It's what fio reports. The relevant bits are: Qemu: Run status group 0 (all jobs): READ: io=204800KB, aggrb=61152KB/s, minb=15655KB/s, maxb=17845KB/s, mint=2938msec, maxt=3349msec WRITE: io=68544KB, aggrb=28045KB/s, minb=6831KB/s, maxb=7858KB/s, mint=2292msec, maxt=2444msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=61779KB/s, minb=15815KB/s, maxb=17189KB/s, mint=3050msec, maxt=3315msec WRITE: io=66576KB, aggrb=24165KB/s, minb=6205KB/s, maxb=7166KB/s, mint=2485msec, maxt=2755msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=6722KB/s, minb=1720KB/s, maxb=1737KB/s, mint=30178msec, maxt=30467msec WRITE: io=65424KB, aggrb=2156KB/s, minb=550KB/s, maxb=573KB/s, mint=29682msec, maxt=30342msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=6994KB/s, minb=1790KB/s, maxb=1834KB/s, mint=28574msec, maxt=29279msec WRITE: io=68192KB, aggrb=2382KB/s, minb=548KB/s, maxb=740KB/s, mint=27121msec, maxt=28625msec Disk stats (read/write): sdb: ios=60583/6652, merge=0/164, ticks=156340/672030, in_queue=828230, util=82.71% tools/kvm: Run status group 0 (all jobs): READ: io=204800KB, aggrb=149162KB/s, minb=38185KB/s, maxb=46030KB/s, mint=1139msec, maxt=1373msec WRITE: io=70528KB, aggrb=79156KB/s, minb=18903KB/s, maxb=23726KB/s, mint=804msec, maxt=891msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=188235KB/s, minb=48188KB/s, maxb=57932KB/s, mint=905msec, maxt=1088msec WRITE: io=64464KB, aggrb=84821KB/s, minb=21751KB/s, maxb=27392KB/s, mint=570msec, maxt=760msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=20005KB/s, minb=5121KB/s, maxb=5333KB/s, mint=9830msec, maxt=10237msec WRITE: io=66624KB, aggrb=6615KB/s, minb=1671KB/s, maxb=1781KB/s, mint=9558msec, maxt=10071msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=66149KB/s, minb=16934KB/s, maxb=17936KB/s, mint=2923msec, maxt=3096msec WRITE: io=69600KB, aggrb=26717KB/s, minb=6595KB/s, maxb=7342KB/s, mint=2530msec, maxt=2605msec Disk stats (read/write): vdb: ios=61002/6654, merge=0/183, ticks=27270/205780, in_queue=232220, util=69.46% -- Sasha. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, Jun 15, 2011 at 9:23 PM, Sasha Levin levinsasha...@gmail.com wrote: On Wed, 2011-06-15 at 21:13 +0100, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hda disk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Prasad, Could you please run this test with '-drive file=/dev/shm/test.qcow2,if=virtio' instead of the '-hdb' thing? Infact I have already tried that. Like Pekka mentioned, the results are still in favour of KVM tools. I machine that I work on is not with me at the moment, I will be able to mail the exact numbers tomorrow. Thanks and Regards, Prasad FIO job file used for measuring the numbers was prasad@prasad-vm:~$ cat fio-mixed.job ; fio-mixed.job for autotest [global] name=fio-sync directory=/mnt rw=randrw rwmixread=67 rwmixwrite=33 bsrange=16K-256K direct=0 end_fsync=1 verify=crc32 ;ioscheduler=x numjobs=4 [file1] size=50M ioengine=sync mem=malloc [file2] stonewall size=50M ioengine=aio mem=shm iodepth=4 [file3] stonewall size=50M ioengine=mmap mem=mmap direct=1 [file4] stonewall size=50M ioengine=splice mem=malloc direct=1 - The test generates 16 file each of ~50MB, so in total ~800MB data was written. - The test.qcow2 was newly created before it was used with QEMU or KVM tool - The size of the QCOW2 image was 1.5GB. - The host machine had 2GB RAM. - The guest machine in both the cases was started with 1GB memory. Thanks and Regards, Prasad btw the dump above is a little hard to interpret. It's what fio reports. The relevant bits are: Qemu: Run status group 0 (all jobs): READ: io=204800KB, aggrb=61152KB/s, minb=15655KB/s, maxb=17845KB/s, mint=2938msec, maxt=3349msec WRITE: io=68544KB, aggrb=28045KB/s, minb=6831KB/s, maxb=7858KB/s, mint=2292msec, maxt=2444msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=61779KB/s, minb=15815KB/s, maxb=17189KB/s, mint=3050msec, maxt=3315msec WRITE: io=66576KB, aggrb=24165KB/s, minb=6205KB/s, maxb=7166KB/s, mint=2485msec, maxt=2755msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=6722KB/s, minb=1720KB/s, maxb=1737KB/s, mint=30178msec, maxt=30467msec WRITE: io=65424KB, aggrb=2156KB/s, minb=550KB/s, maxb=573KB/s, mint=29682msec, maxt=30342msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=6994KB/s, minb=1790KB/s, maxb=1834KB/s, mint=28574msec, maxt=29279msec WRITE: io=68192KB, aggrb=2382KB/s, minb=548KB/s, maxb=740KB/s, mint=27121msec, maxt=28625msec Disk stats (read/write): sdb: ios=60583/6652, merge=0/164, ticks=156340/672030, in_queue=828230, util=82.71% tools/kvm: Run status group 0 (all jobs): READ: io=204800KB, aggrb=149162KB/s, minb=38185KB/s, maxb=46030KB/s, mint=1139msec, maxt=1373msec WRITE: io=70528KB, aggrb=79156KB/s, minb=18903KB/s, maxb=23726KB/s, mint=804msec, maxt=891msec Run status group 1 (all jobs): READ: io=204800KB, aggrb=188235KB/s, minb=48188KB/s, maxb=57932KB/s, mint=905msec, maxt=1088msec WRITE: io=64464KB, aggrb=84821KB/s, minb=21751KB/s, maxb=27392KB/s, mint=570msec, maxt=760msec Run status group 2 (all jobs): READ: io=204800KB, aggrb=20005KB/s, minb=5121KB/s, maxb=5333KB/s, mint=9830msec, maxt=10237msec WRITE: io=66624KB, aggrb=6615KB/s, minb=1671KB/s, maxb=1781KB/s, mint=9558msec, maxt=10071msec Run status group 3 (all jobs): READ: io=204800KB, aggrb=66149KB/s, minb=16934KB/s, maxb=17936KB/s, mint=2923msec, maxt=3096msec WRITE: io=69600KB, aggrb=26717KB/s, minb=6595KB/s, maxb=7342KB/s, mint=2530msec, maxt=2605msec Disk stats (read/write): vdb: ios=61002/6654, merge=0/183, ticks=27270/205780, in_queue=232220, util=69.46% -- Sasha. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/15/2011 10:53 AM, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support. tools/kvm/ now has a highly scalable, largely lockless driver interface and the individual drivers are using finegrained locks. - TAP-based virtio networking - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 What was the commit hash for the QEMU you tested? The following caused a major regression in qcow2: commit a16c53b101a9897b0b2be96a1bb3bde7c04380f2 Author: Anthony Liguori aligu...@us.ibm.com Date: Mon Jun 6 08:25:06 2011 -0500 Fix regression introduced by -machine accel= Commit 85097db6 changed the timing when kvm_allowed is set until after kvm is initialized. During initialization, the ioeventfd initialization cod checks kvm_enabled() and after this change, ioeventfd is effectively disable If it's not in your tree, it would be useful to rerun the test with the latest git. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. Our experience has been that this is actually not a great way to simulate fast storage. Spindle based storage has very different characteristics than memory as there is a significant cost for seeking. -hdb uses IDE too. That's pretty unfair since IDE is limited to a single request at a time whereas virtio can support multiple requests (and native kvm tools is using virtio). Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 16.06.2011, at 00:04, Anthony Liguori wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hdadisk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Looking more closely at native KVM tools, you would need to use the following invocation to have an apples-to-apples comparison: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/15/2011 05:07 PM, Alexander Graf wrote: On 16.06.2011, at 00:04, Anthony Liguori wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hdadisk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Looking more closely at native KVM tools, you would need to use the following invocation to have an apples-to-apples comparison: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? Nope. The relevant code is: /* blk device ?*/ disk= blkdev__probe(filename, st); if (disk) return disk; fd = open(filename, readonly ? O_RDONLY : O_RDWR); if (fd 0) return NULL; /* qcow image ?*/ disk= qcow_probe(fd, readonly); if (disk) return disk; /* raw image ?*/ disk= raw_image__probe(fd, st, readonly); if (disk) return disk; It uses a synchronous I/O model similar to qcow2 in QEMU with what I assume is a global lock that's outside of the actual implementation. I think it lacks some of the caching that Kevin's added recently though so I assume that if QEMU was run with cache=writeback, it would probably do quite a bit better than native KVM tool. It also turns out that while they have the infrastructure to deal with FLUSH, they don't implement it for qcow2 :-/ So even if the guest does an fsync(), it native KVM tool will never actually sync the data to disk... That's probably why it's fast, it doesn't preserve data integrity :( Regards, Anthony Liguori Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On 06/15/2011 05:20 PM, Anthony Liguori wrote: On 06/15/2011 05:07 PM, Alexander Graf wrote: On 16.06.2011, at 00:04, Anthony Liguori wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hdadisk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Looking more closely at native KVM tools, you would need to use the following invocation to have an apples-to-apples comparison: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? Nope. The relevant code is: /* blk device ?*/ disk = blkdev__probe(filename, st); if (disk) return disk; fd = open(filename, readonly ? O_RDONLY : O_RDWR); if (fd 0) return NULL; /* qcow image ?*/ disk = qcow_probe(fd, readonly); if (disk) return disk; /* raw image ?*/ disk = raw_image__probe(fd, st, readonly); if (disk) return disk; It uses a synchronous I/O model similar to qcow2 in QEMU with what I assume is a global lock that's outside of the actual implementation. I think it lacks some of the caching that Kevin's added recently though so I assume that if QEMU was run with cache=writeback, it would probably do quite a bit better than native KVM tool. It also turns out that while they have the infrastructure to deal with FLUSH, they don't implement it for qcow2 :-/ So even if the guest does an fsync(), it native KVM tool will never actually sync the data to disk... That's probably why it's fast, it doesn't preserve data integrity :( Actually, I misread the code. It does unstable writes but it does do fsync() on FLUSH. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Wed, Jun 15, 2011 at 11:04 PM, Anthony Liguori anth...@codemonkey.ws wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hdadisk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Looking more closely at native KVM tools, you would need to use the following invocation to have an apples-to-apples comparison: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio In addition to this it is important to set identical guest RAM sizes (QEMU's -m ram_mb) option. If you are comparing with qemu.git rather than qemu-kvm.git then you need to ./configure --enable-io-thread and launch with QEMU's -enable-kvm option. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 1:44 AM, Anthony Liguori anth...@codemonkey.ws wrote: That's probably why it's fast, it doesn't preserve data integrity :( Actually, I misread the code. It does unstable writes but it does do fsync() on FLUSH. Yes. That's fine, right? Or did we misread how virtio block devices are supposed to work? Btw, unstable writes doesn't really explain why *read* performance is better. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 8:29 AM, Stefan Hajnoczi stefa...@gmail.com wrote: On Wed, Jun 15, 2011 at 11:04 PM, Anthony Liguori anth...@codemonkey.ws wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? Prasad will have the details. Please note that the above are with Qemu defaults which doesn't use virtio. The results with virtio are little better but still in favor of tools/kvm. The qcow2 image used for testing was copied on to /dev/shm to avoid the disk delays in performance measurement. QEMU was invoked with following parameters $ qemu-system-x86_64 -hdadisk image on hard disk -hdb /dev/shm/test.qcow2 -m 1024M Looking more closely at native KVM tools, you would need to use the following invocation to have an apples-to-apples comparison: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio In addition to this it is important to set identical guest RAM sizes (QEMU's -m ram_mb) option. Right. If you are comparing with qemu.git rather than qemu-kvm.git then you need to ./configure --enable-io-thread and launch with QEMU's -enable-kvm option. I think Prasad is testing qemu-kvm.git. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool v2
On Thu, Jun 16, 2011 at 1:07 AM, Alexander Graf ag...@suse.de wrote: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? We don't use AIO at all. It's just normal read()/write() with a thread pool. I actually looked at AIO but didn't really see why we'd want to use it. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Sat, Apr 09, 2011 at 09:40:09AM +0200, Ingo Molnar wrote: * Andrea Arcangeli aarca...@redhat.com wrote: [...] I thought the whole point of a native kvm tool was to go all the paravirt way to provide max performance and maybe also depend on vhost as much as possible. BTW, I should elaborate on the all the paravirt way, going 100% paravirt isn't what I meant. I was thinking at the performance critical drivers mainly like storage and network. The kvm tool could be more hackable and evolve faster by exposing a single hardware view to the linux guest (using only paravirt whenever that improves performance, like network/storage). Whenever full emulation doesn't affect any fast path, it should be preferred rather than inventing new paravirt interfaces for no good. That for example applies first and foremost to the EPT support which is simpler and more optimal than any shadow paravirt pagetables. It'd be a dead end to do all in paravirt performance-wise. I definitely didn't mean any resemblance to lguest when I said full paravirt ;). Sorry for the confusion. To me it's more than that: today i can use it to minimally boot test various native bzImages just by typing: kvm run ./bzImage this will get me past most of the kernel init, up to the point where it would try to mount user-space. ( That's rather powerful to me personally, as i introduce most of my bugs to these stages of kernel bootup - and as a kernel developer i'm not alone there ;-) I would be sad if i were forced to compile in some sort of paravirt support, just to be able to boot-test random native kernel images. Really, if you check the code, serial console and timer support is not a big deal complexity-wise and it is rather useful: Agree with that. git pull git://github.com/penberg/linux-kvm master So i think up to a point hardware emulation is both fun to implement (it's fun to be on the receiving end of hw calls, for a change) and a no-brainer to have from a usability POV. How far it wants to go we'll see! :-) About using the kvm tool as a debugging tool I don't see the point though. It's very unlikely the kvm tool will ever be able to match qemu power and capabilities for debugging, in fact qemu will allow you to do basic debug of several device drivers too (e1000, IDE etc...). I don't really see the point of the kvm tool as a debugging tool considering how qemu is mature in terms of monitor memory inspection commands and gdbstub for that, if it's debug you're going after adding more features to the qemu monitor looks a better way to go. The only way I see this useful is to lead it into a full performance direction, using paravirt whenever it saves CPU (like virtio-blk, vhost-net) and allow it to scale to hundred of cpus doing I/O simultaneously and get there faster than qemu. Now smp scaling with qemu-kvm driver backends hasn't been a big issue according to Avi, so it's not like we're under pressure from it, but clearly someday it may become a bigger issue and having less drivers to deal with (especially only having vhost-blk in userland with vhost-net already being in the kernel) may provide an advantage in allowing a more performance oriented implementation of the backends without breaking lots of existing and valuable full-emulated drivers. In terms of pure kernel debugging I'm afraid this will be dead end and for the kernel testing you describe I think qemu-kvm will work best already. We already have a simpler kvm support in qemu (vs qemu-kvm) and we don't want a third that is even slower than qemu kvm support, so it has to be faster than qemu-kvm or nothing IMHO :). -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/09/2011 01:59 AM, Anthony Liguori wrote: Yeah, if that's the goal, skip all the mini-BIOS junk and just rely on a PV kernel in the guest. I think a mini userspace that assumes that we can change the guest kernel and avoids having a ton of complexity to do things like CMOS emulation would be a really interesting thing to do. Changing the guest kernel is a lot more complicated than writing a full BIOS or the legacy devices needed to support it. See Xen for an example. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
* Andrea Arcangeli aarca...@redhat.com wrote: [...] I thought the whole point of a native kvm tool was to go all the paravirt way to provide max performance and maybe also depend on vhost as much as possible. To me it's more than that: today i can use it to minimally boot test various native bzImages just by typing: kvm run ./bzImage this will get me past most of the kernel init, up to the point where it would try to mount user-space. ( That's rather powerful to me personally, as i introduce most of my bugs to these stages of kernel bootup - and as a kernel developer i'm not alone there ;-) I would be sad if i were forced to compile in some sort of paravirt support, just to be able to boot-test random native kernel images. Really, if you check the code, serial console and timer support is not a big deal complexity-wise and it is rather useful: git pull git://github.com/penberg/linux-kvm master So i think up to a point hardware emulation is both fun to implement (it's fun to be on the receiving end of hw calls, for a change) and a no-brainer to have from a usability POV. How far it wants to go we'll see! :-) Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Fri, Apr 08, 2011 at 09:00:43AM -0500, Anthony Liguori wrote: Really, having a flat table doesn't make sense. You should just send everything to an i440fx directly. Then the i440fx should decode what it can, and send it to the next level, and so forth. No you shouldn't. The i440fx should merge and arbitrate the mappings and then push *direct* links to the handling functions at the top level. Mapping changes don't happen often on modern hardware, and decoding is expensive. Incidentally, you can have special handling functions which are in reality references to kernel handlers, shortcutting userspace entirely for critical ports/mmio ranges. OG. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/09/2011 01:23 PM, Olivier Galibert wrote: On Fri, Apr 08, 2011 at 09:00:43AM -0500, Anthony Liguori wrote: Really, having a flat table doesn't make sense. You should just send everything to an i440fx directly. Then the i440fx should decode what it can, and send it to the next level, and so forth. No you shouldn't. The i440fx should merge and arbitrate the mappings and then push *direct* links to the handling functions at the top level. Mapping changes don't happen often on modern hardware, and decoding is expensive. Decoding is not all that expensive. For non-PCI devices, the addresses are almost always fixed so it becomes a series of conditionals and function calls with a length of no more than 3 or 4. For PCI devices, any downstream devices are going to fall into specific regions that the bridge registers. Even in the pathological case of a bus populated with 32 multi-function devices each having 6 bars, it's still a non-overlapping list of ranges. There's nothing that prevents you from storing a sorted version of the list such that you can binary search to the proper dispatch device. Binary searching a list of 1500 entries is quite fast. In practice, you have no more than 10-20 PCI devices with each device having 2-3 bars. A simple linear search is not going to have a noticeable overhead. Incidentally, you can have special handling functions which are in reality references to kernel handlers, shortcutting userspace entirely for critical ports/mmio ranges. The cost here is the trip from the guest to userspace and back. If you want to short cut in the kernel, you have to do that *before* returning to userspace. In that case, how userspace models I/O flow doesn't matter. The reason flow matters is that PCI controllers alter I/O. Most PCI devices use little endian for device registers and some big endian oriented buses will automatically do endian conversion. Even without those types of controllers, if you use a native endian API, an MMIO dispatch API is going to do endian conversion to the target architecture. However, if you're expecting to return the data in little endian (as PCI registers are expected to usually be), you need to flip the endianness. In QEMU, we handle this by registering bars with a function pointer trampoline to do this. But this is with the special API. If you hook the mapping API, you'll probably get this wrong. Regards, Anthony Liguori OG. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Fri, Apr 8, 2011 at 9:14 AM, Pekka Enberg penb...@kernel.org wrote: Hi Anthony, On Fri, Apr 8, 2011 at 5:14 AM, Anthony Liguori anth...@codemonkey.ws wrote: If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Hey, feel free to help out! ;-) Yeah, helping would be great! -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi! Hi Anthony, On Fri, Apr 8, 2011 at 5:14 AM, Anthony Liguori anth...@codemonkey.ws wrote: If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Hey, feel free to help out! ;-) I don't agree that a working 2500 LOC program is 'repeating the same architectural mistakes' as QEMU. I hope you realize that we've gotten here with just three part-time hackers working from their proverbial basements. So what you call mistakes, we call features for the sake of simplicity. I also don't agree with this sentiment that unless we have SMP, migration, yadda yadda yadda, now, it's impossible to change that in the future. It ignores the fact that this is exactly how the Linux kernel evolved and the fact that we're aggressively trying to keep the code size as small and tidy as possible so that changing things is as easy as possible. Is it possible to find the code maintenance policy on a project site or somewhere? -- for both short run and long run. I may get some interest in using this tool for my debugging/testing/ self-educational porpuses, but cannot know what I can do/expect. Takuya For me, both QEMU and Native Linux KVM tool may be useful! :) But it is, probably I guess, for different porposes. I've looked at QEMU sources over the years and especially over the past year and I think you might be way too familiar with its inner workings to see how complex (even the core code) has become for someone who isn't familiar with it. I think it has to do with lots of indirection and code cleanliness issues (and I think that was the case even before KVM came into the picture). So I don't agree at all that taking QEMU as a starting point would make things any easier. (That is, unless someone intimately familiar with QEMU does it.) -- Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Takuya! On Fri, Apr 8, 2011 at 9:47 AM, Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp wrote: Is it possible to find the code maintenance policy on a project site or somewhere? -- for both short run and long run. I may get some interest in using this tool for my debugging/testing/ self-educational porpuses, but cannot know what I can do/expect. Heh, it's all pretty straight-forward. Fetch the sources from this tree: git clone git://github.com/penberg/linux-kvm.git Find something interesting to hack on and when you have something you want integrated send patches to penb...@kernel.org and CC this list. That's it! In the long run, we hope to live in the main kernel tree under tools/kvm and be part of the regular kernel release cycle. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
I may get some interest in using this tool for my debugging/testing/ self-educational porpuses, but cannot know what I can do/expect. Heh, it's all pretty straight-forward. Fetch the sources from this tree: git clone git://github.com/penberg/linux-kvm.git Find something interesting to hack on and when you have something you want integrated send patches to penb...@kernel.org and CC this list. That's it! Thank you for your answer! Actually, I knew how to get the code because I am checking most of emails on this ML. What I wanted to know is what I can do with this tool in the future. But OK, I got it that nothing is strictly determined at the moment about what features can/should be integrated. Takuya In the long run, we hope to live in the main kernel tree under tools/kvm and be part of the regular kernel release cycle. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 2011-04-08 07:14, Pekka Enberg wrote: Hi Anthony, On Fri, Apr 8, 2011 at 5:14 AM, Anthony Liguori anth...@codemonkey.ws wrote: If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Hey, feel free to help out! ;-) I don't agree that a working 2500 LOC program is 'repeating the same architectural mistakes' as QEMU. I hope you realize that we've gotten here with just three part-time hackers working from their proverbial basements. So what you call mistakes, we call features for the sake of simplicity. I also don't agree with this sentiment that unless we have SMP, migration, yadda yadda yadda, now, it's impossible to change that in the future. It ignores the fact that this is exactly how the Linux kernel evolved and the fact that we're aggressively trying to keep the code size as small and tidy as possible so that changing things is as easy as possible. I agree that it's easy to change 2kSomething LOC for this. But if you now wait too long designing in essential features like SMP, a scalable execution model, and - very important - portability (*), it can get fairly painful to fix such architectural deficits later on. How long did it take for Linux to overcome the BKL? QEMU is in the same unfortunate position. Jan (*) I would consider Anthony's idea to drop anything !=x86 a mistake given where KVM is moving to, today on PPC, tomorrow likely on ARM - just to name two examples. -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Jan, On Fri, 2011-04-08 at 09:39 +0200, Jan Kiszka wrote: I agree that it's easy to change 2kSomething LOC for this. But if you now wait too long designing in essential features like SMP, a scalable execution model, and - very important - portability (*), it can get fairly painful to fix such architectural deficits later on. How long did it take for Linux to overcome the BKL? QEMU is in the same unfortunate position. Yup, and we're taking your feedback seriously (and are thankful for it!). We're hoping to look at SMP in the near future - help is appreciated! Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 2011-04-08 10:27, Pekka Enberg wrote: Hi Jan, On Fri, 2011-04-08 at 09:39 +0200, Jan Kiszka wrote: I agree that it's easy to change 2kSomething LOC for this. But if you now wait too long designing in essential features like SMP, a scalable execution model, and - very important - portability (*), it can get fairly painful to fix such architectural deficits later on. How long did it take for Linux to overcome the BKL? QEMU is in the same unfortunate position. Yup, and we're taking your feedback seriously (and are thankful for it!). We're hoping to look at SMP in the near future - help is appreciated! Honestly, I do not yet see a major advantage for us to invest here instead of / in addition to continuing to improve QEMU. We've spend quite some effort on the latter with IMO noteworthy results. Porting over qemu-kvm to upstream was and still is among those efforts. We (*) are almost done. :) Just one example: Despite QEMU's current deficits, I just have add a handful of (ad-hoc) patches to turn it into a (soft) real-time hypervisor, and that also for certain non-Linux guests. Your approach is yet man years of development and stabilization effort away from getting close to such a level. Don't want to discourage you or other contributors. I wish you that this approach can gather the critical mass and momentum to make it a real alternative, at least for a subset of use cases. We will surely keep an eye on it and re-assess its proscons as it progresses. Jan (*) the QEMU KVM community -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Fri, Apr 8, 2011 at 1:11 PM, Jan Kiszka jan.kis...@siemens.com wrote: On 2011-04-08 10:27, Pekka Enberg wrote: Hi Jan, On Fri, 2011-04-08 at 09:39 +0200, Jan Kiszka wrote: I agree that it's easy to change 2kSomething LOC for this. But if you now wait too long designing in essential features like SMP, a scalable execution model, and - very important - portability (*), it can get fairly painful to fix such architectural deficits later on. How long did it take for Linux to overcome the BKL? QEMU is in the same unfortunate position. Yup, and we're taking your feedback seriously (and are thankful for it!). We're hoping to look at SMP in the near future - help is appreciated! Honestly, I do not yet see a major advantage for us to invest here instead of / in addition to continuing to improve QEMU. We've spend quite some effort on the latter with IMO noteworthy results. Porting over qemu-kvm to upstream was and still is among those efforts. We (*) are almost done. :) Just one example: Despite QEMU's current deficits, I just have add a handful of (ad-hoc) patches to turn it into a (soft) real-time hypervisor, and that also for certain non-Linux guests. Your approach is yet man years of development and stabilization effort away from getting close to such a level. Don't want to discourage you or other contributors. I wish you that this approach can gather the critical mass and momentum to make it a real alternative, at least for a subset of use cases. We will surely keep an eye on it and re-assess its proscons as it progresses. Jan (*) the QEMU KVM community -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux It seems there is a misunderstanding. KVM-tool is quite far from been KVM replacement (if ever). And what we're doing -- extremely tiny/small HV which would help us to debug/test kernel code. So I personally think of two scenarios: 1) QEMU eventually get merged upstream and kvm-tool remains small and tiny example of how to do /dev/kvm ioctls with some positive (I hope) results. Or maybe kvm-tool gets dropped since nobody needs it, this is possible too of course. 2) kvm-tool silently sit in tools/kvm while qemu remains on separated repo. Both go own ways. Not a pleasant scenario but still possible. And we don't consider kvm-tool as a qemu competitor by any means. It simply different weight categories. And of course we're glad to get any feedback (and positive and especially negative). Pointing out that SMP might be such a problem made us scratching the head ;) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 2011-04-08 11:32, Cyrill Gorcunov wrote: It seems there is a misunderstanding. KVM-tool is quite far from been KVM replacement (if ever). And what we're doing -- extremely tiny/small HV which would help us to debug/test kernel code. I think your core team may have this vision, but my impression is that some people here think much further. Also note that even for guest debugging some fairly essential features are missing yet. The gdbstub is among them and the most prominent one. So I personally think of two scenarios: 1) QEMU eventually get merged upstream and kvm-tool remains small and tiny example of how to do /dev/kvm ioctls with some positive (I hope) results. Or maybe kvm-tool gets dropped since nobody needs it, this is possible too of course. 2) kvm-tool silently sit in tools/kvm while qemu remains on separated repo. Both go own ways. Not a pleasant scenario but still possible. For me the separate tree thing is not that important as long as KVM developers continue to hack on both sides (which most of us do). And we don't consider kvm-tool as a qemu competitor by any means. It simply different weight categories. Long-term, IMHO, kvm-tool either has to cover at least one use case qemu is not interested in or it has to be noticeably better in one domain. Just being a small demo that has to be maintained _in_addition_ to qemu /wrt KVM ABI changes will make it suffer quickly. Right now x86 has reached a rather calm period in this regard, but PPC e.g. is about to enter the same stormy times we had with x86 in the past years. We've been through duplicate userland maintenance phase with qemu-kvm vs. upstream qemu for far too long, and it was a real pain (it still is, but the last duplicate bits should disappear before qemu-0.15). And of course we're glad to get any feedback (and positive and especially negative). Pointing out that SMP might be such a problem made us scratching the head ;) One big advantage of qemu is that it can nicely reproduce tricky concurrency issues (not all but many) as it provides true SMP support. We've successfully used this several times for debugging weird kernel and driver issues in the past 4 years. So I personally have no use case for kvm-tool that qemu(-kvm) wouldn't already solve, generally in a more advance way. That may explain my skepticism. :) Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 08.04.2011, at 12:42, Jan Kiszka wrote: On 2011-04-08 11:32, Cyrill Gorcunov wrote: It seems there is a misunderstanding. KVM-tool is quite far from been KVM replacement (if ever). And what we're doing -- extremely tiny/small HV which would help us to debug/test kernel code. I think your core team may have this vision, but my impression is that some people here think much further. I tend to agree. The core team seems to write this as a helping aid of learning the platform and getting to know KVM. I really like that approach :). However, if it's meant to be a toy (and I don't mean this negatively in any way), it really should be declared as such. Calling it kvm for example would be a huge mistake in that case. Either way, I like the idea of having a second user space available for x86. Even if it just means that it verifies that the documentation is correct :). Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Friday, April 8, 2011, Alexander Graf ag...@suse.de wrote: On 08.04.2011, at 12:42, Jan Kiszka wrote: On 2011-04-08 11:32, Cyrill Gorcunov wrote: It seems there is a misunderstanding. KVM-tool is quite far from been KVM replacement (if ever). And what we're doing -- extremely tiny/small HV which would help us to debug/test kernel code. I think your core team may have this vision, but my impression is that some people here think much further. I tend to agree. The core team seems to write this as a helping aid of learning the platform and getting to know KVM. I really like that approach :). However, if it's meant to be a toy (and I don't mean this negatively in any way), it really should be declared as such. Calling it kvm for example would be a huge mistake in that case. Either way, I like the idea of having a second user space available for x86. Even if it just means that it verifies that the documentation is correct :). Alex If we manage to make kvm-tool mature i believe anyone will win in such case. The annonce stated clear the kvm-tool relation to qemu. Of course we have great planes tho :) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/08/2011 12:14 AM, Pekka Enberg wrote:
Hey, feel free to help out! ;-)
I don't agree that a working 2500 LOC program is 'repeating the same
architectural mistakes' as QEMU. I hope you realize that we've gotten
here with just three part-time hackers working from their proverbial
basements. So what you call mistakes, we call features for the sake of
simplicity.
And by all means, it's a good accomplishment.
But the mistakes I'm referring to aren't missing bits of code. It's
that the current code makes really bad assumptions.
An example is ioport_ops. This maps directly to
ioport_{read,write}_table in QEMU. Then you use ioport__register() to
register entries in this table similar register_ioport_{read,write}() in
QEMU.
The use of a struct is a small improvement but the fundamental design is
flawed because it models a view of hardware where all devices are
directly connected to the CPU. This is not how hardware works at all.
On the PC QEMU tries to emulate, a PIO operation flows from the CPU to
the i440fx. The i440fx will do the first level of decoding treating the
PCI host controller ports specially and then posting any I/Os in the PCI
port range to the PCI bus. If no device selects these ports, or the
ports fall into the non-PCI range, the I/O request is then posted to the
PIIX3.
The PIIX3 will handle a good chunk of the I/O requests (via it's Super
I/O chipset) and the remainder will be posted to the ISA bus. One or
more ISA devices may then react to these posted I/O operation.
Really, having a flat table doesn't make sense. You should just send
everything to an i440fx directly. Then the i440fx should decode what it
can, and send it to the next level, and so forth.
You can get 90% of the way to working device model without modelling
this type of flow, but you hit a wall pretty quickly as it's not unusual
for PCI controllers to manipulate I/O requests in some fashion
(particularly on non-x86 platforms). If you treat everything as
directly attached to the CPU, it's impossible to model this.
Likewise, the same flow is true in the opposite direction. You use
guest_flat_to_host() which assumes a linear mapping of guest memory to
host memory. We used to do that too in QEMU (phys_ram_base + X). It
took a long time to get rid of that assumption in QEMU.
There are multiple problems with this sort of assumption. The first is
that you treat all devices as being directly attached to the memory
controller. As with I/O instruction dispatch, this is not the case, and
there are many PCI controllers that will munge these accesses (think
IOMMU, for instance). The second is you assume that you're not doing
I/O to device memory, but this does happen in practice. The
cpu_physical_memory_rw() API is careful to support cases where you're
writing data to I/O memory.
The other big problem here is that if you have open access to guest
memory like this, you cannot easily track dirty information. Userspace
accesses to guest memory will not result in KVM updating the guest dirty
bitmap. You can add another API to explicitly set dirty bits (and
that's exactly what we did a few years ago) but then you'll get
extremely subtle bugs in migration if you're missing a dirty update
somewhere. This is exactly how our API evolved in QEMU.
As I said earlier, there are very good reasons we do the things we do in
QEMU. We're a large code base and there's far too much of the code base
that noone cares about enough but that users are happy with. It's far
too hard to make broad sweeping changes right now (although that's
something we're trying to improve).
But I'd strongly suggest taking some of the advise being offered here.
Don't ignore the hard problems to start out with because as the code
base grows, it'll become more difficult to fix those. That's not to say
that you need to implement migration tomorrow, but at least keep the
constraints in mind and make sure that you're designing interfaces that
let you do things like keep an updated dirty bitmap when you do memory
accesses in userspace.
I also don't agree with this sentiment that unless we have SMP,
migration, yadda yadda yadda, now, it's impossible to change that in
the future. It ignores the fact that this is exactly how the Linux
kernel evolved
Over the course of 20 years. By my count, we still have another decade
of refactoring before I can get on top of my ivory tower and call every
other project terrible.
and the fact that we're aggressively trying to keep the
code size as small and tidy as possible so that changing things is as
easy as possible.
I've looked at QEMU sources over the years and especially over the
past year and I think you might be way too familiar with its inner
workings to see how complex (even the core code) has become for
someone who isn't familiar with it.
I have no doubts about the complexity of QEMU. But the 'goo' factor is
not due to complexity,
Re: [ANNOUNCE] Native Linux KVM tool
On Fri, Apr 08, 2011 at 01:32:24PM +0400, Cyrill Gorcunov wrote: It seems there is a misunderstanding. KVM-tool is quite far from been KVM replacement (if ever). And what we're doing -- extremely tiny/small HV which would help us to debug/test kernel code. If that's true, then perhaps the command-line invocation shouldn't be named kvm? The collision on the name of executable that claims that it will replace the kvm shipped in qemu seems to make the claim quite clearly that it's going to replace qemu's kvm in short order? - Ted -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Thu, 7 Apr 2011 21:14:06 -0500 Anthony Liguori anth...@codemonkey.ws wrote: If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Supporting only a single architecture sounds like a significant architectural mistake... only x86 deserves clean code? -Scott -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Anthony,
On Fri, Apr 08, 2011 at 09:00:43AM -0500, Anthony Liguori wrote:
An example is ioport_ops. This maps directly to
ioport_{read,write}_table in QEMU. Then you use ioport__register() to
register entries in this table similar register_ioport_{read,write}() in
QEMU.
The use of a struct is a small improvement but the fundamental design is
flawed because it models a view of hardware where all devices are
directly connected to the CPU. This is not how hardware works at all.
Not sure if I've the whole picture on this but I see no answer to your
email and I found your remark above the most interesting. This is
because I thought the whole point of a native kvm tool was to go all
the paravirt way to provide max performance and maybe also depend on
vhost as much as possible.
I mean if we have to care to emulate hardware _again_ and end up
replicating qemu (with the only exception of TCG) I don't see an need
of an alternative userland, let's not understimate how qemu is already
mature and good to emulate real hardware. I thought the whole point
was to exactly avoid any complaint like this is not how the hardware
works and focus only to optimize for smp and max scalability and
ignore how a real hardware would actually work to get there faster
than qemu can.
I had no time to read/try it yet I'm just reading the thread here...
Thanks,
Andrea
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/08/2011 10:59 AM, Scott Wood wrote: On Thu, 7 Apr 2011 21:14:06 -0500 Anthony Liguorianth...@codemonkey.ws wrote: If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Supporting only a single architecture sounds like a significant architectural mistake... only x86 deserves clean code? No, you just have to start somewhere. Since x86 is probably the ugliest, I think it's the best place to start. Regards, Anthony Liguori -Scott -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/08/2011 02:20 PM, Andrea Arcangeli wrote:
Hi Anthony,
On Fri, Apr 08, 2011 at 09:00:43AM -0500, Anthony Liguori wrote:
An example is ioport_ops. This maps directly to
ioport_{read,write}_table in QEMU. Then you use ioport__register() to
register entries in this table similar register_ioport_{read,write}() in
QEMU.
The use of a struct is a small improvement but the fundamental design is
flawed because it models a view of hardware where all devices are
directly connected to the CPU. This is not how hardware works at all.
Not sure if I've the whole picture on this but I see no answer to your
email and I found your remark above the most interesting. This is
because I thought the whole point of a native kvm tool was to go all
the paravirt way to provide max performance and maybe also depend on
vhost as much as possible.
Yeah, if that's the goal, skip all the mini-BIOS junk and just rely on a
PV kernel in the guest.
I think a mini userspace that assumes that we can change the guest
kernel and avoids having a ton of complexity to do things like CMOS
emulation would be a really interesting thing to do.
Regards,
Anthony Liguori
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/06/2011 05:55 AM, Ingo Molnar wrote: Splitting up a project into several trees, often unnecessarily, is a self-inflicted wound really. There's certainly something to this but the bit that surprises me is the approaching being taken. Why not take perf and all the other tools, stick them in their own git repos, and use git submodules to track them in the main kernel source tree. It seems like a nicer way to separate git histories while still getting the benefits of a shared repository. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/06/2011 04:33 AM, Ingo Molnar wrote: * Avi Kivitya...@redhat.com wrote: Sure, any succcesful project becomes an ugly gooball. It's almost a compliment. I disagree strongly with that sentiment and there's several good counter examples: - the Git project is also highly successful and is kept very clean (and has a project size comparable to Qemu) - the Linux kernel is also very clean in all areas i care about and has most of its ugliness stuffed into drivers/staging/ (and has a project size more than an order of magnitude larger than Qemu). In fact i claim the exact opposite: certain types of projects can only grow beyond a certain size and stay healthy if they are *not* ugly gooballs. Examples: X11 and GCC - both were struggling for years to break through magic invisible barriers of growth and IMHO a lot of it had to do with the lack of code (and development model) cleanliness. So what makes Native Linux KVM tool so much cleaner? As far as I can tell, it's architecturally identical to QEMU. In fact, it's reminiscent of QEMU from about 5 years ago. It makes the same mistakes of having a linear I/O dispatch model, makes no attempt to enable a threaded execution model, ignores thing like migration and manageability. So no, your kind of cynical, defeatist sentiment about code quality is by no means true in my experience. Projects become ugly gooballs once maintainers stop caring enough. It think sweeping generalizations are always wrong :-) I struggle with a lot of things in QEMU. Compatibility is just a nightmare to maintain because so many of the previous interfaces and functionality were so poorly thought through. If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Too much effort in QEMU goes into working around previous mistakes. That doesn't mean that QEMU doesn't have a lot of useful bits in it and hasn't figured out a lot of good ways to do things. Regards, Anthony Liguori Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Anthony, On Fri, Apr 8, 2011 at 5:14 AM, Anthony Liguori anth...@codemonkey.ws wrote: If someone was going to seriously go about doing something like this, a better approach would be to start with QEMU and remove anything non-x86 and all of the UI/command line/management bits and start there. There's nothing more I'd like to see than a viable alternative to QEMU but ignoring any of the architectural mistakes in QEMU and repeating them in a new project isn't going to get there. Hey, feel free to help out! ;-) I don't agree that a working 2500 LOC program is 'repeating the same architectural mistakes' as QEMU. I hope you realize that we've gotten here with just three part-time hackers working from their proverbial basements. So what you call mistakes, we call features for the sake of simplicity. I also don't agree with this sentiment that unless we have SMP, migration, yadda yadda yadda, now, it's impossible to change that in the future. It ignores the fact that this is exactly how the Linux kernel evolved and the fact that we're aggressively trying to keep the code size as small and tidy as possible so that changing things is as easy as possible. I've looked at QEMU sources over the years and especially over the past year and I think you might be way too familiar with its inner workings to see how complex (even the core code) has become for someone who isn't familiar with it. I think it has to do with lots of indirection and code cleanliness issues (and I think that was the case even before KVM came into the picture). So I don't agree at all that taking QEMU as a starting point would make things any easier. (That is, unless someone intimately familiar with QEMU does it.) On Fri, Apr 8, 2011 at 5:14 AM, Anthony Liguori anth...@codemonkey.ws wrote: Too much effort in QEMU goes into working around previous mistakes. That doesn't mean that QEMU doesn't have a lot of useful bits in it and hasn't figured out a lot of good ways to do things. Completely agreed. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Anthony Liguori anth...@codemonkey.ws writes: On 04/03/2011 05:11 AM, Avi Kivity wrote: On 04/03/2011 12:59 PM, Pekka Enberg wrote: Hi Avi, On Sun, Apr 3, 2011 at 11:23 AM, Avi Kivitya...@redhat.com wrote: Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. What are the pain points with qemu at the moment? It's an ugly gooball. Because it solves a lot of very difficult problems. And the solutions emerged / evolved over a long time. Meanwhile, goals shifted. It wasn't designed as user space for KVM, it got shoehorned into that role (successfully). It has some solutions it should have left to other tools. For instance, it shouldn't be in the network configuration business. You could drop all of the TCG support and it'd still be an ugly gooball. Supporting lots of different emulated hardware devices, live migration, tons of different types of networking and image formats, etc., all adds up over time. It does. Still, a fresh start could lead to a less ugly gooball. SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. If it is to be more than a toy, then Windows (really generic guest) support, manageability, live migration, hotplug, etc. are all crucial. I concur that SMP is probably one of those features you need to start with if you're designing something from scratch. Certainly. Another one that doesn't like retrofitting is security. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Wed, Apr 06, 2011 at 10:59:45AM +0200, Markus Armbruster wrote: Anthony Liguori anth...@codemonkey.ws writes: On 04/03/2011 05:11 AM, Avi Kivity wrote: On 04/03/2011 12:59 PM, Pekka Enberg wrote: Hi Avi, On Sun, Apr 3, 2011 at 11:23 AM, Avi Kivitya...@redhat.com wrote: Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. What are the pain points with qemu at the moment? It's an ugly gooball. Because it solves a lot of very difficult problems. And the solutions emerged / evolved over a long time. Meanwhile, goals shifted. It wasn't designed as user space for KVM, it got shoehorned into that role (successfully). It has some solutions it should have left to other tools. For instance, it shouldn't be in the network configuration business. You could drop all of the TCG support and it'd still be an ugly gooball. Supporting lots of different emulated hardware devices, live migration, tons of different types of networking and image formats, etc., all adds up over time. It does. Still, a fresh start could lead to a less ugly gooball. SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. If it is to be more than a toy, then Windows (really generic guest) support, manageability, live migration, hotplug, etc. are all crucial. I concur that SMP is probably one of those features you need to start with if you're designing something from scratch. Certainly. Another one that doesn't like retrofitting is security. And migration :) -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
* Avi Kivity a...@redhat.com wrote: Sure, any succcesful project becomes an ugly gooball. It's almost a compliment. I disagree strongly with that sentiment and there's several good counter examples: - the Git project is also highly successful and is kept very clean (and has a project size comparable to Qemu) - the Linux kernel is also very clean in all areas i care about and has most of its ugliness stuffed into drivers/staging/ (and has a project size more than an order of magnitude larger than Qemu). In fact i claim the exact opposite: certain types of projects can only grow beyond a certain size and stay healthy if they are *not* ugly gooballs. Examples: X11 and GCC - both were struggling for years to break through magic invisible barriers of growth and IMHO a lot of it had to do with the lack of code (and development model) cleanliness. So no, your kind of cynical, defeatist sentiment about code quality is by no means true in my experience. Projects become ugly gooballs once maintainers stop caring enough. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Wed, Apr 06, 2011 at 11:33:33AM +0200, Ingo Molnar wrote: So no, your kind of cynical, defeatist sentiment about code quality is by no means true in my experience. Projects become ugly gooballs once maintainers stop caring enough. In case of Qemu it was other way around. Maintainers started caring to late. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
* Gleb Natapov g...@redhat.com wrote: On Wed, Apr 06, 2011 at 11:33:33AM +0200, Ingo Molnar wrote: So no, your kind of cynical, defeatist sentiment about code quality is by no means true in my experience. Projects become ugly gooballs once maintainers stop caring enough. In case of Qemu it was other way around. Maintainers started caring too late. Nah, i do not think it's ever too late to care. Example: arch/i386 - arch/x86_64/ was very messy for many, many years and we turned it around and can be proud of arch/x86/ today - but i guess i'm somewhat biased there ;-) In my experience it's entirely possible to turn a messy gooball into something you can be proud of - it's all reversible. Start small, with the core bits you care about most - then extend those concepts to other areas of the code base, gradually. There might be subsystems that will never turn around before becoming obsolete - that's not a big problem. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/06/2011 12:46 PM, Ingo Molnar wrote: * Gleb Natapovg...@redhat.com wrote: On Wed, Apr 06, 2011 at 11:33:33AM +0200, Ingo Molnar wrote: So no, your kind of cynical, defeatist sentiment about code quality is by no means true in my experience. Projects become ugly gooballs once maintainers stop caring enough. In case of Qemu it was other way around. Maintainers started caring too late. Nah, i do not think it's ever too late to care. Example: arch/i386 - arch/x86_64/ was very messy for many, many years and we turned it around and can be proud of arch/x86/ today - but i guess i'm somewhat biased there ;-) In my experience it's entirely possible to turn a messy gooball into something you can be proud of - it's all reversible. Start small, with the core bits you care about most - then extend those concepts to other areas of the code base, gradually. There might be subsystems that will never turn around before becoming obsolete - that's not a big problem. That is what we're trying to do with qemu. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Wed, Apr 06, 2011 at 11:46:12AM +0200, Ingo Molnar wrote: * Gleb Natapov g...@redhat.com wrote: On Wed, Apr 06, 2011 at 11:33:33AM +0200, Ingo Molnar wrote: So no, your kind of cynical, defeatist sentiment about code quality is by no means true in my experience. Projects become ugly gooballs once maintainers stop caring enough. In case of Qemu it was other way around. Maintainers started caring too late. Nah, i do not think it's ever too late to care. Example: arch/i386 - arch/x86_64/ was very messy for many, many years and we turned it around and can be proud of arch/x86/ today - but i guess i'm somewhat biased there ;-) In my experience it's entirely possible to turn a messy gooball into something you can be proud of - it's all reversible. Start small, with the core bits you care about most - then extend those concepts to other areas of the code base, gradually. There might be subsystems that will never turn around before becoming obsolete - that's not a big problem. I do not disagree, but then qemu has a chance because maintainers do care now, but not about all bits. And there should be willingness to drop bits nobody cares about and I do not see this yet. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Wed, Apr 06, 2011 at 11:33:33AM +0200, Ingo Molnar wrote: Examples: X11 and GCC - both were struggling for years to break through magic invisible barriers of growth and IMHO a lot of it had to do with the lack of code (and development model) cleanliness. A large part of what's killing X11 and qemu is the decomposition in multiple trees and the requirement that every version must work with every other version. For X11 you have: - the server - the protocol headers - the individual 2D drivers - libdrm - the kernel - mesa - the video decoding driver/libs For qemu you have: - qemu - qemu-kvm - the kernel - libvirt - seabios Any reaching change ends up hitting most of the trees, with all to coordination that means. And in any case you're supposed to handle any version of the other components. Virtualbox works in part because they provide everything, from the dhcp server to the kernel modules. The NVidia closed-source drivers works in part because they provide everything, from the glx interface to the kernel modules. And both of them refuse to start if everything is not in lockstep. Meanwhile the open source idealists are a way smaller number and expect to support every combination of everything with everything as long as it appeared one day in a release, no matter how buggy or badly designed it was. So less people, additional hurdles that experience has shown not to be a necessity (people cope with lockstep updates, vbox and nvidia prove it), and one wonders why the open solutions end up way inferior? OG. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
* Olivier Galibert galib...@pobox.com wrote: On Wed, Apr 06, 2011 at 11:33:33AM +0200, Ingo Molnar wrote: Examples: X11 and GCC - both were struggling for years to break through magic invisible barriers of growth and IMHO a lot of it had to do with the lack of code (and development model) cleanliness. A large part of what's killing X11 and qemu is the decomposition in multiple trees and the requirement that every version must work with every other version. For X11 you have: - the server - the protocol headers - the individual 2D drivers - libdrm - the kernel - mesa - the video decoding driver/libs For qemu you have: - qemu - qemu-kvm - the kernel - libvirt - seabios Any reaching change ends up hitting most of the trees, with all to coordination that means. And in any case you're supposed to handle any version of the other components. Splitting up a project into several trees, often unnecessarily, is a self-inflicted wound really. Smaller projects can hurt from that as well: a well-known example is oprofile. Pointing to the stupidity of overmodularization is one of my pet peeves, i consider it a development model cleanliness bug that needlessly exposes OSS projects to the negative effects of technical and social forks and complicates/shackles them. I flamed^W argued about it before, in the KVM / Qemu context as well. There are good examples of successful, highly integrated projects: - FreeBSD - it has achieved Linux-alike results with a fraction of the manpower - Android - on the desktop it has achieved much more than Linux, with a fraction of the manpower And that concept can be brought to its logical conclusion: i think it's only a matter of time until someone takes the Linux kernel, integrates klibc and a toolchain into it with some good initial userspace and goes wild with that concept, as a single, sane, 100% self-hosting and self-sufficient OSS project, tracking the release schedule of the Linux kernel. It might not happen on PC hardware (which is *way* too OSS-hostile), but it will eventually happen IMO. It's the eventual OSS killer feature and weirdly enough no-one has tried it yet. (Android comes close in a sense) Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
* Pekka Enberg penb...@kernel.org wrote: Well, this is bound to cause confusion as the tool is yet quite immature. Yes, that's really unfortunate. I don't care too much what we call the tool but I definitely agree with Ingo that 'kvm' is more discoverable to users. Any suggestions? Well, lets keep it 'kvm' - if it stays immature it doesnt matter much. If it improves it has the right and obvious tool name. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
* Anthony Liguori anth...@codemonkey.ws wrote: On 03/31/2011 12:30 PM, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! Neat! As something of a lesson of history, I'd suggest picking a more unique name while it's still a prototype :-) I disagree, i find it pretty handy and intuitive to run 'kvm ./disk.img' to boot KVM and this particular tool name has not been taken yet either. perf uses a similar concept: the kernel subsystem is generally called 'perf', and the (Linux specific) user-space tool is called 'perf' as well. It makes quite a bit of sense. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 03/31/2011 07:30 PM, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! So that's where you disappeared - I was following your old repository. The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 09:21 AM, Ingo Molnar wrote: * Anthony Liguorianth...@codemonkey.ws wrote: On 03/31/2011 12:30 PM, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! Neat! As something of a lesson of history, I'd suggest picking a more unique name while it's still a prototype :-) I disagree, i find it pretty handy and intuitive to run 'kvm ./disk.img' to boot KVM and this particular tool name has not been taken yet either. Some distributions install qemu-kvm as /usr/bin/kvm. perf uses a similar concept: the kernel subsystem is generally called 'perf', and the (Linux specific) user-space tool is called 'perf' as well. It makes quite a bit of sense. Well, this is bound to cause confusion as the tool is yet quite immature. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Anthony, On Sat, Apr 2, 2011 at 11:38 PM, Anthony Liguori anth...@codemonkey.ws wrote: The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. I see you do provide 16-bit entry points for Linux. Are you planning on paravirtualizing this within Linux to truly eliminate the BIOS dependency? No, we aren't planning that at the moment. We're trying to support out-of-the-box distro kernels when possible which is why we went for E820 emulation in the first place. The only hard requirement for bootung userspace is CONFIG_VIRTIO_BLK but otherwise kernel binaries should just work. Furthermore, as the BIOS glue is really really small, I'm not sure if we need to get rid of it completely. Do you have some scenario in mind where paravirt solution would help? Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi, On Sun, Apr 3, 2011 at 11:24 AM, Avi Kivity a...@redhat.com wrote: On 04/03/2011 09:21 AM, Ingo Molnar wrote: * Anthony Liguorianth...@codemonkey.ws wrote: On 03/31/2011 12:30 PM, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! Neat! As something of a lesson of history, I'd suggest picking a more unique name while it's still a prototype :-) I disagree, i find it pretty handy and intuitive to run 'kvm ./disk.img' to boot KVM and this particular tool name has not been taken yet either. Some distributions install qemu-kvm as /usr/bin/kvm. perf uses a similar concept: the kernel subsystem is generally called 'perf', and the (Linux specific) user-space tool is called 'perf' as well. It makes quite a bit of sense. Well, this is bound to cause confusion as the tool is yet quite immature. Yes, that's really unfortunate. I don't care too much what we call the tool but I definitely agree with Ingo that 'kvm' is more discoverable to users. Any suggestions? Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Thu, Mar 31, 2011 at 08:30:56PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. I've looked at how to add spice to this, the qxl device should be relatively easy to add as it's just another pci device and you already support the virtio block pci device. But to add the spice server library there needs to be some simple fd and timer (i.e. select/epoll) event loop, which I see is missing. Are you planning on adding something like that? It's great as a learning tool if you want to get your feet wet in virtualization land: it's only 5 KLOC of clean C code that can already boot a guest Linux image. Right now it can boot a Linux image and provide you output via a serial console, over the host terminal, i.e. you can use it to boot a guest Linux image in a terminal or over ssh and log into the guest without much guest or host side setup work needed. 1. To try out the tool, clone the git repository: git clone git://github.com/penberg/linux-kvm.git or alternatively, if you already have a kernel source tree: git checkout -b kvm/tool git pull git://github.com/penberg/linux-kvm.git 2. Compile the tool: cd tools/kvm make 3. Download a raw userspace image: wget http://wiki.qemu.org/download/linux-0.2.img.bz2 bunzip2 linux-0.2.img.bz2 4. Build a kernel with CONFIG_VIRTIO_BLK=y and CONFIG_SERIAL_8250_CONSOLE=y configuration options. Note: also make sure you have CONFIG_EXT2_FS or CONFIG_EXT4_FS if you use the above image. 5. And finally, launch the hypervisor: ./kvm --image=linux-0.2.img --kernel=../../arch/x86/boot/bzImage The tool has been written by Pekka Enberg, Cyrill Gorcunov, and Asias He. Special thanks to Avi Kivity for his help on KVM internals and Ingo Molnar for all-around support and encouragement! See the following thread for original discussion for motivation of this project: http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620 Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 12:53 PM, Pekka Enberg wrote: Hi, On Sun, Apr 3, 2011 at 11:24 AM, Avi Kivity a...@redhat.com wrote: On 04/03/2011 09:21 AM, Ingo Molnar wrote: * Anthony Liguorianth...@codemonkey.ws wrote: On 03/31/2011 12:30 PM, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! Neat! As something of a lesson of history, I'd suggest picking a more unique name while it's still a prototype :-) I disagree, i find it pretty handy and intuitive to run 'kvm ./disk.img' to boot KVM and this particular tool name has not been taken yet either. Some distributions install qemu-kvm as /usr/bin/kvm. perf uses a similar concept: the kernel subsystem is generally called 'perf', and the (Linux specific) user-space tool is called 'perf' as well. It makes quite a bit of sense. Well, this is bound to cause confusion as the tool is yet quite immature. Yes, that's really unfortunate. I don't care too much what we call the tool but I definitely agree with Ingo that 'kvm' is more discoverable to users. Any suggestions? Pekka Well, I personally do not care much either. If there a fear we might interfere with some distribution probably we could re-name it nkvm (ie from Native KVM). I've googled it and found there is no such name used yet. Hm? -- Cyrill -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 11:51 AM, Pekka Enberg wrote: Hi Anthony, On Sat, Apr 2, 2011 at 11:38 PM, Anthony Liguorianth...@codemonkey.ws wrote: The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. I see you do provide 16-bit entry points for Linux. Are you planning on paravirtualizing this within Linux to truly eliminate the BIOS dependency? No, we aren't planning that at the moment. We're trying to support out-of-the-box distro kernels when possible which is why we went for E820 emulation in the first place. The only hard requirement for bootung userspace is CONFIG_VIRTIO_BLK but otherwise kernel binaries should just work. Furthermore, as the BIOS glue is really really small, I'm not sure if we need to get rid of it completely. Do you have some scenario in mind where paravirt solution would help? It would be a easier to support the bios than implement everything it provides in a different way. SMP support, cpu hotplug, device hotplug, NUMA, and probably other features all rely on the bios. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Avi, On Sun, Apr 3, 2011 at 11:23 AM, Avi Kivity a...@redhat.com wrote: Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. What are the pain points with qemu at the moment? SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. I'd also love to have GPU support for X and friends. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Sun, 2011-04-03 at 12:01 +0300, Alon Levy wrote: On Thu, Mar 31, 2011 at 08:30:56PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. I've looked at how to add spice to this, the qxl device should be relatively easy to add as it's just another pci device and you already support the virtio block pci device. But to add the spice server library there needs to be some simple fd and timer (i.e. select/epoll) event loop, which I see is missing. Are you planning on adding something like that? We have kvm__start_timer() in tools/kvm/kvm.c. Can you use that as a base for qxl? Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 12:59 PM, Pekka Enberg wrote: Hi Avi, On Sun, Apr 3, 2011 at 11:23 AM, Avi Kivitya...@redhat.com wrote: Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. What are the pain points with qemu at the moment? It's an ugly gooball. SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. If it is to be more than a toy, then Windows (really generic guest) support, manageability, live migration, hotplug, etc. are all crucial. I'd also love to have GPU support for X and friends. Should be easy to get by integrating spice (but that gives you a remote-optimized display, not local). -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Sun, Apr 03, 2011 at 01:01:38PM +0300, Pekka Enberg wrote: On Sun, 2011-04-03 at 12:01 +0300, Alon Levy wrote: On Thu, Mar 31, 2011 at 08:30:56PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. I've looked at how to add spice to this, the qxl device should be relatively easy to add as it's just another pci device and you already support the virtio block pci device. But to add the spice server library there needs to be some simple fd and timer (i.e. select/epoll) event loop, which I see is missing. Are you planning on adding something like that? We have kvm__start_timer() in tools/kvm/kvm.c. Can you use that as a base for qxl? Haven't looked at it close enough yet, but if I can set an arbitrary time ahead wakeup using it then it's good enough for timers. For waking up as a result of a socket being ready for read or write I need something else. Maybe I'm missing something and you already have something like that. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Avi, On Sun, Apr 3, 2011 at 1:11 PM, Avi Kivity a...@redhat.com wrote: SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. If it is to be more than a toy, then Windows (really generic guest) support, manageability, live migration, hotplug, etc. are all crucial. It's definitely not a toy, it's my main virtualization tool of choice for kernel development! ;-) The features you mention are crucial for servers but not for desktop. I personally don't have much need for managing and live-migrating Windows guests but if someone is interested in working on that, we're happy to take patches! Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 01:17 PM, Pekka Enberg wrote: Hi Avi, On Sun, Apr 3, 2011 at 1:11 PM, Avi Kivitya...@redhat.com wrote: SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. If it is to be more than a toy, then Windows (really generic guest) support, manageability, live migration, hotplug, etc. are all crucial. It's definitely not a toy, it's my main virtualization tool of choice for kernel development! ;-) The features you mention are crucial for servers but not for desktop. I personally don't have much need for managing and live-migrating Windows guests but if someone is interested in working on that, we're happy to take patches! Well, I'd say you do need generic guest support for desktop (but not the other stuff I mentioned). Are you planning to add a real GUI? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Sun, Apr 03, 2011 at 11:53:34AM +0300, Pekka Enberg wrote: Yes, that's really unfortunate. I don't care too much what we call the tool but I definitely agree with Ingo that 'kvm' is more discoverable Indeed. It's great for finding info on keyboard-video-mouse switches. :) -- A search of his car uncovered pornography, a homemade sex aid, women's stockings and a Jack Russell terrier. - http://www.dailytelegraph.com.au/news/wacky/indeed/story-e6frev20-118083480 -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
Hi Avi, On Sun, Apr 3, 2011 at 1:22 PM, Avi Kivity a...@redhat.com wrote: It's definitely not a toy, it's my main virtualization tool of choice for kernel development! ;-) The features you mention are crucial for servers but not for desktop. I personally don't have much need for managing and live-migrating Windows guests but if someone is interested in working on that, we're happy to take patches! Well, I'd say you do need generic guest support for desktop (but not the other stuff I mentioned). Are you planning to add a real GUI? Yes, we'll hopefully add GUI at some point. Generic guest support would be cool but like I said, I personally don't need it because I'm mostly interested in running Linux guests. Windows is totally uninteresting and I'm hoping we can get away with virtio drivers for other FOSS kernels. Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 05:11 AM, Avi Kivity wrote: On 04/03/2011 12:59 PM, Pekka Enberg wrote: Hi Avi, On Sun, Apr 3, 2011 at 11:23 AM, Avi Kivitya...@redhat.com wrote: Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. What are the pain points with qemu at the moment? It's an ugly gooball. Because it solves a lot of very difficult problems. You could drop all of the TCG support and it'd still be an ugly gooball. Supporting lots of different emulated hardware devices, live migration, tons of different types of networking and image formats, etc., all adds up over time. SMP, networking, and simpler guest to host communication from shell are most interesting missing features for me. If it is to be more than a toy, then Windows (really generic guest) support, manageability, live migration, hotplug, etc. are all crucial. I concur that SMP is probably one of those features you need to start with if you're designing something from scratch. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 04/03/2011 04:09 PM, Anthony Liguori wrote: On 04/03/2011 05:11 AM, Avi Kivity wrote: On 04/03/2011 12:59 PM, Pekka Enberg wrote: Hi Avi, On Sun, Apr 3, 2011 at 11:23 AM, Avi Kivitya...@redhat.com wrote: Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. Mind posting a roadmap? I would put smp support near the top. This sort of thing has to be designed in, otherwise you wind up with a big lock like qemu. What are the pain points with qemu at the moment? It's an ugly gooball. Because it solves a lot of very difficult problems. You could drop all of the TCG support and it'd still be an ugly gooball. Supporting lots of different emulated hardware devices, live migration, tons of different types of networking and image formats, etc., all adds up over time. Sure, any succcesful project becomes an ugly gooball. It's almost a compliment. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 03/31/2011 12:30 PM, Pekka Enberg wrote: Hi all, We’re proud to announce the native Linux KVM tool! Neat! As something of a lesson of history, I'd suggest picking a more unique name while it's still a prototype :-) The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. I see you do provide 16-bit entry points for Linux. Are you planning on paravirtualizing this within Linux to truly eliminate the BIOS dependency? Regards, Anthony Liguori Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. It's great as a learning tool if you want to get your feet wet in virtualization land: it's only 5 KLOC of clean C code that can already boot a guest Linux image. Right now it can boot a Linux image and provide you output via a serial console, over the host terminal, i.e. you can use it to boot a guest Linux image in a terminal or over ssh and log into the guest without much guest or host side setup work needed. 1. To try out the tool, clone the git repository: git clone git://github.com/penberg/linux-kvm.git or alternatively, if you already have a kernel source tree: git checkout -b kvm/tool git pull git://github.com/penberg/linux-kvm.git 2. Compile the tool: cd tools/kvm make 3. Download a raw userspace image: wget http://wiki.qemu.org/download/linux-0.2.img.bz2; bunzip2 linux-0.2.img.bz2 4. Build a kernel with CONFIG_VIRTIO_BLK=y and CONFIG_SERIAL_8250_CONSOLE=y configuration options. Note: also make sure you have CONFIG_EXT2_FS or CONFIG_EXT4_FS if you use the above image. 5. And finally, launch the hypervisor: ./kvm --image=linux-0.2.img --kernel=../../arch/x86/boot/bzImage The tool has been written by Pekka Enberg, Cyrill Gorcunov, and Asias He. Special thanks to Avi Kivity for his help on KVM internals and Ingo Molnar for all-around support and encouragement! See the following thread for original discussion for motivation of this project: http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620 Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On 31.03.2011 23:47, Alexander Graf wrote: Did you take a look at kuli? Right. If you want to take kuli as a start, and GPL is acceptable for you ;-). If kuli seems right, I suggest to go ahead and make an upstream git repository for it. I'd be happy to keep contributing the s390 parts to whoever feels responsible for maintaining the tree. cheers, Carsten -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Friday, April 1, 2011, Carsten Otte carst...@de.ibm.com wrote: On 31.03.2011 23:47, Alexander Graf wrote: Did you take a look at kuli? Right. If you want to take kuli as a start, and GPL is acceptable for you ;-). If kuli seems right, I suggest to go ahead and make an upstream git repository for it. I'd be happy to keep contributing the s390 parts to whoever feels responsible for maintaining the tree. cheers, Carsten Cool! I personally didnt hear of kuli (thanks for the hint) but will take a look. Carsten, Pekka is the primary maintainer, so if youre interested to maintain s390 part, i think we are glad to accept it ;) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] Native Linux KVM tool
On Thu, Mar 31, 2011 at 08:30:56PM +0300, Pekka Enberg wrote: The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) That line looks very familiar... I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. ;) -- Steve -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[ANNOUNCE] Native Linux KVM tool
Hi all, We’re proud to announce the native Linux KVM tool! The goal of this tool is to provide a clean, from-scratch, lightweight KVM host tool implementation that can boot Linux guest images (just a hobby, won't be big and professional like QEMU) with no BIOS dependencies and with only the minimal amount of legacy device emulation. Note that this is a development prototype for the time being: there's no networking support and no graphics support, amongst other missing essentials. It's great as a learning tool if you want to get your feet wet in virtualization land: it's only 5 KLOC of clean C code that can already boot a guest Linux image. Right now it can boot a Linux image and provide you output via a serial console, over the host terminal, i.e. you can use it to boot a guest Linux image in a terminal or over ssh and log into the guest without much guest or host side setup work needed. 1. To try out the tool, clone the git repository: git clone git://github.com/penberg/linux-kvm.git or alternatively, if you already have a kernel source tree: git checkout -b kvm/tool git pull git://github.com/penberg/linux-kvm.git 2. Compile the tool: cd tools/kvm make 3. Download a raw userspace image: wget http://wiki.qemu.org/download/linux-0.2.img.bz2 bunzip2 linux-0.2.img.bz2 4. Build a kernel with CONFIG_VIRTIO_BLK=y and CONFIG_SERIAL_8250_CONSOLE=y configuration options. Note: also make sure you have CONFIG_EXT2_FS or CONFIG_EXT4_FS if you use the above image. 5. And finally, launch the hypervisor: ./kvm --image=linux-0.2.img --kernel=../../arch/x86/boot/bzImage The tool has been written by Pekka Enberg, Cyrill Gorcunov, and Asias He. Special thanks to Avi Kivity for his help on KVM internals and Ingo Molnar for all-around support and encouragement! See the following thread for original discussion for motivation of this project: http://thread.gmane.org/gmane.linux.kernel/962051/focus=962620 Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
