Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Andrea Arcangeli
On Tue, Apr 01, 2008 at 01:55:32PM -0700, Christoph Lameter wrote: +/* Perform a callback */ +int __emm_notify(struct mm_struct *mm, enum emm_operation op, + unsigned long start, unsigned long end) +{ + struct emm_notifier *e = rcu_dereference(mm)-emm_notifier; + int x;

Re: [kvm-devel] Ubuntu Gutsy host / XP guest / -smp 2

2008-04-02 Thread Avi Kivity
David Abrahams wrote: With the title combination, the guest takes nearly 100% of my real CPU time and still only sees one CPU. Is this a known problem, and does it have a known solution? Can you send the output of 'kvm_stat -1'? Also, what does the guest think it is doing (in task

Re: [kvm-devel] [PATCH] Fix endianness for virtio-blk config space

2008-04-02 Thread Avi Kivity
Anthony Liguori wrote: The virtio config space is little endian. Make sure that in virtio-blk we store the values in little endian format. Applied, thanks. -- error compiling committee.c: too many arguments to function

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 07:32:35AM +0300, Avi Kivity wrote: It ought to work. gfn_to_hfn() (old gfn_to_page) will still need to take a refcount if possible. This reminds me, that mmu notifiers we could implement gfn_to_hfn only with follow_page and skip the refcounting on the struct page.

[kvm-devel] True meaning of showing wealth

2008-04-02 Thread Jerardo Fullgraf
Fantastic offer on luxuries The clock was invented to be re-invented to have a spot on your wrist. http://www.poretiage.com/ - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Avi Kivity
Andrea Arcangeli wrote: On Wed, Apr 02, 2008 at 07:32:35AM +0300, Avi Kivity wrote: It ought to work. gfn_to_hfn() (old gfn_to_page) will still need to take a refcount if possible. This reminds me, that mmu notifiers we could implement gfn_to_hfn only with follow_page and skip

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 12:50:50PM +0300, Avi Kivity wrote: Isn't it faster though? We don't need to pull in the cacheline containing the struct page anymore. Exactly, not only that, get_user_pages is likely a bit slower that we need for just kvm pte lookup. GRU uses follow_page directly

Re: [kvm-devel] [RFC/PATCH 05/15 v3] kvm-s390: s390 arch backend for the kvm kernel module

2008-04-02 Thread Christian Borntraeger
Am Montag, 31. März 2008 schrieb Arnd Bergmann: Hello Arnd, thanks for the review. On Tuesday 25 March 2008, Carsten Otte wrote: + +static inline void __user *__guestaddr_to_user(struct kvm_vcpu *vcpu, + u64 guestaddr) +{ + u64 prefix =

Re: [kvm-devel] [PATCH 2/2] Move in-kernel PIT device to separate file

2008-04-02 Thread Avi Kivity
Anthony Liguori wrote: This patch is mostly code motion to move the newly refactored in-kernel PIT device to a separate file. Applied both, thanks. -- error compiling committee.c: too many arguments to function -

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Robin Holt
On Wed, Apr 02, 2008 at 08:49:52AM +0200, Andrea Arcangeli wrote: Most other patches will apply cleanly on top of my coming mmu notifiers #v10 that I hope will go in -mm. For #v10 the only two left open issues to discuss are: Does your v10 allow sleeping inside the callbacks? Thanks, Robin

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Avi Kivity
Andrea Arcangeli wrote: On Wed, Apr 02, 2008 at 12:50:50PM +0300, Avi Kivity wrote: Isn't it faster though? We don't need to pull in the cacheline containing the struct page anymore. Exactly, not only that, get_user_pages is likely a bit slower that we need for just kvm pte

Re: [kvm-devel] KVM hangs with windows XP guest - help with debugging?

2008-04-02 Thread Jonathan Underwood
On 01/04/2008, Avi Kivity [EMAIL PROTECTED] wrote: Please run 'kvm_stat -1' (found in the tarball) while the guest is hanging and report the output. # /home/jgu/Desktop/kvm_stat -1 efer_reload 8757563 129 exits 17039786 362 halt_exits

Re: [kvm-devel] [PATCH] KVM: add kvm_get_kvm and kvm_put_kvm functions

2008-04-02 Thread Avi Kivity
Izik Eidus wrote: From ebb9fe4765f1572314d2249e29a7ef4d0de07273 Mon Sep 17 00:00:00 2001 From: Izik Eidus [EMAIL PROTECTED] Date: Sun, 30 Mar 2008 15:48:35 +0300 Subject: [PATCH] KVM: add kvm_get_kvm and kvm_put_kvm functions, the main purpose of adding this functions is the abilaty to

Re: [kvm-devel] KVM hangs with windows XP guest - help with debugging?

2008-04-02 Thread Jonathan Underwood
On 02/04/2008, Jonathan Underwood [EMAIL PROTECTED] wrote: On 01/04/2008, Avi Kivity [EMAIL PROTECTED] wrote: Please run 'kvm_stat -1' (found in the tarball) while the guest is hanging and report the output. # /home/jgu/Desktop/kvm_stat -1 efer_reload 8757563 129

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 02:16:41PM +0300, Avi Kivity wrote: Ugh, there's still mark_page_accessed() and SetPageDirty(). btw, like PG_dirty is only set if the spte is writeable, mark_page_accessed should only run if the accessed bit is set in the spte. It doesn't matter now as nobody could

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 01:50:19PM +0200, Andrea Arcangeli wrote: if (pfn_valid(pfn)) { page = pfn_to_page(pfn); if (!PageReserved(page)) { BUG_ON(page_count(page) != 1); if (is_writeable_pte(*spte))

[kvm-devel] How guest virtual address is translated to physical address?

2008-04-02 Thread Guillaume Thouvenin
Hello, I have a question about how guest page table and shadow page table work together and more precisely, about how host is involved when guest access a page that it's already in the page table. The guest maintains its page table to translate guest virtual address to guest physical address.

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Robin Holt
I must have missed v10. Could you repost so I can build xpmem against it to see how it operates? To help reduce confusion, you should probably comandeer the patches from Christoph's set which you think are needed to make it sleep. Thanks, Robin On Wed, Apr 02, 2008 at 01:16:51PM +0200, Andrea

Re: [kvm-devel] [PATCH 1/1] direct mmio for passthrough - kernel part

2008-04-02 Thread Anthony Liguori
Avi Kivity wrote: Andrea Arcangeli wrote: On Wed, Apr 02, 2008 at 12:50:50PM +0300, Avi Kivity wrote: Isn't it faster though? We don't need to pull in the cacheline containing the struct page anymore. Exactly, not only that, get_user_pages is likely a bit slower that we need for

Re: [kvm-devel] How guest virtual address is translated to physical address?

2008-04-02 Thread Anthony Liguori
Guillaume Thouvenin wrote: Hello, I have a question about how guest page table and shadow page table work together and more precisely, about how host is involved when guest access a page that it's already in the page table. The guest maintains its page table to translate guest virtual

Re: [kvm-devel] Ubuntu Gutsy host / XP guest / -smp 2

2008-04-02 Thread David Abrahams
on Wed Apr 02 2008, Avi Kivity avi-AT-qumranet.com wrote: David Abrahams wrote: With the title combination, the guest takes nearly 100% of my real CPU time and still only sees one CPU. Is this a known problem, and does it have a known solution? Can you send the output of 'kvm_stat

Re: [kvm-devel] [patch 5/9] Convert anon_vma lock to rw_sem and refcount

2008-04-02 Thread Andrea Arcangeli
On Tue, Apr 01, 2008 at 01:55:36PM -0700, Christoph Lameter wrote: This results in f.e. the Aim9 brk performance test to got down by 10-15%. I guess it's more likely because of overscheduling for small crtitical sections, did you counted the total number of context switches? I guess there will

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: There are much bigger issues besides the rcu safety in this patch, proper aging of the secondary mmu through access bits set by hardware is unfixable with this model (you would need to do age |= e-callback), which is the proof of why this isn't

Re: [kvm-devel] [patch 5/9] Convert anon_vma lock to rw_sem and refcount

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: On Tue, Apr 01, 2008 at 01:55:36PM -0700, Christoph Lameter wrote: This results in f.e. the Aim9 brk performance test to got down by 10-15%. I guess it's more likely because of overscheduling for small crtitical sections, did you counted the

[kvm-devel] EMM: Fixup return value handling of emm_notify()

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Christoph Lameter wrote: Here f.e. We can add a special emm_age() function that iterates differently and does the | for you. Well maybe not really necessary. How about this fix? Its likely a problem to stop callbacks if one callback returned an error. Subject: EMM:

[kvm-devel] [PATCH] Don't assume struct page for x86 MMU

2008-04-02 Thread Anthony Liguori
This patch introduces a gfn_to_pfn() function and corresponding functions like kvm_release_pfn_dirty(). Using these new functions, we can modify the x86 MMU to no longer assume that it can always get a struct page for any given gfn. We don't want to eliminate gfn_to_page() entirely because a

Re: [kvm-devel] EMM: Fixup return value handling of emm_notify()

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 12:03:50PM -0700, Christoph Lameter wrote: + /* + * Callback may return a positive value to indicate a count + * or a negative error code. We keep the first error code + * but

[kvm-devel] EMM: Require single threadedness for registration.

2008-04-02 Thread Christoph Lameter
Here is a patch to require single threaded execution during emm_register. This also allows an easy implementation of an unregister function and gets rid of the races that Andrea worried about. The approach here is similar to what was used in selinux for security context changes (see

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 10:59:50AM -0700, Christoph Lameter wrote: Did I see #v10? Could you start a new subject when you post please? Do not respond to some old message otherwise the threading will be wrong. I wasn't clear enough, #v10 was in the works... I was thinking about the last two

Re: [kvm-devel] [patch 5/9] Convert anon_vma lock to rw_sem and refcount

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 11:15:26AM -0700, Christoph Lameter wrote: On Wed, 2 Apr 2008, Andrea Arcangeli wrote: On Tue, Apr 01, 2008 at 01:55:36PM -0700, Christoph Lameter wrote: This results in f.e. the Aim9 brk performance test to got down by 10-15%. I guess it's more likely

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: Hmmm... Okay that is one solution that would just require a BUG_ON in the registration methods. Perhaps you didn't notice that this solution can't work if you call range_begin/end not in the current context and try_to_unmap_cluster does

[kvm-devel] [PATCH 2 of 8] Moves all mmu notifier methods outside the PT lock (first and not last

2008-04-02 Thread Andrea Arcangeli
# HG changeset patch # User Andrea Arcangeli [EMAIL PROTECTED] # Date 1207159010 -7200 # Node ID fe00cb9deeb31467396370c835cb808f4b85209a # Parent a406c0cc686d0ca94a4d890d661cdfa48cfba09f Moves all mmu notifier methods outside the PT lock (first and not last step to make them sleep capable).

[kvm-devel] [PATCH 4 of 8] The conversion to a rwsem allows callbacks during rmap traversal

2008-04-02 Thread Andrea Arcangeli
# HG changeset patch # User Andrea Arcangeli [EMAIL PROTECTED] # Date 1207159011 -7200 # Node ID 3c3787c496cab1fc590ba3f97e7904bdfaab5375 # Parent d880c227ddf345f5d577839d36d150c37b653bfd The conversion to a rwsem allows callbacks during rmap traversal for files in a non atomic context. A rw

Re: [kvm-devel] [patch 5/9] Convert anon_vma lock to rw_sem and refcount

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: paging), hence the slowdown. What you benchmarked is the write side, which is also the fast path when the system is heavily CPU bound. I've to say aim is a great benchmark to test this regression. I am a bit surprised that brk performance is that

[kvm-devel] [PATCH 0 of 8] mmu notifiers #v10

2008-04-02 Thread Andrea Arcangeli
Hello, this is the mmu notifier #v10. Patches 1 and 2 are the only difference between this and EMM V2. The rest is the same as with Christoph's patches. I think maximum priority should be given in merging patch 1 and 2 into -mm and ASAP in mainline. Patches from 3 to 8 can go in -mm for testing

Re: [kvm-devel] EMM: Fixup return value handling of emm_notify()

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: but anyway it's silly to be hardwired to such an interface that worst of all requires switch statements instead of proper pointer to functions and a fixed set of parameters and retval semantics for all methods. The EMM API with a single callback is

[kvm-devel] [PATCH 5 of 8] We no longer abort unmapping in unmap vmas because we can reschedule while

2008-04-02 Thread Andrea Arcangeli
# HG changeset patch # User Andrea Arcangeli [EMAIL PROTECTED] # Date 1207159055 -7200 # Node ID 316e5b1e4bf388ef0198c91b3067ed1e4171d7f6 # Parent 3c3787c496cab1fc590ba3f97e7904bdfaab5375 We no longer abort unmapping in unmap vmas because we can reschedule while unmapping since we are holding a

[kvm-devel] [PATCH 8 of 8] This patch adds a lock ordering rule to avoid a potential deadlock when

2008-04-02 Thread Andrea Arcangeli
# HG changeset patch # User Andrea Arcangeli [EMAIL PROTECTED] # Date 1207159059 -7200 # Node ID f3f119118b0abd9c4624263ef388dc7230d937fe # Parent 31fc23193bd039cc595fba1ca149a9715f7d0fb2 This patch adds a lock ordering rule to avoid a potential deadlock when multiple mmap_sems need to be locked.

[kvm-devel] [PATCH 3 of 8] Move the tlb flushing into free_pgtables. The conversion of the locks

2008-04-02 Thread Andrea Arcangeli
# HG changeset patch # User Andrea Arcangeli [EMAIL PROTECTED] # Date 1207159010 -7200 # Node ID d880c227ddf345f5d577839d36d150c37b653bfd # Parent fe00cb9deeb31467396370c835cb808f4b85209a Move the tlb flushing into free_pgtables. The conversion of the locks taken for reverse map scanning would

Re: [kvm-devel] [PATCH 2 of 8] Moves all mmu notifier methods outside the PT lock (first and not last

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: diff --git a/mm/memory.c b/mm/memory.c --- a/mm/memory.c +++ b/mm/memory.c @@ -1626,9 +1626,10 @@ */ page_table = pte_offset_map_lock(mm, pmd, address,

Re: [kvm-devel] EMM: Require single threadedness for registration.

2008-04-02 Thread Christoph Lameter
On Thu, 3 Apr 2008, Andrea Arcangeli wrote: That would work for #v10 if I remove the invalidate_range_start from try_to_unmap_cluster, it can't work for EMM because you've emm_invalidate_start firing anywhere outside the context of the current task (even regular rmap code, not just nonlinear

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 02:54:52PM -0700, Christoph Lameter wrote: On Wed, 2 Apr 2008, Andrea Arcangeli wrote: Hmmm... Okay that is one solution that would just require a BUG_ON in the registration methods. Perhaps you didn't notice that this solution can't work if you call

Re: [kvm-devel] [patch 5/9] Convert anon_vma lock to rw_sem and refcount

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 02:56:25PM -0700, Christoph Lameter wrote: I am a bit surprised that brk performance is that important. There may be I think it's not brk but fork that is being slowed down, did you oprofile? AIM forks a lot... The write side fast path generating the overscheduling I

Re: [kvm-devel] EMM: Require single threadedness for registration.

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 03:06:19PM -0700, Christoph Lameter wrote: On Thu, 3 Apr 2008, Andrea Arcangeli wrote: That would work for #v10 if I remove the invalidate_range_start from try_to_unmap_cluster, it can't work for EMM because you've emm_invalidate_start firing anywhere outside the

Re: [kvm-devel] [PATCH 1 of 8] Core of mmu notifiers

2008-04-02 Thread Christoph Lameter
On Wed, 2 Apr 2008, Andrea Arcangeli wrote: + void (*invalidate_page)(struct mmu_notifier *mn, + struct mm_struct *mm, + unsigned long address); + + void (*invalidate_range_start)(struct mmu_notifier *mn, +

Re: [kvm-devel] [patch 1/9] EMM Notifier: The notifier calls

2008-04-02 Thread Christoph Lameter
On Thu, 3 Apr 2008, Andrea Arcangeli wrote: I said try_to_unmap_cluster, not get_user_pages. CPU0CPU1 try_to_unmap_cluster: emm_invalidate_start in EMM (or mmu_notifier_invalidate_range_start in #v10) walking the list by hand in EMM (or with

[kvm-devel] [patch 0/3] separate thread for IO handling V3

2008-04-02 Thread Marcelo Tosatti
This version fixes the vmdk problems found by the regression testing. Dor, regarding the option to disable the IO thread, it would require duplicating most of the changed code. For now I believe its better to get the patch into a state where its considered stable enough for inclusion. Please

[kvm-devel] [patch 1/3] QEMU/KVM: separate thread for IO handling

2008-04-02 Thread Marcelo Tosatti
Move IO processing from vcpu0 to a dedicated thread. This removes load on vcpu0 by allowing better cache locality and also improves latency. Signed-off-by: Marcelo Tosatti [EMAIL PROTECTED] Index: kvm-userspace.io/qemu/qemu-kvm.c

[kvm-devel] [patch 3/3] QEMU/KVM: notify IO thread of pending bhs

2008-04-02 Thread Marcelo Tosatti
This fixes the slow vmdk issue found in the regression tests. Signed-off-by: Marcelo Tosatti [EMAIL PROTECTED] Index: kvm-userspace.io/qemu/qemu-kvm.c === --- kvm-userspace.io.orig/qemu/qemu-kvm.c +++

[kvm-devel] [patch 2/3] QEMU/KVM: add function to handle signals

2008-04-02 Thread Marcelo Tosatti
SIGUSR1 has no handler, and the SIGUSR2 one does nothing useful anymore (thats also true for SIGIO on tap fd, which runs host_alarm_handler unnecessarily). Signed-off-by: Marcelo Tosatti [EMAIL PROTECTED] Index: kvm-userspace.io/qemu/qemu-kvm.c

Re: [kvm-devel] [02/17][PATCH] Implement smp_call_function_mask for ia64 - V8

2008-04-02 Thread Luck, Tony
Have you looked at Jens Axboe's patches to make all this stuff a lot more arch-common? Nope, do you have a pointer? Check your favourite archive for this Subject line: Generic smp_call_function(), improvements, and smp_call_function_single() -Tony

Re: [kvm-devel] [PATCH 1 of 8] Core of mmu notifiers

2008-04-02 Thread Andrea Arcangeli
On Wed, Apr 02, 2008 at 03:34:01PM -0700, Christoph Lameter wrote: Still two methods ... Yes, the invalidate_page is called with the core VM holding a reference on the page _after_ the tlb flush. The invalidate_end is called after the page has been freed already and after the tlb flush. They've

Re: [kvm-devel] [PATCH 1 of 8] Core of mmu notifiers

2008-04-02 Thread Christoph Lameter
Thinking about this adventurous locking some more: I think you are misunderstanding what a seqlock is. It is *not* a spinlock. The critical read section with the reading of a version before and after allows you access to a certain version of memory how it is or was some time ago (caching

Re: [kvm-devel] Ubuntu Gutsy host / XP guest / -smp 2

2008-04-02 Thread Liu, Eric E
David Abrahams wrote: on Wed Apr 02 2008, Avi Kivity avi-AT-qumranet.com wrote: David Abrahams wrote: With the title combination, the guest takes nearly 100% of my real CPU time and still only sees one CPU. Is this a known problem, and does it have a known solution? Can you send

[kvm-devel] EMM: disable other notifiers before register and unregister

2008-04-02 Thread Christoph Lameter
Ok lets forget about the single theaded thing to solve the registration races. As Andrea pointed out this still has ssues with other subscribed subsystems (and also try_to_unmap). We could do something like what stop_machine_run does: First disable all running subsystems before registering a

Re: [kvm-devel] Ubuntu Gutsy host / XP guest / -smp 2

2008-04-02 Thread Jim Paris
David Abrahams wrote: What HAL do you see in device manager? In the guest? I don't see anything that obviously appears to be HAL in device manager. Could you be more specific? Hi, He's referring to the Windows computer type -- the entry under computer in device manager. See

[kvm-devel] [Patch][00/18] kvm-ia64 for kernel V10

2008-04-02 Thread Zhang, Xiantao
Compared with V9, just fixed indentation issues in patch 12. I put it the patchset in git://git.kernel.org/pub/scm/linux/kernel/git/xiantao/kvm-ia64.git kvm-ia64-mc10. Please help to review. Specially, the first two patches (TR Management patch and smp_call_function_mask patch) needs Tony's