Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-18 Thread Roland Dreier
It seems that we've come up with two reasonable cases where it makes sense to use these notifiers for InfiniBand/RDMA: First, the ability to safely to DMA to/from userspace memory with the memory regions mlock()ed but the pages not pinned. In this case the notifiers here would seem to suit us

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-17 Thread Robin Holt
On Sun, Feb 17, 2008 at 04:01:20AM +0100, Andrea Arcangeli wrote: On Sat, Feb 16, 2008 at 11:21:07AM -0800, Christoph Lameter wrote: On Fri, 15 Feb 2008, Andrew Morton wrote: What is the status of getting infiniband to use this facility? Well we are talking about this it seems. It

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-16 Thread Avi Kivity
Andrew Morton wrote: How important is this feature to KVM? Very. kvm pins pages that are referenced by the guest; a 64-bit guest will easily pin its entire memory with the kernel map. So this is critical for guest swapping to actually work. Other nice features like page migration are

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-16 Thread Andrew Morton
On Sat, 16 Feb 2008 10:45:50 +0200 Avi Kivity [EMAIL PROTECTED] wrote: Andrew Morton wrote: How important is this feature to KVM? Very. kvm pins pages that are referenced by the guest; hm. Why does it do that? a 64-bit guest will easily pin its entire memory with the kernel map.

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-16 Thread Avi Kivity
Andrew Morton wrote: Very. kvm pins pages that are referenced by the guest; hm. Why does it do that? It was deemed best not to allow the guest to write to a page that has been swapped out and assigned to an unrelated host process. One way to view the kvm shadow page tables

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-16 Thread Christoph Lameter
On Fri, 15 Feb 2008, Andrew Morton wrote: What is the status of getting infiniband to use this facility? Well we are talking about this it seems. How important is this feature to KVM? Andrea can answer this. To xpmem? Without this feature we are stuck with page pinning by increasing

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-16 Thread Doug Maxey
On Fri, 15 Feb 2008 19:37:19 PST, Andrew Morton wrote: Which other potential clients have been identified and how important it it to those? The powerpc ehea utilizes its own mmu. Not sure about the importance to the driver. (But will investigate :) ++doug

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-15 Thread Andrew Morton
On Thu, 14 Feb 2008 22:49:00 -0800 Christoph Lameter [EMAIL PROTECTED] wrote: MMU notifiers are used for hardware and software that establishes external references to pages managed by the Linux kernel. These are page table entriews or tlb entries or something else that allows hardware (such

[kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-14 Thread Christoph Lameter
MMU notifiers are used for hardware and software that establishes external references to pages managed by the Linux kernel. These are page table entriews or tlb entries or something else that allows hardware (such as DMA engines, scatter gather devices, networking, sharing of address spaces across

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-02-05 Thread Christoph Lameter
On Tue, 5 Feb 2008, Andy Whitcroft wrote: + if (unlikely(!hlist_empty(mm-mmu_notifier.head))) { + rcu_read_lock(); + hlist_for_each_entry_safe_rcu(mn, n, t, + mm-mmu_notifier.head, hlist) { + if (mn-ops-release)

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Jack Steiner
On Wed, Jan 30, 2008 at 04:37:49PM +0100, Andrea Arcangeli wrote: On Tue, Jan 29, 2008 at 06:29:10PM -0800, Christoph Lameter wrote: +void mmu_notifier_release(struct mm_struct *mm) +{ + struct mmu_notifier *mn; + struct hlist_node *n, *t; + + if

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Andrea Arcangeli
On Tue, Jan 29, 2008 at 06:29:10PM -0800, Christoph Lameter wrote: +void mmu_notifier_release(struct mm_struct *mm) +{ + struct mmu_notifier *mn; + struct hlist_node *n, *t; + + if (unlikely(!hlist_empty(mm-mmu_notifier.head))) { + rcu_read_lock(); +

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Andrea Arcangeli
On Wed, Jan 30, 2008 at 09:53:06AM -0600, Jack Steiner wrote: That will also resolve the problem we discussed yesterday. I want to unregister my mmu_notifier when a GRU segment is unmapped. This would not necessarily be at task termination. My proof that there is something wrong in the smp

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Robin Holt
Back to one of Andrea's points from a couple days ago, I think we still have a problem with the PageExternalRmap page flag. If I had two drivers with external rmap implementations, there is no way I can think of for a simple flag to coordinate a single page being exported and maintained by the

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Christoph Lameter
Ok. So I added the following patch: --- include/linux/mmu_notifier.h |1 + mm/mmu_notifier.c| 12 2 files changed, 13 insertions(+) Index: linux-2.6/include/linux/mmu_notifier.h === ---

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Christoph Lameter
On Wed, 30 Jan 2008, Jack Steiner wrote: Moving to a different lock solves the problem. Well it gets us back to the issue why we removed the lock. As Robin said before: If its global then we can have a huge number of tasks contending for the lock on startup of a process with a large number of

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Christoph Lameter
How about just taking the mmap_sem writelock in release? We have only a single caller of mmu_notifier_release() in mm/mmap.c and we know that we are not holding mmap_sem at that point. So just acquire it when needed? Index: linux-2.6/mm/mmu_notifier.c

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Christoph Lameter
On Thu, 31 Jan 2008, Andrea Arcangeli wrote: I think Andrea's original concept of the lock in the mmu_notifier_head structure was the best. I agree with him that it should be a spinlock instead of the rw_lock. BTW, I don't see the scalability concern with huge number of tasks: the lock

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-30 Thread Christoph Lameter
On Thu, 31 Jan 2008, Andrea Arcangeli wrote: H.. exit_mmap is only called when the last reference is removed against the mm right? So no tasks are running anymore. No pages are left. Do we need to serialize at all for mmu_notifier_release? KVM sure doesn't need any locking there.

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-29 Thread Andrea Arcangeli
On Mon, Jan 28, 2008 at 12:28:41PM -0800, Christoph Lameter wrote: +struct mmu_notifier_head { + struct hlist_head head; +}; + struct mm_struct { struct vm_area_struct * mmap; /* list of VMAs */ struct rb_root mm_rb; @@ -219,6 +223,8 @@ struct mm_struct {

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-29 Thread Robin Holt
I am going to seperate my comments into individual replies to help reduce the chance they are lost. +void mmu_notifier_release(struct mm_struct *mm) ... + hlist_for_each_entry_safe_rcu(mn, n, t, + mm-mmu_notifier.head, hlist) { +

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-29 Thread Avi Kivity
Christoph Lameter wrote: On Tue, 29 Jan 2008, Andrea Arcangeli wrote: + struct mmu_notifier_head mmu_notifier; /* MMU notifier list */ }; Not sure why you prefer to waste ram when MMU_NOTIFIER=n, this is a regression (a minor one though). Andrew does not like #ifdefs

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-29 Thread Christoph Lameter
On Tue, 29 Jan 2008, Andrea Arcangeli wrote: + struct mmu_notifier_head mmu_notifier; /* MMU notifier list */ }; Not sure why you prefer to waste ram when MMU_NOTIFIER=n, this is a regression (a minor one though). Andrew does not like #ifdefs and it makes it possible to verify calling

[kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-29 Thread Christoph Lameter
Core code for mmu notifiers. Signed-off-by: Christoph Lameter [EMAIL PROTECTED] Signed-off-by: Andrea Arcangeli [EMAIL PROTECTED] --- include/linux/list.h | 14 ++ include/linux/mm_types.h |6 + include/linux/mmu_notifier.h | 210 +++

[kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-28 Thread Christoph Lameter
Core code for mmu notifiers. Signed-off-by: Christoph Lameter [EMAIL PROTECTED] Signed-off-by: Andrea Arcangeli [EMAIL PROTECTED] --- include/linux/list.h | 14 ++ include/linux/mm_types.h |6 + include/linux/mmu_notifier.h | 210 +++

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-28 Thread Christoph Lameter
mmu core: Need to use hlist_del Wrong type of list del in mmu_notifier_release() Signed-off-by: Christoph Lameter [EMAIL PROTECTED] --- mm/mmu_notifier.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: linux-2.6/mm/mmu_notifier.c

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-28 Thread Robin Holt
+void mmu_notifier_release(struct mm_struct *mm) ... + hlist_for_each_entry_safe_rcu(mn, n, t, + mm-mmu_notifier.head, hlist) { + if (mn-ops-release) + mn-ops-release(mn, mm); +

Re: [kvm-devel] [patch 1/6] mmu_notifier: Core code

2008-01-28 Thread Christoph Lameter
On Mon, 28 Jan 2008, Robin Holt wrote: USE_AFTER_FREE!!! I made this same comment as well as other relavent comments last week. Must have slipped somehow. Patch needs to be applied after the rcu fix. Please repeat the other relevant comments if they are still relevant I thought I had