Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-13 Thread KAMEZAWA Hiroyuki
On Thu, 13 Nov 2008 12:38:07 +0200 Izik Eidus [EMAIL PROTECTED] wrote: If KSM pages are on radix-tree, it will be accounted automatically. Now, we have Unevictable LRU and mlocked() pages are smartly isolated into its own LRU. So, just doing - inode's radix-tree - make all pages

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-13 Thread Izik Eidus
ציטוט KAMEZAWA Hiroyuki: Thank you for answers. On Wed, 12 Nov 2008 13:11:12 +0200 Izik Eidus [EMAIL PROTECTED] wrote: Avi Kivity wrote: KAMEZAWA Hiroyuki wrote: Can I make a question ? (I'm working for memory cgroup.) Now, we do charge to anonymous page when - charge(+1)

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Izik Eidus
Avi Kivity wrote: KAMEZAWA Hiroyuki wrote: Can I make a question ? (I'm working for memory cgroup.) Now, we do charge to anonymous page when - charge(+1) when it's mapped firstly (mapcount 0-1) - uncharge(-1) it's fully unmapped (mapcount 1-0) vir page_remove_rmap(). My quesion is -

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Avi Kivity
KAMEZAWA Hiroyuki wrote: Can I make a question ? (I'm working for memory cgroup.) Now, we do charge to anonymous page when - charge(+1) when it's mapped firstly (mapcount 0-1) - uncharge(-1) it's fully unmapped (mapcount 1-0) vir page_remove_rmap(). My quesion is - PageKSM pages are not

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 09:10:45PM -0600, Christoph Lameter wrote: get_user_pages() cannot get to it since the pagetables have already been modified. If get_user_pages runs then the fault handling will occur which will block the thread until migration is complete. migrate.c does nothing for

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Lee Schermerhorn
On Wed, 2008-11-12 at 18:32 +0100, Andrea Arcangeli wrote: On Tue, Nov 11, 2008 at 09:10:45PM -0600, Christoph Lameter wrote: get_user_pages() cannot get to it since the pagetables have already been modified. If get_user_pages runs then the fault handling will occur which will block the

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Christoph Lameter
On Wed, 12 Nov 2008, Andrea Arcangeli wrote: On Tue, Nov 11, 2008 at 09:10:45PM -0600, Christoph Lameter wrote: get_user_pages() cannot get to it since the pagetables have already been modified. If get_user_pages runs then the fault handling will occur which will block the thread until

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Christoph Lameter
On Wed, 12 Nov 2008, Lee Schermerhorn wrote: Might want/need to check for migration entry in do_swap_page() and loop back to migration_entry_wait() call when the changed pte is detected rather than returning an error to the caller. Does that sound reasonable? The reference count freezing

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Andrea Arcangeli
On Wed, Nov 12, 2008 at 05:09:03PM -0500, Lee Schermerhorn wrote: Maybe not so wild, given the complexity of these interactions... Perhaps Christoph's right it's just wild ideas, but see below. You both seem to agree the first theory of the tree_lock is bogus as it's lockless for

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Andrea Arcangeli
On Thu, Nov 13, 2008 at 03:00:59AM +0100, Andrea Arcangeli wrote: CPU0 migrate.cCPU1 filemap.c --- -- find_get_page radix_tree_lookup_slot returns the oldpage

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread Nick Piggin
On Thursday 13 November 2008 13:31, Andrea Arcangeli wrote: On Thu, Nov 13, 2008 at 03:00:59AM +0100, Andrea Arcangeli wrote: CPU0 migrate.c CPU1 filemap.c --- -- find_get_page

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-12 Thread KAMEZAWA Hiroyuki
Thank you for answers. On Wed, 12 Nov 2008 13:11:12 +0200 Izik Eidus [EMAIL PROTECTED] wrote: Avi Kivity wrote: KAMEZAWA Hiroyuki wrote: Can I make a question ? (I'm working for memory cgroup.) Now, we do charge to anonymous page when - charge(+1) when it's mapped firstly (mapcount

[PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
From: Izik Eidus [EMAIL PROTECTED] this function is needed in cases you want to change the userspace virtual mapping into diffrent physical page, KSM need this for merging the identical pages. this function is working by removing the oldpage from the rmap and calling put_page on it, and by

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrew Morton
On Tue, 11 Nov 2008 15:21:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: From: Izik Eidus [EMAIL PROTECTED] this function is needed in cases you want to change the userspace virtual mapping into diffrent physical page, Not sure that I understand that description. We want to replace a live

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 15:21:39 +0200 Izik Eidus [EMAIL PROTECTED] wrote: From: Izik Eidus [EMAIL PROTECTED] this function is needed in cases you want to change the userspace virtual mapping into diffrent physical page, Not sure that I understand that description.

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 11:45:55AM -0800, Andrew Morton wrote: page migration already kinda does that. Is there common ground? btw, page_migration likely is buggy w.r.t. o_direct too (and now unfixable with gup_fast until the 2.4 brlock is added around it or similar) if it does the same thing

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Christoph Lameter
On Tue, 11 Nov 2008, Izik Eidus wrote: yes but it replace it with kernel allocated page. page migration already kinda does that. Is there common ground? page migration as far as i saw cant migrate anonymous page into kernel page. if you want we can change page_migration to do that, but

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Christoph Lameter wrote: page migration as far as i saw cant migrate anonymous page into kernel page. if you want we can change page_migration to do that, but i thought you will rather have ksm changes separate. What do you mean by kernel page? The kernel can allocate a page and then

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 03:21:45PM -0600, Christoph Lameter wrote: What do you mean by kernel page? The kernel can allocate a page and then point a user space pte to it. That is how page migration works. Just to make an example, remove_migration_pte adds the page back to rmap layer. We can't do

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Christoph Lameter
On Tue, 11 Nov 2008, Andrea Arcangeli wrote: btw, page_migration likely is buggy w.r.t. o_direct too (and now unfixable with gup_fast until the 2.4 brlock is added around it or similar) if it does the same thing but without any page_mapcount vs page_count check. Details please?

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Christoph Lameter wrote: Currently page migration assumes that the page will continue to be part of the existing file or anon vma. exactly, and ksm really need it to get out of the existing anon vma! What you want sounds like assigning a swap pte to an anonymous page? That way a anon

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Avi Kivity
Christoph Lameter wrote: page migration requires the page to be on the LRU. That could be changed if you have a different means of isolating a page from its page tables. Isn't rmap the means of isolating a page from its page tables? I guess I'm misunderstanding something. -- I have a

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Christoph Lameter
On Tue, 11 Nov 2008, Izik Eidus wrote: What do you mean by kernel page? The kernel can allocate a page and then point a user space pte to it. That is how page migration works. i mean filebacked page (!AnonPage()) ok. ksm need the pte inside the vma to point from anonymous page into

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Christoph Lameter
On Tue, 11 Nov 2008, Avi Kivity wrote: Christoph Lameter wrote: page migration requires the page to be on the LRU. That could be changed if you have a different means of isolating a page from its page tables. Isn't rmap the means of isolating a page from its page tables? I guess I'm

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Izik Eidus
Christoph Lameter wrote: On Tue, 11 Nov 2008, Avi Kivity wrote: Christoph Lameter wrote: page migration requires the page to be on the LRU. That could be changed if you have a different means of isolating a page from its page tables. Isn't rmap the means of isolating a page

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 03:26:57PM -0600, Christoph Lameter wrote: On Tue, 11 Nov 2008, Andrea Arcangeli wrote: btw, page_migration likely is buggy w.r.t. o_direct too (and now unfixable with gup_fast until the 2.4 brlock is added around it or similar) if it does the same thing but

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 03:31:18PM -0600, Christoph Lameter wrote: ksm need the pte inside the vma to point from anonymous page into filebacked page can migrate.c do it without changes? So change anonymous to filebacked page? Currently page migration assumes that the page will continue

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Avi Kivity
Christoph Lameter wrote: On Tue, 11 Nov 2008, Avi Kivity wrote: Christoph Lameter wrote: page migration requires the page to be on the LRU. That could be changed if you have a different means of isolating a page from its page tables. Isn't rmap the means of isolating a page

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 04:30:22PM -0600, Christoph Lameter wrote: On Tue, 11 Nov 2008, Andrea Arcangeli wrote: this page_count check done with only the tree_lock won't prevent a task to start O_DIRECT after page_count has been read in the above line. If a thread starts O_DIRECT on the

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Wed, Nov 12, 2008 at 12:17:22AM +0100, Andrea Arcangeli wrote: We don't have to check the page_count vs mapcount later in replace_page because we know if anybody started an O_DIRECT read from disk, it would have triggered a cow, and the pte_same check that we have to do for other reasons

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Christoph Lameter
On Wed, 12 Nov 2008, Andrea Arcangeli wrote: O_DIRECT does not take a refcount on the page in order to prevent this? It definitely does, it's also the only thing it does. Then page migration will not occur because there is an unresolved reference. The whole point is that O_DIRECT can start

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread KAMEZAWA Hiroyuki
On Tue, 11 Nov 2008 23:24:21 +0100 Andrea Arcangeli [EMAIL PROTECTED] wrote: On Tue, Nov 11, 2008 at 03:31:18PM -0600, Christoph Lameter wrote: ksm need the pte inside the vma to point from anonymous page into filebacked page can migrate.c do it without changes? So change

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Andrea Arcangeli
On Tue, Nov 11, 2008 at 06:27:09PM -0600, Christoph Lameter wrote: Then page migration will not occur because there is an unresolved reference. So are you checking if there's an unresolved reference only in the very place I just quoted in the previous email? If answer is yes: what should

Re: [PATCH 2/4] Add replace_page(), change the mapping of pte from one page into another

2008-11-11 Thread Christoph Lameter
On Wed, 12 Nov 2008, Andrea Arcangeli wrote: So are you checking if there's an unresolved reference only in the very place I just quoted in the previous email? If answer is yes: what should prevent get_user_pages from running in parallel from another thread? get_user_pages will trigger a