[PATCH] Revert mm: introduce VM_POPULATE flag to better deal with racy userspace programs

2013-03-28 Thread Hugh Dickins
as there isn't any strong argument to mandate a new vm_flag. Signed-off-by: Michel Lespinasse wal...@google.com Signed-off-by: Hugh Dickins hu...@google.com --- Andrew has taken this patch into his tree, but remarked that he is away from keyboard until the end of next week: though not a serious matter, I

Re: [RFC] mm: remove swapcache page early

2013-03-29 Thread Hugh Dickins
On Fri, 29 Mar 2013, Minchan Kim wrote: On Thu, Mar 28, 2013 at 11:19:12AM -0700, Dan Magenheimer wrote: From: Minchan Kim [mailto:minc...@kernel.org] On Wed, Mar 27, 2013 at 03:24:00PM -0700, Dan Magenheimer wrote: From: Hugh Dickins [mailto:hu...@google.com] Subject: Re: [RFC

Re: [RFC] mm: remove swapcache page early

2013-04-01 Thread Hugh Dickins
On Tue, 2 Apr 2013, Minchan Kim wrote: On Fri, Mar 29, 2013 at 01:01:14PM -0700, Hugh Dickins wrote: On Fri, 29 Mar 2013, Minchan Kim wrote: On Thu, Mar 28, 2013 at 11:19:12AM -0700, Dan Magenheimer wrote: I wonder if something like this would have a similar result for zram

Re: [PATCH] THP: Use explicit memory barrier

2013-04-02 Thread Hugh Dickins
On Tue, 2 Apr 2013, Minchan Kim wrote: On Mon, Apr 01, 2013 at 04:35:38PM -0700, David Rientjes wrote: On Mon, 1 Apr 2013, Minchan Kim wrote: __do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set

RE: [PATCHv2, RFC 20/30] ramfs: enable transparent huge page cache

2013-04-02 Thread Hugh Dickins
On Tue, 2 Apr 2013, Kirill A. Shutemov wrote: Kirill A. Shutemov wrote: From: Kirill A. Shutemov kirill.shute...@linux.intel.com ramfs is the most simple fs from page cache point of view. Let's start transparent huge page cache enabling here. For now we allocate only non-movable

Re: remap kernel static memory to user space

2013-03-23 Thread Hugh Dickins
On Wed, 20 Mar 2013, Eduardo Cruz wrote: Hello. I'm trying to remap some kernel static memory to user space using remap_pfn_range. For that, I wrote a module that initializes a device, which later I use mknod, etc. When I allocate the memory with kmalloc, everything works: (where

Re: linux-v3.9-rc3: BUG: Bad page map in process trinity-child6 pte:002f9045 pmd:29e421e1

2013-03-26 Thread Hugh Dickins
On Tue, 26 Mar 2013, Toralf Foerster wrote: On 03/25/2013 11:53 PM, Andrew Morton wrote: On Fri, 22 Mar 2013 18:28:36 +0100 Toralf Foerster toralf.foers...@gmx.de wrote: Using trinity I often trigger under a user mode linux image with host kernel 3.8.4 and guest kernel

Revert VM_POPULATE?

2013-03-26 Thread Hugh Dickins
Michel, I propose that we revert 3.9-rc1's VM_POPULATE flag - 186930500985 mm: introduce VM_POPULATE flag to better deal with racy userspace programs. Konstantin's 3.7 cleanup of VM_flags has left several bits below 32 free, but sooner or later someone will want to come through again and free

Re: [RFC] mm: remove swapcache page early

2013-03-27 Thread Hugh Dickins
or SWP_SOLIDSTATE easily(ie, actually, zram is already done) but I have no idea to use it for frontswap. Any idea? Other optimize point is we remove it unconditionally when we found it's exclusive when swap in happen. It could help frontswap family, too. What do you think about it? Cc: Hugh

RE: [RFC] mm: remove swapcache page early

2013-03-27 Thread Hugh Dickins
On Wed, 27 Mar 2013, Dan Magenheimer wrote: From: Hugh Dickins [mailto:hu...@google.com] Subject: Re: [RFC] mm: remove swapcache page early On Wed, 27 Mar 2013, Minchan Kim wrote: Swap subsystem does lazy swap slot free with expecting the page would be swapped out again so we

Re: [PATCH] mm: page_alloc: remove branch operation in free_pages_prepare()

2013-03-07 Thread Hugh Dickins
On Fri, 8 Mar 2013, Joonsoo Kim wrote: On Thu, Mar 07, 2013 at 10:54:15AM -0800, Hugh Dickins wrote: On Thu, 7 Mar 2013, Joonsoo Kim wrote: When we found that the flag has a bit of PAGE_FLAGS_CHECK_AT_PREP, we reset the flag. If we always reset the flag, we can reduce one branch

Re: mm/ksm.c:492: error: implicit declaration of function ‘pfn_to_nid’

2013-03-04 Thread Hugh Dickins
away without it so far, so fix the build in mm/ksm.c. Reported-by: Geert Uytterhoeven ge...@linux-m68k.org Signed-off-by: Hugh Dickins hu...@google.com --- mm/ksm.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- 3.9-rc1/mm/ksm.c2013-03-03 19:35:40.120006587 -0800 +++ linux/mm

[PATCH] ksm: fix m68k build: only NUMA needs pfn_to_nid

2013-03-04 Thread Hugh Dickins
arch/mips/include/asm/mmzone.h for example). Or perhaps it is only expected when CONFIG_NUMA=y: too much of a maze, and m68k got away without it so far, so fix the build in mm/ksm.c. Reported-by: Geert Uytterhoeven ge...@linux-m68k.org Signed-off-by: Hugh Dickins hu...@google.com --- Sorry, the non

Re: [PATCH 2/2] tmpfs: fix mempolicy object leaks

2013-03-05 Thread Hugh Dickins
On Mon, 4 Mar 2013, Will Huck wrote: Could you explain me why shmem has more relationship with mempolicy? It seems that there are many codes in shmem handle mempolicy, but other components in mm subsystem just have little. NUMA mempolicy is mostly handled in mm/mempolicy.c, which services

Re: [PATCH 2/7] ksm: treat unstable nid like in stable tree

2013-03-05 Thread Hugh Dickins
On Wed, 6 Mar 2013, Will Huck wrote: How ksm treat a ksm forked page? IIUC, it's not merged in ksm stable tree. It will just be ignore? No, it's there in the stable tree, as it was before it got forked. And when ksmd comes around to find the new mm, it will allocate an rmap_item for that

Re: [PATCH 2/7] ksm: treat unstable nid like in stable tree

2013-03-05 Thread Hugh Dickins
On Wed, 6 Mar 2013, Ric Mason wrote: [ I've deleted the context because that was about the unstable tree, and here you have moved to asking about a case in the stable tree. ] For the case of a ksm page is migrated to a different NUMA node and migrate its stable node to the right tree and

Re: change of behavior for madvise in 3.9-rc1

2013-03-07 Thread Hugh Dickins
On Thu, 7 Mar 2013, Shaohua Li wrote: On Wed, Mar 06, 2013 at 11:05:04PM -0500, CAI Qian wrote: Bisecting indicated that this commit, 1998cc048901109a29924380b8e91bc049b32951 mm: make madvise(MADV_WILLNEED) support swap file prefetch Caused an LTP test failure, http://goo.gl/1FVPy

Re: [PATCH] mm: page_alloc: remove branch operation in free_pages_prepare()

2013-03-07 Thread Hugh Dickins
On Thu, 7 Mar 2013, Joonsoo Kim wrote: When we found that the flag has a bit of PAGE_FLAGS_CHECK_AT_PREP, we reset the flag. If we always reset the flag, we can reduce one branch operation. So remove it. Cc: Hugh Dickins hu...@google.com Signed-off-by: Joonsoo Kim iamjoonsoo@lge.com I

Re: [PATCH] tmpfs: support SEEK_DATA and SEEK_HOLE (reprise)

2012-11-29 Thread Hugh Dickins
On Thu, 29 Nov 2012, Andrew Morton wrote: On Wed, 28 Nov 2012 17:22:03 -0800 (PST) Hugh Dickins hu...@google.com wrote: +/* + * llseek SEEK_DATA or SEEK_HOLE through the radix_tree. + */ +static pgoff_t shmem_seek_hole_data(struct address_space *mapping

Re: [PATCH 5/5] mempolicy: fix a memory corruption by refcount imbalance in alloc_pages_vma()

2012-12-04 Thread Hugh Dickins
shmem_alloc_page() went on expecting alloc_page_vma() to drop the refcount it had acquired. This deserves a rework: but for now fix the leak in shmem_alloc_page(). Reported-by: Tommi Rantala tt.rant...@gmail.com Awaiting-Signed-off-by: Mel Gorman mgor...@suse.de Signed-off-by: Hugh Dickins hu...@google.com Cc

Re: [PATCH 5/5] mempolicy: fix a memory corruption by refcount imbalance in alloc_pages_vma()

2012-12-04 Thread Hugh Dickins
On Tue, 4 Dec 2012, Hugh Dickins wrote: Yes, your patch fixes it Mel, but I prefer it as below, with a couple of mods: removing the no longer true comment, and leaving shmem_swapin() alone with just a comment. It appears to be the job of the rather weird mpol_cond_copy() to drop

[PATCH] tmpfs: fix shared mempolicy leak

2012-12-04 Thread Hugh Dickins
...@suse.de Signed-off-by: Hugh Dickins hu...@google.com Cc: sta...@vger.kernel.org --- include/linux/mempolicy.h | 16 mm/mempolicy.c| 22 -- mm/shmem.c| 26 -- 3 files changed, 16 insertions(+), 48

Re: [PATCH 1/11] ksm: allow trees per NUMA node

2013-02-07 Thread Hugh Dickins
On Tue, 5 Feb 2013, Mel Gorman wrote: On Fri, Jan 25, 2013 at 05:54:53PM -0800, Hugh Dickins wrote: From: Petr Holasek phola...@redhat.com Introduces new sysfs boolean knob /sys/kernel/mm/ksm/merge_across_nodes which control merging pages across different numa nodes. When it is set

Re: [PATCH 4/11] ksm: reorganize ksm_check_stable_tree

2013-02-07 Thread Hugh Dickins
On Tue, 5 Feb 2013, Mel Gorman wrote: On Fri, Jan 25, 2013 at 05:59:35PM -0800, Hugh Dickins wrote: Memory hotremove's ksm_check_stable_tree() is pitifully inefficient (restarting whenever it finds a stale node to remove), but rearrange so that at least it does not needlessly restart from

Re: [PATCH 5/11] ksm: get_ksm_page locked

2013-02-07 Thread Hugh Dickins
On Tue, 5 Feb 2013, Mel Gorman wrote: On Fri, Jan 25, 2013 at 06:00:50PM -0800, Hugh Dickins wrote: In some places where get_ksm_page() is used, we need the page to be locked. When KSM migration is fully enabled, we shall want that to make sure that the page just acquired cannot

Re: [PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-02-08 Thread Hugh Dickins
On Tue, 5 Feb 2013, Mel Gorman wrote: On Fri, Jan 25, 2013 at 06:01:59PM -0800, Hugh Dickins wrote: Switching merge_across_nodes after running KSM is liable to oops on stale nodes still left over from the previous stable tree. It's not something that people will often want to do

Re: [PATCH 7/11] ksm: make KSM page migration possible

2013-02-08 Thread Hugh Dickins
, 5 Feb 2013, Mel Gorman wrote: On Fri, Jan 25, 2013 at 06:03:31PM -0800, Hugh Dickins wrote: KSM page migration is already supported in the case of memory hotremove, which takes the ksm_thread_mutex across all its migrations to keep life simple. But the new KSM NUMA merge_across_nodes

Re: [PATCH 11/11] ksm: stop hotremove lockdep warning

2013-02-11 Thread Hugh Dickins
On Fri, 8 Feb 2013, Gerald Schaefer wrote: On Fri, 25 Jan 2013 18:10:18 -0800 (PST) Hugh Dickins hu...@google.com wrote: Complaints are rare, but lockdep still does not understand the way ksm_memory_callback(MEM_GOING_OFFLINE) takes ksm_thread_mutex, and holds it until

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-12 Thread Hugh Dickins
On Tue, 12 Feb 2013, Dave Jones wrote: My Thinkpad T430s suspend/resumes fine most of the time. But every so often (like one in ten times or so), as soon as I suspend, I get a black screen, and a blinking power button. (Note: Not the capslock lights like when we panic, this laptop

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-12 Thread Hugh Dickins
On Tue, 12 Feb 2013, Dave Jones wrote: On Tue, Feb 12, 2013 at 04:26:22PM -0800, Hugh Dickins wrote: On Tue, 12 Feb 2013, Dave Jones wrote: My Thinkpad T430s suspend/resumes fine most of the time. But every so often (like one in ten times or so), as soon as I suspend, I get

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-12 Thread Hugh Dickins
On Tue, 12 Feb 2013, Dave Jones wrote: On Tue, Feb 12, 2013 at 04:56:55PM -0800, Hugh Dickins wrote: so I can hit this pretty quickly with this.. while [ 1 ]; do rtcwake -s 5 -m no ; pm-suspend done Which should make it bisectable at least.. Thanks

Re: Questin about swap_slot free and invalidate page

2013-02-03 Thread Hugh Dickins
On Thu, 31 Jan 2013, Minchan Kim wrote: When I reviewed zswap, I was curious about frontswap_store. It said following as. * If frontswap already contains a page with matching swaptype and * offset, the frontswap implementation may either overwrite the data and * return success or

Re: [patch] mm: shmem: use new radix tree iterator

2013-02-03 Thread Hugh Dickins
-by: Hugh Dickins hu...@google.com --- mm/shmem.c | 25 - 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index a368a1c..c5dc8ae 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -336,19 +336,19 @@ static unsigned

Re: [patch] mm: shmem: use new radix tree iterator

2013-02-03 Thread Hugh Dickins
On Sat, 2 Feb 2013, Konstantin Khlebnikov wrote: Johannes Weiner wrote: In shmem_find_get_pages_and_swap, use the faster radix tree iterator construct from 78c1d78 radix-tree: introduce bit-optimized iterator. Signed-off-by: Johannes Weinerhan...@cmpxchg.org Hmm, ACK.

[PATCH] memcg: stop warning on memcg_propagate_kmem

2013-02-03 Thread Hugh Dickins
-off-by: Hugh Dickins hu...@google.com --- mm/memcontrol.c |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- 3.8-rc6/mm/memcontrol.c 2012-12-22 09:43:27.628015582 -0800 +++ linux/mm/memcontrol.c 2013-02-02 16:56:06.188325771 -0800 @@ -4969,6 +4969,7 @@ out: return

Re: boot warnings due to swap: make each swap partition have one address_space

2013-02-03 Thread Hugh Dickins
On Wed, 30 Jan 2013, Shaohua Li wrote: On Sun, Jan 27, 2013 at 01:40:40PM -0800, Hugh Dickins wrote: I'm glad Minchan has now pointed you to Rik's posting of two years ago: I think there are more important changes to be made in that direction. Not sure how others use multiple swaps

[PATCH] tmpfs: add support for read_iter and write_iter

2013-02-03 Thread Hugh Dickins
Convert tmpfs do_shmem_file_read() to shmem_file_read_iter(). Make file_read_iter_actor() global so tmpfs can use it too: delete file_read_actor(), which was made global in 2.4.4 for use by tmpfs. Replace tmpfs generic_file_aio_write() by generic_file_write_iter(). Signed-off-by: Hugh Dickins hu

Re: [PATCH v2] printk: Avoid softlockups in console_unlock()

2013-02-06 Thread Hugh Dickins
On Wed, 6 Feb 2013, Jan Kara wrote: On Wed 06-02-13 09:58:48, David Rientjes wrote: On Wed, 6 Feb 2013, Jan Kara wrote: Yes, I noticed that thread just yesterday and also though that using similar trick might be viable. I'll experiment if we could use the same method for handling

Re: [PATCH] mm: Rename page struct field helpers

2013-01-30 Thread Hugh Dickins
this unhelpful and suggested a rename of page_xchg_last_nid to keep with a verb_struct_field naming pattern. This patch replaces mm-rename-page-struct-field-helpers.patch. Suggested-by: Hugh Dickins hu...@google.com Signed-off-by: Mel Gorman mgor...@suse.de FWIW looks just right to me (I don't think

Re: [PATCH, RFC 00/16] Transparent huge page cache

2013-01-30 Thread Hugh Dickins
On Tue, 29 Jan 2013, Kirill A. Shutemov wrote: Hugh Dickins wrote: On Mon, 28 Jan 2013, Kirill A. Shutemov wrote: From: Kirill A. Shutemov kirill.shute...@linux.intel.com Here's first steps towards huge pages in page cache. The intend of the work is get code ready to enable

Re: [PATCH 2/3] mm: accelerate mm_populate() treatment of THP pages

2013-01-30 Thread Hugh Dickins
On Wed, 30 Jan 2013, Michel Lespinasse wrote: This change adds a page_mask argument to follow_page. follow_page sets *page_mask to HPAGE_PMD_NR - 1 when it encounters a THP page, and to 0 in other cases. __get_user_pages() makes use of this in order to accelerate populating THP ranges -

Re: [PATCH] idr: fix a subtle bug in idr_get_next()

2013-02-03 Thread Hugh Dickins
On Sat, 2 Feb 2013, Randy Dunlap wrote: On 02/02/13 15:11, Tejun Heo wrote: On Sat, Feb 02, 2013 at 03:10:48PM -0800, Tejun Heo wrote: Fix it by ensuring proceeding to the next slot doesn't carry over the unaligned offset - ie. use round_up(id + 1, slot_distance) instead of id +=

Re: [PATCH 2/7] ksm: treat unstable nid like in stable tree

2013-03-01 Thread Hugh Dickins
On Fri, 1 Mar 2013, Ric Mason wrote: I think the ksm implementation for num awareness is buggy. Sorry, I just don't understand your comments below, but will try to answer or question them as best I can. For page migratyion stuff, new page is allocated from node *which page is migrated

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Fri, 1 Mar 2013, Simon Jeons wrote: On 03/01/2013 05:22 PM, Simon Jeons wrote: On 02/23/2013 01:56 AM, Johannes Weiner wrote: Mapped file pages have to get scanned twice before they are reclaimed because we don't have enough usage information after the first scan. It seems that

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed page. Right, that's hard to understand without historical background. I think the quick answer

Re: [PATCH 2/7] ksm: treat unstable nid like in stable tree

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Ric Mason wrote: On 03/02/2013 04:03 AM, Hugh Dickins wrote: On Fri, 1 Mar 2013, Ric Mason wrote: I think the ksm implementation for num awareness is buggy. Sorry, I just don't understand your comments below, but will try to answer or question them as best I can

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Hugh Dickins
On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed

[PATCH 0/11] ksm: NUMA trees and page migration

2013-01-25 Thread Hugh Dickins
Here's a KSM series, based on mmotm 2013-01-23-17-04: starting with Petr's v7 KSM: numa awareness sysfs knob; then fixing the two issues we had with that, fully enabling KSM page migration on the way. (A different kind of KSM/NUMA issue which I've certainly not begun to address here: when KSM

[PATCH 1/11] ksm: allow trees per NUMA node

2013-01-25 Thread Hugh Dickins
nodes still left over from the previous stable tree; 2) memory hotremove may migrate KSM pages, but there is no provision here for !merge_across_nodes to migrate nodes to the proper tree. Signed-off-by: Petr Holasek phola...@redhat.com Signed-off-by: Hugh Dickins hu...@google.com Acked-by: Rik van

[PATCH 2/11] ksm: add sysfs ABI Documentation

2013-01-25 Thread Hugh Dickins
From: Petr Holasek phola...@redhat.com This patch adds sysfs documentation for Kernel Samepage Merging (KSM) including new merge_across_nodes knob. Signed-off-by: Petr Holasek phola...@redhat.com Signed-off-by: Hugh Dickins hu...@google.com --- Documentation/ABI/testing/sysfs-kernel-mm-ksm

[PATCH 3/11] ksm: trivial tidyups

2013-01-25 Thread Hugh Dickins
. Signed-off-by: Hugh Dickins hu...@google.com --- mm/ksm.c | 48 ++-- 1 file changed, 22 insertions(+), 26 deletions(-) --- mmotm.orig/mm/ksm.c 2013-01-25 14:36:38.608205618 -0800 +++ mmotm/mm/ksm.c 2013-01-25 14:36:52.152205940 -0800 @@ -41,6 +41,14

[PATCH 4/11] ksm: reorganize ksm_check_stable_tree

2013-01-25 Thread Hugh Dickins
Memory hotremove's ksm_check_stable_tree() is pitifully inefficient (restarting whenever it finds a stale node to remove), but rearrange so that at least it does not needlessly restart from nid 0 each time. And add a couple of comments: here is why we keep pfn instead of page. Signed-off-by: Hugh

[PATCH 5/11] ksm: get_ksm_page locked

2013-01-25 Thread Hugh Dickins
for it. There's no need for rcu_read_lock() around get_page_unless_zero() (and mapping checks) here. Cut out that silliness before making this any harder to understand. Signed-off-by: Hugh Dickins hu...@google.com --- mm/ksm.c | 23 +-- 1 file changed, 13 insertions(+), 10 deletions

[PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-01-25 Thread Hugh Dickins
it then marks Uptodate. Fix this case by not copying, letting do_swap_page() discover the error. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/ksm.h | 18 ++--- mm/ksm.c| 83 +++--- mm/memory.c | 19 - 3 files changed

[PATCH 7/11] ksm: make KSM page migration possible

2013-01-25 Thread Hugh Dickins
heavy hammer of move_vma()'s call to ksm_madvise(,,,MADV_UNMERGEABLE,). Signed-off-by: Hugh Dickins hu...@google.com --- mm/ksm.c | 94 ++--- mm/migrate.c |5 ++ 2 files changed, 77 insertions(+), 22 deletions(-) --- mmotm.orig/mm/ksm.c 2013

[PATCH 8/11] ksm: make !merge_across_nodes migration safe

2013-01-25 Thread Hugh Dickins
of stale entries from time to time (scan_get_next_rmap_item() does it once each full scan): whereas stale nodes in the stable tree get naturally pruned as searches try to brush past them, these migrate_nodes may get forgotten and accumulate. Signed-off-by: Hugh Dickins hu...@google.com --- mm/ksm.c

[PATCH 9/11] ksm: enable KSM page migration

2013-01-25 Thread Hugh Dickins
MADV_MERGEABLE on any area where this is a worry. Signed-off-by: Hugh Dickins hu...@google.com --- mm/mempolicy.c |3 +-- mm/migrate.c | 21 +++-- 2 files changed, 4 insertions(+), 20 deletions(-) --- mmotm.orig/mm/mempolicy.c 2013-01-24 12:28:38.848127553 -0800 +++ mmotm/mm

[PATCH 10/11] mm: remove offlining arg to migrate_pages

2013-01-25 Thread Hugh Dickins
No functional change, but the only purpose of the offlining argument to migrate_pages() etc, was to ensure that __unmap_and_move() could migrate a KSM page for memory hotremove (which took ksm_thread_mutex) but not for other callers. Now all cases are safe, remove the arg. Signed-off-by: Hugh

[PATCH 11/11] ksm: stop hotremove lockdep warning

2013-01-25 Thread Hugh Dickins
, and wait_while_offlining() checks, to achieve the same lockout without being caught by lockdep. This is less elegant for KSM, but it's more important to keep lockdep useful to other users - and I apologize for how long it took to fix. Reported-by: Gerald Schaefer gerald.schae...@de.ibm.com Signed-off-by: Hugh

Re: boot warnings due to swap: make each swap partition have one address_space

2013-01-26 Thread Hugh Dickins
On Fri, 25 Jan 2013, Shaohua Li wrote: On Thu, Jan 24, 2013 at 10:45:57PM -0500, Sasha Levin wrote: Hi folks, Commit swap: make each swap partition have one address_space is triggering a series of warnings on boot: [3.446071] [ cut here ] [3.446664]

Re: [PATCH 1/11] ksm: allow trees per NUMA node

2013-01-26 Thread Hugh Dickins
On Sat, 26 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 17:54 -0800, Hugh Dickins wrote: From: Petr Holasek phola...@redhat.com @@ -1122,6 +1166,18 @@ struct rmap_item *unstable_tree_search_i return NULL

Re: [PATCH 6/6] mm: numa: Cleanup flow of transhuge page migration

2013-01-27 Thread Hugh Dickins
On Tue, 22 Jan 2013, Mel Gorman wrote: From: Hugh Dickins hu...@google.com When correcting commit 04fa5d6a (mm: migrate: check page_count of THP before migrating) Hugh Dickins noted that the control flow for transhuge migration was difficult to follow. Unconditionally calling put_page

Re: boot warnings due to swap: make each swap partition have one address_space

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Shaohua Li wrote: On Sat, Jan 26, 2013 at 06:16:05PM -0800, Hugh Dickins wrote: On Fri, 25 Jan 2013, Shaohua Li wrote: On Thu, Jan 24, 2013 at 10:45:57PM -0500, Sasha Levin wrote: Subject: give-each-swapper-space-separate-backing_dev_info

[PATCH next/mmotm] swap: add per-partition lock for swapfile fix

2013-01-27 Thread Hugh Dickins
I had all cpus spinning in swap_info_get(), for the lock on an area being swapped off: probably because get_swap_page() forgot to unlock. Signed-off-by: Hugh Dickins hu...@google.com --- mm/swapfile.c |5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- mmotm.orig/mm/swapfile.c

Re: [PATCH 1/11] ksm: allow trees per NUMA node

2013-01-27 Thread Hugh Dickins
On Sat, 26 Jan 2013, Simon Jeons wrote: On Sat, 2013-01-26 at 18:54 -0800, Hugh Dickins wrote: So you'd like us to add code for moving a node from one tree to another in ksm_migrate_page() (and what would it do when it collides with an Without numa awareness, I still can't understand

Re: [PATCH 5/11] ksm: get_ksm_page locked

2013-01-27 Thread Hugh Dickins
On Sat, 26 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 18:00 -0800, Hugh Dickins wrote: In some places where get_ksm_page() is used, we need the page to be locked. In function get_ksm_page, why check page-mapping = get_page_unless_zero = check page-mapping instead

Re: [PATCH 5/11] ksm: get_ksm_page locked

2013-01-27 Thread Hugh Dickins
On Sat, 26 Jan 2013, Simon Jeons wrote: BTW, what's the meaning of ksm page forked? A ksm page is mapped into a process's mm, then that process calls fork(): the ksm page then appears in the child's mm, before ksmd has tracked it. -- To unsubscribe from this list: send the line unsubscribe

Re: [PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-01-27 Thread Hugh Dickins
On Sat, 26 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 18:01 -0800, Hugh Dickins wrote: Switching merge_across_nodes after running KSM is liable to oops on stale nodes still left over from the previous stable tree. It's not something that people will often want to do, but it would

Re: [PATCH 7/11] ksm: make KSM page migration possible

2013-01-27 Thread Hugh Dickins
On Sat, 26 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 18:03 -0800, Hugh Dickins wrote: + while (!get_page_unless_zero(page)) { + /* +* Another check for page-mapping != expected_mapping would +* work here too. We have chosen the !PageSwapCache

Re: [PATCH 8/11] ksm: make !merge_across_nodes migration safe

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 18:05 -0800, Hugh Dickins wrote: @@ -1344,10 +1401,29 @@ static void cmp_and_merge_page(struct pa unsigned int checksum; int err; - remove_rmap_item_from_tree(rmap_item); + stable_node = page_stable_node(page

Re: [PATCH 11/11] ksm: stop hotremove lockdep warning

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 18:10 -0800, Hugh Dickins wrote: @@ -2098,15 +2117,15 @@ static int ksm_memory_callback(struct no switch (action) { case MEM_GOING_OFFLINE: /* -* Keep it very simple for now: just lock out ksmd

Re: [PATCH 5/11] ksm: get_ksm_page locked

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Simon Jeons wrote: On Sun, 2013-01-27 at 14:08 -0800, Hugh Dickins wrote: On Sat, 26 Jan 2013, Simon Jeons wrote: Why the parameter lock passed from stable_tree_search/insert is true, but remove_rmap_item_from_tree is false? The other way round

Re: [PATCH 7/11] ksm: make KSM page migration possible

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Simon Jeons wrote: On Sun, 2013-01-27 at 15:12 -0800, Hugh Dickins wrote: On Sat, 26 Jan 2013, Simon Jeons wrote: Could you explain why need check page-mapping twice after get page? Once for the !locked case, which should not return page if mapping changed

Re: [PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Simon Jeons wrote: On Sun, 2013-01-27 at 15:05 -0800, Hugh Dickins wrote: On Sat, 26 Jan 2013, Simon Jeons wrote: How can this happen? We only permit switching merge_across_nodes when pages_shared is 0, and usually set run 2 to force that beforehand, which

Re: [PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-01-27 Thread Hugh Dickins
On Sun, 27 Jan 2013, Simon Jeons wrote: On Fri, 2013-01-25 at 18:01 -0800, Hugh Dickins wrote: Switching merge_across_nodes after running KSM is liable to oops on stale nodes still left over from the previous stable tree. It's not something Since this patch solve the problem, so

Re: [PATCH 0/11] ksm: NUMA trees and page migration

2013-01-28 Thread Hugh Dickins
On Mon, 28 Jan 2013, Andrew Morton wrote: On Fri, 25 Jan 2013 17:53:10 -0800 (PST) Hugh Dickins hu...@google.com wrote: Here's a KSM series Sanity check: do you have a feeling for how useful KSM is? Performance/space improvements for typical (or atypical) workloads? Are people using

Re: [PATCH 1/11] ksm: allow trees per NUMA node

2013-01-28 Thread Hugh Dickins
On Mon, 28 Jan 2013, Andrew Morton wrote: On Fri, 25 Jan 2013 17:54:53 -0800 (PST) Hugh Dickins hu...@google.com wrote: --- mmotm.orig/Documentation/vm/ksm.txt 2013-01-25 14:36:31.724205455 -0800 +++ mmotm/Documentation/vm/ksm.txt 2013-01-25 14:36:38.608205618 -0800 @@ -58,6

Re: [PATCH 1/11] ksm: allow trees per NUMA node

2013-01-28 Thread Hugh Dickins
On Mon, 28 Jan 2013, Andrew Morton wrote: On Fri, 25 Jan 2013 17:54:53 -0800 (PST) Hugh Dickins hu...@google.com wrote: +/* Zeroed when merging across nodes is not allowed */ +static unsigned int ksm_merge_across_nodes = 1; I spose this should be __read_mostly. If __read_mostly

Re: [PATCH 3/11] ksm: trivial tidyups

2013-01-28 Thread Hugh Dickins
On Mon, 28 Jan 2013, Andrew Morton wrote: On Fri, 25 Jan 2013 17:58:11 -0800 (PST) Hugh Dickins hu...@google.com wrote: +#ifdef CONFIG_NUMA +#define NUMA(x)(x) +#define DO_NUMA(x) (x) Did we consider #define DO_NUMA do { (x) } while (0) ? It didn't occur to me

Re: [PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-01-28 Thread Hugh Dickins
On Mon, 28 Jan 2013, Andrew Morton wrote: On Fri, 25 Jan 2013 18:01:59 -0800 (PST) Hugh Dickins hu...@google.com wrote: +static int remove_all_stable_nodes(void) +{ + struct stable_node *stable_node; + int nid; + int err = 0; + + for (nid = 0; nid nr_node_ids; nid

Re: [PATCH] mm: Rename page struct field helpers

2013-01-28 Thread Hugh Dickins
On Thu, 24 Jan 2013, Mel Gorman wrote: The function names page_xchg_last_nid(), page_last_nid() and reset_page_last_nid() were judged to be inconsistent so rename them to a struct_field_op style pattern. As it looked jarring to have reset_page_mapcount() and page_nid_reset_last() beside each

Re: [PATCH, RFC 00/16] Transparent huge page cache

2013-01-28 Thread Hugh Dickins
On Mon, 28 Jan 2013, Kirill A. Shutemov wrote: From: Kirill A. Shutemov kirill.shute...@linux.intel.com Here's first steps towards huge pages in page cache. The intend of the work is get code ready to enable transparent huge page cache for the most simple fs -- ramfs. It's not yet near

iwlwifi: regression in 3.8-rc4 and 3.7.3

2013-01-19 Thread Hugh Dickins
After sending the first 2MB, scp over wireless becomes unbearably slow, with frequent stalls: on this ThinkPad T420s running 3.8-rc4 or 3.7.3. Not always, but often. Bisection led to commit f590dcec944552f9a4a61155810f3abd17d6465d iwlwifi: fix the reclaimed packet tracking upon flush queue

RE: iwlwifi: regression in 3.8-rc4 and 3.7.3

2013-01-19 Thread Hugh Dickins
On Sun, 20 Jan 2013, Grumbach, Emmanuel wrote: On Sun, Jan 20, 2013 at 6:56 AM, Hugh Dickins hu...@google.com wrote: After sending the first 2MB, scp over wireless becomes unbearably slow, with frequent stalls: on this ThinkPad T420s running 3.8-rc4 or 3.7.3. Not always

Re: [PATCH] mm: Limit pgd range freeing to mm-task_size

2013-02-14 Thread Hugh Dickins
On Wed, 13 Feb 2013, Andrew Morton wrote: On Wed, 13 Feb 2013 11:39:29 + Catalin Marinas catalin.mari...@arm.com wrote: ARM processors with LPAE enabled use 3 levels of page tables, with an entry in the top level (pgd) covering 1GB of virtual space. Because of the branch relocation

Re: [PATCH 6/11] ksm: remove old stable nodes more thoroughly

2013-02-14 Thread Hugh Dickins
On Thu, 14 Feb 2013, Mel Gorman wrote: On Fri, Feb 08, 2013 at 11:33:40AM -0800, Hugh Dickins wrote: What I found is that a 4th cause emerges once KSM migration is properly working: that interval during page migration when the old page has been fully unmapped but the new not yet mapped

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-14 Thread Hugh Dickins
On Thu, 14 Feb 2013, Dave Jones wrote: On Wed, Feb 13, 2013 at 11:56:25AM -0800, Linus Torvalds wrote: On Wed, Feb 13, 2013 at 11:34 AM, Dave Jones da...@redhat.com wrote: My test was a loop of 100 suspend/resume cycles before calling something 'good'. The 'bad' cases all failed

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-16 Thread Hugh Dickins
On Thu, 14 Feb 2013, Linus Torvalds wrote: On Thu, Feb 14, 2013 at 6:09 PM, Hugh Dickins hu...@google.com wrote: Which won't affect my case since I never enabled it. Well, in theory, you may have the same bug Dave just made it easier to trigger for himself with the forced config option

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-16 Thread Hugh Dickins
On Sat, 16 Feb 2013, Linus Torvalds wrote: On Sat, Feb 16, 2013 at 1:45 PM, Hugh Dickins hu...@google.com wrote: I hacked around on your PM_TRACE set_magic_time() / read_magic_time() yesterday, to save an oopsing core kernel ip there, instead of hashed pm trace info (it makes sense

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-16 Thread Hugh Dickins
On Sat, 16 Feb 2013, Hugh Dickins wrote: On Sat, 16 Feb 2013, Linus Torvalds wrote: I think it's worth it to give them a heads-up already. So I've cc'd the main suspects here.. Okay, thanks. Daniel, Dave - any comments about a NULL fb in intel_choose_pipe_bpp_dither() during

Re: Debugging Thinkpad T430s occasional suspend failure.

2013-02-17 Thread Hugh Dickins
On Sun, 17 Feb 2013, Daniel Vetter wrote: On Sun, Feb 17, 2013 at 3:21 AM, Hugh Dickins hu...@google.com wrote: On Sat, 16 Feb 2013, Hugh Dickins wrote: On Sat, 16 Feb 2013, Linus Torvalds wrote: I think it's worth it to give them a heads-up already. So I've cc'd the main suspects

Re: Should a swapped out page be deleted from swap cache?

2013-02-18 Thread Hugh Dickins
On Mon, 18 Feb 2013, Li Haifeng wrote: For explain my question, the two points should be displayed as below. 1. If an anonymous page is swapped out, this page will be deleted from swap cache and be put back into buddy system. Yes, unless the page is referenced again before it comes to be

Re: [PATCH 1/2] mm: Allow arch code to control the user page table ceiling

2013-02-19 Thread Hugh Dickins
On Mon, 18 Feb 2013, Catalin Marinas wrote: From: Hugh Dickins hu...@google.com You're being generous to me :) Thanks for doing most of the work, yes, this looks fine. BUt I'd have expected a Cc stable below: see comment on 2/2. Hugh On architectures where a pgd entry may be shared between

Re: [PATCH 2/2] arm: Set the page table freeing ceiling to TASK_SIZE

2013-02-19 Thread Hugh Dickins
Marinas catalin.mari...@arm.com Cc: Russell King li...@arm.linux.org.uk Cc: Hugh Dickins hu...@google.com Cc: Andrew Morton a...@linux-foundation.org --- arch/arm/include/asm/pgtable.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm

Re: Should a swapped out page be deleted from swap cache?

2013-02-19 Thread Hugh Dickins
On Tue, 19 Feb 2013, Ric Mason wrote: There is a call of try_to_free_swap in function swap_writepage, if swap_writepage is call from shrink_page_list path, PageSwapCache(page) == trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, then will delete the page from swap cache and

Re: Should a swapped out page be deleted from swap cache?

2013-02-19 Thread Hugh Dickins
On Tue, 19 Feb 2013, Will Huck wrote: Another question: I don't see the connection to deleting a swapped out page from swap cache. Why kernel memory mapping use direct mapping instead of kmalloc/vmalloc which will setup mapping on demand? I may misunderstand you, and kernel memory

Re: Should a swapped out page be deleted from swap cache?

2013-02-19 Thread Hugh Dickins
On Tue, 19 Feb 2013, Sha Zhengju wrote: On Tue, Feb 19, 2013 at 2:53 PM, Li Haifeng omy...@gmail.com wrote: 2013/2/19 Will Huck will.hu...@gmail.com: On 02/19/2013 10:04 AM, Li Haifeng wrote: If a anonymous page is swapped out and comes to be reclaimable, shrink_page_list() will call

Re: [PATCH 1/2] tmpfs: fix use-after-free of mempolicy object

2013-02-20 Thread Hugh Dickins
the corruption which 2.6.35 then immediately bestowed upon mpol=local. Signed-off-by: Greg Thelen gthe...@google.com Acked-by: Hugh Dickins hu...@google.com I'm inclined to add Cc: sta...@vger.kernel.org to this one, but not to the 2/2 leak one; but I'm perfectly happy if Andrew chooses differently

Re: [PATCH 2/2] tmpfs: fix mempolicy object leaks

2013-02-20 Thread Hugh Dickins
, and nicely fixed - but one doubt below. If you resolve that, please add my Acked-by: Hugh Dickins hu...@google.com Signed-off-by: Greg Thelen gthe...@google.com --- mm/shmem.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index

[PATCH 0/7] ksm: responses to NUMA review

2013-02-21 Thread Hugh Dickins
Here's a second KSM series, based on mmotm 2013-02-19-17-20: partly in response to Mel's review feedback, partly fixes to issues that I found myself in doing more review and testing. None of the issues fixed are truly show-stoppers, though I would prefer them fixed sooner than later. 1 ksm: add

<    1   2   3   4   5   6   7   8   9   10   >