[Qemu-devel] [PATCH] mm: compaction: Correct the nr_strict_isolated check for CMA

2012-10-16 Thread Mel Gorman
fails and I missed that this fix was not picked up before RC1. This same problem has been identified in 3.7-RC1 by Tony Prisk and should be addressed by the following patch. Signed-off-by: Mel Gorman mgor...@suse.de Tested-by: Tony Prisk li...@prisktech.co.nz --- mm/compaction.c |2 +- 1 file

[Qemu-devel] [PATCH] mm: compaction: cache if a pageblock was scanned and no pages were isolated -fix2

2012-09-27 Thread Mel Gorman
The clearing of PG_migrate_skip potentially takes a long time if the zone is massive. Be safe and check if it needs to reschedule. This is a fix for mm-compaction-cache-if-a-pageblock-was-scanned-and-no-pages-were-isolated.patch Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c

Re: [Qemu-devel] [PATCH 8/9] mm: compaction: Cache if a pageblock was scanned and no pages were isolated

2012-09-27 Thread Mel Gorman
On Wed, Sep 26, 2012 at 09:49:30AM +0900, Minchan Kim wrote: On Tue, Sep 25, 2012 at 10:12:07AM +0100, Mel Gorman wrote: On Mon, Sep 24, 2012 at 02:26:44PM -0700, Andrew Morton wrote: On Mon, 24 Sep 2012 10:39:38 +0100 Mel Gorman mgor...@suse.de wrote: On Fri, Sep 21, 2012 at 02

Re: [Qemu-devel] [PATCH 8/9] mm: compaction: Cache if a pageblock was scanned and no pages were isolated

2012-09-27 Thread Mel Gorman
On Tue, Sep 25, 2012 at 01:03:52PM -0700, Andrew Morton wrote: On Tue, 25 Sep 2012 10:12:07 +0100 Mel Gorman mgor...@suse.de wrote: First, we'd introduce a variant of get_pageblock_migratetype() that returns all the bits for the pageblock flags and then helpers to extract either

Re: [Qemu-devel] [PATCH 5/9] mm: compaction: Acquire the zone-lru_lock as late as possible

2012-09-26 Thread Mel Gorman
without preventing THP * splitting the page underneath us may return * surprising results. -- Mel Gorman SUSE Labs

Re: [Qemu-devel] [PATCH 5/9] mm: compaction: Acquire the zone-lru_lock as late as possible

2012-09-25 Thread Mel Gorman
On Tue, Sep 25, 2012 at 04:05:17PM +0900, Minchan Kim wrote: Hi Mel, I have a question below. On Fri, Sep 21, 2012 at 11:46:19AM +0100, Mel Gorman wrote: Compactions migrate scanner acquires the zone-lru_lock when scanning a range of pages looking for LRU pages to acquire. It does

Re: [Qemu-devel] [PATCH 8/9] mm: compaction: Cache if a pageblock was scanned and no pages were isolated

2012-09-25 Thread Mel Gorman
On Mon, Sep 24, 2012 at 02:26:44PM -0700, Andrew Morton wrote: On Mon, 24 Sep 2012 10:39:38 +0100 Mel Gorman mgor...@suse.de wrote: On Fri, Sep 21, 2012 at 02:36:56PM -0700, Andrew Morton wrote: Also, what has to be done to avoid the polling altogether? eg/ie, zap a pageblock's

Re: [Qemu-devel] [PATCH 6/9] mm: compaction: Acquire the zone-lock as late as possible

2012-09-24 Thread Mel Gorman
On Fri, Sep 21, 2012 at 02:35:57PM -0700, Andrew Morton wrote: On Fri, 21 Sep 2012 11:46:20 +0100 Mel Gorman mgor...@suse.de wrote: Compactions free scanner acquires the zone-lock when checking for PageBuddy pages and isolating them. It does this even if there are no PageBuddy pages

Re: [Qemu-devel] [PATCH 8/9] mm: compaction: Cache if a pageblock was scanned and no pages were isolated

2012-09-24 Thread Mel Gorman
On Fri, Sep 21, 2012 at 02:36:56PM -0700, Andrew Morton wrote: On Fri, 21 Sep 2012 11:46:22 +0100 Mel Gorman mgor...@suse.de wrote: When compaction was implemented it was known that scanning could potentially be excessive. The ideal was that a counter be maintained for each pageblock

Re: [Qemu-devel] [PATCH 0/6] Reduce compaction scanning and lock contention

2012-09-21 Thread Mel Gorman
for the detailed reporting and the testing, it's much appreciated. I've already rebased the patches to Andrew's tree and tested them overnight and the figures look good on my side. I'll update the changelog and push them shortly. -- Mel Gorman SUSE Labs

Re: [Qemu-devel] [PATCH 0/6] Reduce compaction scanning and lock contention

2012-09-21 Thread Mel Gorman
| | | | | do_huge_pmd_anonymous_page This whole section is interesting simply because it shows the per-cpu draining cost. It's low enough that I'm not going to put much thought into it but it's not often the per-cpu allocator sticks out like this. Thanks Richard. -- Mel Gorman SUSE Labs

[Qemu-devel] [PATCH 2/9] Revert mm-compaction-abort-compaction-loop-if-lock-is-contended-or-run-too-long-fix

2012-09-21 Thread Mel Gorman
This reverts mm-compaction-abort-compaction-loop-if-lock-is-contended-or-run-too-long-fix as it is replaced by a later patch in the series. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c |3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/compaction.c b/mm

[Qemu-devel] [PATCH 4/9] mm: compaction: Abort compaction loop if lock is contended or run too long

2012-09-21 Thread Mel Gorman
: Putback pages isolated for migration if aborting] [a...@linux-foundation.org: compact_zone_order requires non-NULL arg contended] Signed-off-by: Andrea Arcangeli aarca...@redhat.com Signed-off-by: Shaohua Li s...@fusionio.com Signed-off-by: Mel Gorman mgor...@suse.de Acked-by: Rik van Riel r

[Qemu-devel] [PATCH 8/9] mm: compaction: Cache if a pageblock was scanned and no pages were isolated

2012-09-21 Thread Mel Gorman
. Signed-off-by: Mel Gorman mgor...@suse.de Acked-by: Rik van Riel r...@redhat.com --- include/linux/mmzone.h |3 ++ include/linux/pageblock-flags.h | 19 +++- mm/compaction.c | 93 +-- mm/internal.h |1 + mm

[Qemu-devel] [PATCH 3/9] Revert mm: compaction: abort compaction loop if lock is contended or run too long

2012-09-21 Thread Mel Gorman
This reverts mm-compaction-abort-compaction-loop-if-lock-is-contended-or-run-too-long.patch as it is replaced by a later patch in the series. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c | 12 +--- mm/internal.h |2 +- 2 files changed, 6 insertions(+), 8

[Qemu-devel] [PATCH 9/9] mm: compaction: Restart compaction from near where it left off

2012-09-21 Thread Mel Gorman
-by: Mel Gorman mgor...@suse.de Acked-by: Rik van Riel r...@redhat.com --- include/linux/mmzone.h |4 mm/compaction.c| 54 mm/internal.h |4 3 files changed, 53 insertions(+), 9 deletions(-) diff --git a/include/linux

[Qemu-devel] [PATCH 1/9] Revert mm: compaction: check lock contention first before taking lock

2012-09-21 Thread Mel Gorman
This reverts mm-compaction-check-lock-contention-first-before-taking-lock.patch as it is replaced by a later patch in the series. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c |5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mm/compaction.c b/mm

[Qemu-devel] [PATCH 5/9] mm: compaction: Acquire the zone-lru_lock as late as possible

2012-09-21 Thread Mel Gorman
which reduces contention on zone-lru_lock. Signed-off-by: Mel Gorman mgor...@suse.de Acked-by: Rik van Riel r...@redhat.com --- mm/compaction.c | 63 +-- 1 file changed, 43 insertions(+), 20 deletions(-) diff --git a/mm/compaction.c b/mm

[Qemu-devel] [PATCH 0/9] Reduce compaction scanning and lock contention

2012-09-21 Thread Mel Gorman
Hi Andrew, Richard Davies and Shaohua Li have both reported lock contention problems in compaction on the zone and LRU locks as well as significant amounts of time being spent in compaction. This series aims to reduce lock contention and scanning rates to reduce that CPU usage. Richard reported

[Qemu-devel] [PATCH 7/9] Revert mm: have order 0 compaction start off where it left

2012-09-21 Thread Mel Gorman
but the implementation is complex and tricky to understand. A later patch will cache what pageblocks should be skipped and reimplements the concept of compact_cached_free_pfn on top for both migration and free scanners. Signed-off-by: Mel Gorman mgor...@suse.de Acked-by: Rik van Riel r...@redhat.com --- include/linux

[Qemu-devel] [PATCH 6/9] mm: compaction: Acquire the zone-lock as late as possible

2012-09-21 Thread Mel Gorman
then the lock will not be acquired at all which reduces contention on zone-lock. Signed-off-by: Mel Gorman mgor...@suse.de Acked-by: Rik van Riel r...@redhat.com --- mm/compaction.c | 141 ++- 1 file changed, 78 insertions(+), 63 deletions(-) diff --git

[Qemu-devel] [PATCH 0/6] Reduce compaction scanning and lock contention

2012-09-20 Thread Mel Gorman
Hi Richard, This series is following up from your mail at http://www.spinics.net/lists/kvm/msg80080.html . I am pleased the lock contention is now reduced but acknowledge that the scanning rates are stupidly high. Fortunately, I am reasonably confident I know what is going wrong. If all goes

[Qemu-devel] [PATCH 2/6] mm: compaction: Acquire the zone-lru_lock as late as possible

2012-09-20 Thread Mel Gorman
which reduces contention on zone-lru_lock. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c | 63 +-- 1 file changed, 43 insertions(+), 20 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index a8de20d..6450c3e 100644

[Qemu-devel] [PATCH 6/6] mm: compaction: Restart compaction from near where it left off

2012-09-20 Thread Mel Gorman
-by: Mel Gorman mgor...@suse.de --- include/linux/mmzone.h |4 mm/compaction.c| 54 mm/internal.h |4 3 files changed, 53 insertions(+), 9 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h

[Qemu-devel] [PATCH 5/6] mm: compaction: Cache if a pageblock was scanned and no pages were isolated

2012-09-20 Thread Mel Gorman
. Signed-off-by: Mel Gorman mgor...@suse.de --- include/linux/mmzone.h |3 ++ include/linux/pageblock-flags.h | 19 +++- mm/compaction.c | 93 +-- mm/internal.h |1 + mm/page_alloc.c |1

[Qemu-devel] [PATCH 4/6] Revert mm: have order 0 compaction start off where it left

2012-09-20 Thread Mel Gorman
but the implementation is complex and tricky to understand. A later patch will cache what pageblocks should be skipped and reimplements the concept of compact_cached_free_pfn on top for both migration and free scanners. Signed-off-by: Mel Gorman mgor...@suse.de --- include/linux/mmzone.h |4 --- mm/compaction.c

[Qemu-devel] [PATCH 1/6] mm: compaction: Abort compaction loop if lock is contended or run too long

2012-09-20 Thread Mel Gorman
: Putback pages isolated for migration if aborting] [a...@linux-foundation.org: compact_zone_order requires non-NULL arg contended] Signed-off-by: Andrea Arcangeli aarca...@redhat.com Signed-off-by: Shaohua Li s...@fusionio.com Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c | 17

[Qemu-devel] [PATCH 3/6] mm: compaction: Acquire the zone-lock as late as possible

2012-09-20 Thread Mel Gorman
then the lock will not be acquired at all which reduces contention on zone-lock. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c | 141 +-- 1 file changed, 75 insertions(+), 66 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c

Re: [Qemu-devel] [PATCH -v2 2/2] make the compaction skip ahead logic robust

2012-09-18 Thread Mel Gorman
in the pageblock then the lock will not be acquired at all. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c | 80 --- 1 file changed, 47 insertions(+), 33 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index a5d698f..57ff9ef 100644

Re: [Qemu-devel] [PATCH -v2 2/2] make the compaction skip ahead logic robust

2012-09-17 Thread Mel Gorman
the LRU lock for as long as possible. In cases where transparent hugepages are encountered the LRU lock will not be acquired at all. Signed-off-by: Mel Gorman mgor...@suse.de --- mm/compaction.c | 65 +-- 1 file changed, 44 insertions(+), 21

Re: [Qemu-devel] [PATCH -v2 2/2] make the compaction skip ahead logic robust

2012-09-17 Thread Mel Gorman
are behaving as expected and from there decide if the migrate scanner needs similar logic. -- Mel Gorman SUSE Labs

Re: [Qemu-devel] Windows VM slow boot

2012-09-13 Thread Mel Gorman
On Wed, Sep 12, 2012 at 05:46:15PM +0100, Richard Davies wrote: Hi Mel - thanks for replying to my underhand bcc! Mel Gorman wrote: I see that this is an old-ish bug but I did not read the full history. Is it now booting faster than 3.5.0 was? I'm asking because I'm interested to see

Re: [Qemu-devel] Windows VM slow boot

2012-09-12 Thread Mel Gorman
the lock contention. We'd better check if the lock is contended first. compact_trylock_irqsave perfectly meets the requirement. Signed-off-by: Shaohua Li s...@fusionio.com Acked-by: Mel Gorman mgor...@suse.de Acked-by: Minchan Kim minc...@kernel.org Signed-off-by: Andrew Morton a...@linux-foundation.org