RE: Re: [PATCH v2 13/18] mm/compaction: support non-lru movable pagemigration
Hmmm... But, in failure case, is it safe to call putback_lru_page() for them?
And, PageIsolated() would be left. Is it okay? It's not symmetric that
isolated page can be freed by decreasing ref count without calling
putback function. This should be clarified and documented.
I agree Joonsoo's idea.Freeing isolated page out of putback() could be confused.Every detail cannot be documented. And more documents mean less elegant code.Is it possible to free isolated page in putback()?In move_to_new_page(), can we call a_ops->migratepage like following?move_to_new_page(){mapping = page_mapping(page)if (!mapping) rc = migrate_pageelse if (mapping->a_ops->migratepage && IsolatePage(page)) rc = mapping->a_ops->migratepageelse rc = fallback_migrate_page... return rc}I'm sorry that I couldn't review in detail because I forgot many details.
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v1 03/19] fs/anon_inodes: new interface to create new inode
On 11.03.2016 09:05, Al Viro wrote:
On Fri, Mar 11, 2016 at 04:30:07PM +0900, Minchan Kim wrote:
From: Gioh Kim <guru...@hanmail.net>
The anon_inodes has already complete interfaces to create manage
many anonymous inodes but don't have interface to get
new inode. Other sub-modules can create anonymous inode
without creating and mounting it's own pseudo filesystem.
IMO that's a bad idea. In case of aio "creating and mounting" takes this:
static struct dentry *aio_mount(struct file_system_type *fs_type,
int flags, const char *dev_name, void *data)
{
static const struct dentry_operations ops = {
.d_dname= simple_dname,
};
return mount_pseudo(fs_type, "aio:", NULL, , AIO_RING_MAGIC);
}
and
static struct file_system_type aio_fs = {
.name = "aio",
.mount = aio_mount,
.kill_sb= kill_anon_super,
};
aio_mnt = kern_mount(_fs);
All of 12 lines. Your export is not much shorter. To quote old mail on
the same topic:
I know what aio_setup() does. It can be a solution.
But I thought creating anon_inode_new() is simpler than several drivers
create its own pseudo filesystem.
Creating a filesystem requires memory allocation and locking some lists
even though it is pseudo.
Could you inform me if there is a reason we should avoid creating
anonymous inode?
Note that anon_inodes.c reason to exist was "it's for situations where
all context lives on struct file and we don't need separate inode for
them". Going from that to "it happens to contain a handy function for inode
allocation"...
--
Best regards,
Gioh Kim
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 0/4] enable migration of driver pages
2015-07-29 오후 9:46에 Daniel Vetter 이(가) 쓴 글: On Wed, Jul 29, 2015 at 01:16:14PM +0100, Mel Gorman wrote: On Wed, Jul 29, 2015 at 12:55:54PM +0200, Daniel Vetter wrote: On Wed, Jul 29, 2015 at 11:49:45AM +0100, Mel Gorman wrote: On Mon, Jul 13, 2015 at 05:35:15PM +0900, Gioh Kim wrote: My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. The primary motivation of this series is to reduce fragmentation by allowing more kernel pages to be moved. Conceptually that is a worthwhile goal but there should be at least one major in-kernel user and while balloon pages were a good starting point, I think we really need to see what the zram changes look like at the same time. I think gpu drivers really would be the perfect candidate for compacting kernel page allocations. And this also seems the primary motivation for this patch series, so I think that's really what we should use to judge these patches. Of course then there's the seemingly eternal chicken/egg problem of upstream gpu drivers for SoCs :( I recognised that the driver he had modified was not an in-tree user so it did not really help the review or the design. I did not think it was very fair to ask that an in-tree GPU driver be converted when it would not help the embedded platform of interest. Converting zram is both a useful illustration of the aops requirements and is expected to be beneficial on the embedded platform. Now, if a GPU driver author was willing to convert theirs as an example then that would be useful! Well my concern is more with merging infrastructure to upstream for drivers which aren't upstream and with no plan to make that happen anytime soon. Seems like just offload a bit to me ... but in the end core mm isn't my thing so not my decision. -Daniel I get idea from the out-tree driver but this infrastructure will be useful for zram and balloon. That is agreed by the maintainers of each driver. I'm currently accepting feedbacks from balloon and zram and trying to be applicable for them. Of course I hope there will be more application. It'll be more useful if it has more application. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 4/4] mm: remove direct calling of migration
2015-07-27 오후 10:58에 Vlastimil Babka 이(가) 쓴 글:
On 07/13/2015 10:35 AM, Gioh Kim wrote:
From: Gioh Kim guru...@hanmail.net
Migration is completely generalized so that migrating mobile page
is processed with lru-pages in move_to_new_page.
Signed-off-by: Gioh Kim gioh@lge.com
Acked-by: Rafael Aquini aqu...@redhat.com
Why not just fold this to Patch 3? You already modify this hunk there, and
prior to patch 3, the hunk was balloon-pages specific. You made it look generic
only to remove it, which is unneeded code churn and I don't think it adds
anything wrt e.g. bisectability.
Yes, you have a point.
I'll be merged into 3/4 at next spin.
I wanted to show the process how migration is generalized with this patch.
On second thought it is not necessary.
---
mm/migrate.c | 15 ---
1 file changed, 15 deletions(-)
diff --git a/mm/migrate.c b/mm/migrate.c
index 53f0081d..e6644ac 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -844,21 +844,6 @@ static int __unmap_and_move(struct page *page, struct page
*newpage,
}
}
-if (unlikely(mobile_page(page))) {
-/*
- * A mobile page does not need any special attention from
- * physical to virtual reverse mapping procedures.
- * Skip any attempt to unmap PTEs or to remap swap cache,
- * in order to avoid burning cycles at rmap level, and perform
- * the page migration right away (proteced by page lock).
- */
-lock_page(newpage);
-rc = page-mapping-a_ops-migratepage(page-mapping,
- newpage, page, mode);
-unlock_page(newpage);
-goto out_unlock;
-}
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 2/4] mm/compaction: enable mobile-page migration
On Mon, Jul 27, 2015 at 4:55 PM, Vlastimil Babka vba...@suse.cz wrote:
On 07/13/2015 10:35 AM, Gioh Kim wrote:
From: Gioh Kim guru...@hanmail.net
Add framework to register callback functions and check page mobility.
There are some modes for page isolation so that isolate interface
has arguments of page address and isolation mode while putback
interface has only page address as argument.
Note that unlike what subject suggest, this doesn't really enable
mobile-page migration inside compaction, since that only happens with patch
3. This might theoretically affect some cherry-pick backports that don't
care about balloon pages. I can imagine that can easily happen in the world
of mobile devices?
It would thus be somewhat cleaner if this patch was complete in that sense.
You have a point.
Current 2/4 patch is lack of calling migrate/isolate.
It is not complete without 3/4.
I'll add calling migrate/isolate() at next spin.
Signed-off-by: Gioh Kim gioh@lge.com
Acked-by: Rafael Aquini aqu...@redhat.com
---
fs/proc/page.c | 3 ++
include/linux/compaction.h | 80
++
include/linux/fs.h | 2 +
include/linux/page-flags.h | 19
include/uapi/linux/kernel-page-flags.h | 1 +
5 files changed, 105 insertions(+)
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8..a4f5a00 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -146,6 +146,9 @@ u64 stable_page_flags(struct page *page)
if (PageBalloon(page))
u |= 1 KPF_BALLOON;
+ if (PageMobile(page))
+ u |= 1 KPF_MOBILE;
PageMovable() would probably be as good a name and correspond to
MIGRATE_MOVABLE somewhat, unlike a completely new term. Whatever driver
starts to using this should probably change allocation flags to allocate
MIGRATE_MOVABLE, so that it works fine with what fragmentation avoidance
expects. Guess I should have said that earlier, but can you still
reconsider?
Well, I've suggested to name it mobile because there's already a lot of things
called movable. Mobile pages are special subset of movable pages: they
are non-lru pages and define their own rules of moving in address
space operations.
Also there's a little pun: I guess main user will zram which is used
mostly in embedded/mobile devices.
I like mobile.
movable is for pages allocated with GFP_MOVABLE.
I think movable is a little ambiguous in this situation.
+
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index aa8f61c..f693072 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h
@@ -1,6 +1,9 @@
#ifndef _LINUX_COMPACTION_H
#define _LINUX_COMPACTION_H
+#include linux/page-flags.h
+#include linux/pagemap.h
+
/* Return values for compact_zone() and try_to_compact_pages() */
/* compaction didn't start as it was deferred due to past failures */
#define COMPACT_DEFERRED 0
@@ -51,6 +54,70 @@ extern void compaction_defer_reset(struct zone *zone,
int order,
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+static inline bool mobile_page(struct page *page)
+{
+ return page-mapping (PageMobile(page) || PageBalloon(page));
+}
I would put this definition to linux/page-flags.h and rename it to
page_mobile (or better page_movable()), which is more common ordering.
+
+static inline bool isolate_mobilepage(struct page *page, isolate_mode_t
mode)
Does this have to be in compaction.h? The only user is compaction.c so
probably move it there, and if there ever is another module using this in
the future, we can move it to a more appropriate place and declare it in
e.g. mm/internal.h.
I think compaction.c is suitable.
+{
+ bool ret = false;
+
+ /*
+* Avoid burning cycles with pages that are yet under
__free_pages(),
+* or just got freed under us.
+*
+* In case we 'win' a race for a mobile page being freed under us
and
+* raise its refcount preventing __free_pages() from doing its job
+* the put_page() at the end of this block will take care of
+* release this page, thus avoiding a nasty leakage.
+*/
+ if (unlikely(!get_page_unless_zero(page)))
+ goto out;
+
+ /*
+* As mobile pages are not isolated from LRU lists, concurrent
+* compaction threads can race against page migration functions
+* as well as race against the releasing a page.
+*
+* In order to avoid having an already isolated mobile page
+* being (wrongly) re-isolated while it is under migration,
+* or to avoid attempting to isolate pages being released,
+* lets be sure we have the page lock
+* before proceeding
Re: [RFCv3 2/5] mm/compaction: enable mobile-page migration
@@ -51,6 +54,66 @@ extern void compaction_defer_reset(struct zone *zone, int
order,
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+static inline bool mobile_page(struct page *page)
+{
+ return page-mapping page-mapping-a_ops
Dereferncing mapping-a_ops isn't safe without page-lock and isn't required:
all mappings always have -a_ops.
I got it.
+static inline void putback_mobilepage(struct page *page)
+{
+ /*
+* 'lock_page()' stabilizes the page and prevents races against
+* concurrent isolation threads attempting to re-isolate it.
+*/
+ lock_page(page);
+ if (mobile_page(page) page-mapping-a_ops-putbackpage) {
It seems if (page-mapping page-mapping-a_ops-putbackpage)
should be enough: we already seen that page as mobile.
Ditto.
+ page-mapping-a_ops-putbackpage(page);
+ /* drop the extra ref count taken for mobile page isolation */
+ put_page(page);
+ }
+ unlock_page(page);
call put_page() after unlock and do that always -- putback must drop
page reference from caller.
lock_page(page);
if (page-mapping page-mapping-a_ops-putbackpage)
page-mapping-a_ops-putbackpage(page);
unlock_page();
put_page(page);
Ditto.
+}
#else
static inline unsigned long try_to_compact_pages(gfp_t gfp_mask,
unsigned int order, int alloc_flags,
@@ -83,6 +146,19 @@ static inline bool compaction_deferred(struct zone *zone,
int order)
return true;
}
+static inline bool mobile_page(struct page *page)
+{
+ return false;
+}
+
+static inline bool isolate_mobilepage(struct page *page, isolate_mode_t mode)
+{
+ return false;
+}
+
+static inline void putback_mobilepage(struct page *page)
+{
+}
#endif /* CONFIG_COMPACTION */
#if defined(CONFIG_COMPACTION) defined(CONFIG_SYSFS)
defined(CONFIG_NUMA)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 35ec87e..33c9aa5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -395,6 +395,8 @@ struct address_space_operations {
*/
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
+ bool (*isolatepage) (struct page *, isolate_mode_t);
+ void (*putbackpage) (struct page *);
int (*launder_page) (struct page *);
int (*is_partially_uptodate) (struct page *, unsigned long,
unsigned long);
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index f34e040..abef145 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -582,6 +582,25 @@ static inline void __ClearPageBalloon(struct page *page)
atomic_set(page-_mapcount, -1);
}
+#define PAGE_MOBILE_MAPCOUNT_VALUE (-255)
+
+static inline int PageMobile(struct page *page)
+{
+ return atomic_read(page-_mapcount) == PAGE_MOBILE_MAPCOUNT_VALUE;
+}
+
+static inline void __SetPageMobile(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(page-_mapcount) != -1, page);
+ atomic_set(page-_mapcount, PAGE_MOBILE_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageMobile(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageMobile(page), page);
+ atomic_set(page-_mapcount, -1);
+}
+
/*
* If network-based swap is enabled, sl*b must keep track of whether pages
* were allocated from pfmemalloc reserves.
diff --git a/include/uapi/linux/kernel-page-flags.h
b/include/uapi/linux/kernel-page-flags.h
index a6c4962..d50d9e8 100644
--- a/include/uapi/linux/kernel-page-flags.h
+++ b/include/uapi/linux/kernel-page-flags.h
@@ -33,6 +33,7 @@
#define KPF_THP22
#define KPF_BALLOON23
#define KPF_ZERO_PAGE 24
+#define KPF_MOBILE 25
#endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
--
2.1.4
I fixed the code as your comments and I found patch 3/5 and 4/5 could not be
applied separately.
So I merge them and report new [PATCH].
I appreciate your reviews.
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 4/4] mm: remove direct calling of migration
From: Gioh Kim guru...@hanmail.net
Migration is completely generalized so that migrating mobile page
is processed with lru-pages in move_to_new_page.
Signed-off-by: Gioh Kim gioh@lge.com
Acked-by: Rafael Aquini aqu...@redhat.com
---
mm/migrate.c | 15 ---
1 file changed, 15 deletions(-)
diff --git a/mm/migrate.c b/mm/migrate.c
index 53f0081d..e6644ac 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -844,21 +844,6 @@ static int __unmap_and_move(struct page *page, struct page
*newpage,
}
}
- if (unlikely(mobile_page(page))) {
- /*
-* A mobile page does not need any special attention from
-* physical to virtual reverse mapping procedures.
-* Skip any attempt to unmap PTEs or to remap swap cache,
-* in order to avoid burning cycles at rmap level, and perform
-* the page migration right away (proteced by page lock).
-*/
- lock_page(newpage);
- rc = page-mapping-a_ops-migratepage(page-mapping,
- newpage, page, mode);
- unlock_page(newpage);
- goto out_unlock;
- }
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
--
2.1.4
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 3/4] mm/balloon: apply mobile page migratable into balloon
From: Gioh Kim guru...@hanmail.net
Apply mobile page migration into balloon driver.
The balloong driver has an anonymous inode that manages
address_space_operation for page migration.
Compaction calls interfaces of mobile page migration
instead of calling balloon migration directly.
Signed-off-by: Gioh Kim gioh@lge.com
Acked-by: Rafael Aquini aqu...@redhat.com
---
drivers/virtio/virtio_balloon.c| 3 ++
include/linux/balloon_compaction.h | 15 ++--
mm/balloon_compaction.c| 72 --
mm/compaction.c| 8 ++---
mm/migrate.c | 21 ++-
5 files changed, 54 insertions(+), 65 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 82e80e0..ef5b9b5 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -30,6 +30,7 @@
#include linux/balloon_compaction.h
#include linux/oom.h
#include linux/wait.h
+#include linux/anon_inodes.h
/*
* Balloon device works in 4K page units. So each page is pointed to by
@@ -505,6 +506,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
vb-vb_dev_info.migratepage = virtballoon_migratepage;
+ vb-vb_dev_info.inode = anon_inode_new();
+ vb-vb_dev_info.inode-i_mapping-a_ops = balloon_aops;
#endif
err = init_vqs(vb);
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..a9e0bde 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -48,6 +48,7 @@
#include linux/migrate.h
#include linux/gfp.h
#include linux/err.h
+#include linux/fs.h
/*
* Balloon device information descriptor.
@@ -62,6 +63,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct inode *inode;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -73,12 +75,16 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
spin_lock_init(balloon-pages_lock);
INIT_LIST_HEAD(balloon-pages);
balloon-migratepage = NULL;
+ balloon-inode = NULL;
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
+extern const struct address_space_operations balloon_aops;
+extern bool balloon_page_isolate(struct page *page,
+isolate_mode_t mode);
extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
struct page *page, enum migrate_mode mode);
/*
@@ -124,6 +130,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
struct page *page)
{
__SetPageBalloon(page);
+ page-mapping = balloon-inode-i_mapping;
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
list_add(page-lru, balloon-pages);
@@ -140,6 +147,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
static inline void balloon_page_delete(struct page *page)
{
__ClearPageBalloon(page);
+ page-mapping = NULL;
set_page_private(page, 0);
if (PagePrivate(page)) {
ClearPagePrivate(page);
@@ -191,7 +199,8 @@ static inline bool isolated_balloon_page(struct page *page)
return false;
}
-static inline bool balloon_page_isolate(struct page *page)
+static inline bool balloon_page_isolate(struct page *page,
+ isolate_mode_t mode)
{
return false;
}
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index fcad832..8fbcf9c 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -131,43 +131,16 @@ static inline void __putback_balloon_page(struct page
*page)
}
/* __isolate_lru_page() counterpart for a ballooned page */
-bool balloon_page_isolate(struct page *page)
+bool balloon_page_isolate(struct page *page, isolate_mode_t mode)
{
/*
-* Avoid burning cycles with pages that are yet under __free_pages(),
-* or just got freed under us.
-*
-* In case we 'win' a race for a balloon page being freed under us and
-* raise its refcount preventing __free_pages() from doing its job
-* the put_page() at the end of this block will take care of
-* release this page, thus avoiding a nasty leakage.
+* A ballooned page, by default, has PagePrivate set.
+* Prevent concurrent compaction threads from isolating
Re: [PATCH 0/4] enable migration of driver pages
2015-07-13 오후 6:24에 Konstantin Khlebnikov 이(가) 쓴 글: On Mon, Jul 13, 2015 at 11:35 AM, Gioh Kim gioh@lge.com wrote: From: Gioh Kim guru...@hanmail.net Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. So I thought there needs a interface to combine driver and kernel compaction. This patch adds a generic isolate/migrate/putback callbacks for page address-space and a new interface to create anon-inode to manage address_space_operation. The zram and GPU, and any other modules can create anon_inode and register its own migration method. The kernel compaction can call the registered migration when it does compaction. My GPU driver source is not in-kernel driver so that I apply the interface into balloon driver. The balloon driver is already merged into the kernel compaction as a corner-case. This patch have the balloon driver migration be called by the generic interface. This patch set combines 4 patches. 1. patch 1/4: get inode from anon_inodes This patch adds new interface to create inode from anon_inodes. 2. patch 2/4: framework to isolate/migrate/putback page Add isolatepage, putbackpage into address_space_operations and wrapper function to call them. 3. patch 3/4: apply the framework into balloon driver The balloon driver is applied into the framework. It gets a inode from anon_inodes and register operations in the inode. The kernel compaction calls generic interfaces, not balloon driver interfaces. Any other drivers can register operations via inode like this to migrate it's pages. 4. patch 4/4: remove direct calling of migration of driver pages Non-lru pages are also migrated with lru pages by move_to_new_page(). The whole patchset looks good. Reviewed-by: Konstantin Khlebnikov koc...@gmail.com This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated Another simple test is several instances of tools/testing/selftests/vm/transhuge-stress.c runnng in parallel with balloon inflating/deflating. (transparent huge pages must be enabled of course) That catched a lot of races in ballooning code. Great! I'll do it and inform you the result in this week. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 0/4] enable migration of driver pages
From: Gioh Kim guru...@hanmail.net Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. So I thought there needs a interface to combine driver and kernel compaction. This patch adds a generic isolate/migrate/putback callbacks for page address-space and a new interface to create anon-inode to manage address_space_operation. The zram and GPU, and any other modules can create anon_inode and register its own migration method. The kernel compaction can call the registered migration when it does compaction. My GPU driver source is not in-kernel driver so that I apply the interface into balloon driver. The balloon driver is already merged into the kernel compaction as a corner-case. This patch have the balloon driver migration be called by the generic interface. This patch set combines 4 patches. 1. patch 1/4: get inode from anon_inodes This patch adds new interface to create inode from anon_inodes. 2. patch 2/4: framework to isolate/migrate/putback page Add isolatepage, putbackpage into address_space_operations and wrapper function to call them. 3. patch 3/4: apply the framework into balloon driver The balloon driver is applied into the framework. It gets a inode from anon_inodes and register operations in the inode. The kernel compaction calls generic interfaces, not balloon driver interfaces. Any other drivers can register operations via inode like this to migrate it's pages. 4. patch 4/4: remove direct calling of migration of driver pages Non-lru pages are also migrated with lru pages by move_to_new_page(). This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated My thanks to Konstantin Khlebnikov for his reviews of the RFC patch set. Most of the changes were based on his feedback. This patch-set is based on v4.1 Gioh Kim (4): fs/anon_inodes: new interface to create new inode mm/compaction: enable mobile-page migration mm/balloon: apply mobile page migratable into balloon mm: remove direct calling of migration drivers/virtio/virtio_balloon.c| 3 ++ fs/anon_inodes.c | 6 +++ fs/proc/page.c | 3 ++ include/linux/anon_inodes.h| 1 + include/linux/balloon_compaction.h | 15 +-- include/linux/compaction.h | 80 ++ include/linux/fs.h | 2 + include/linux/page-flags.h | 19 include/uapi/linux/kernel-page-flags.h | 1 + mm/balloon_compaction.c| 72 ++ mm/compaction.c| 8 ++-- mm/migrate.c | 24 +++--- 12 files changed, 160 insertions(+), 74 deletions(-) -- 2.1.4 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 2/4] mm/compaction: enable mobile-page migration
From: Gioh Kim guru...@hanmail.net
Add framework to register callback functions and check page mobility.
There are some modes for page isolation so that isolate interface
has arguments of page address and isolation mode while putback
interface has only page address as argument.
Signed-off-by: Gioh Kim gioh@lge.com
Acked-by: Rafael Aquini aqu...@redhat.com
---
fs/proc/page.c | 3 ++
include/linux/compaction.h | 80 ++
include/linux/fs.h | 2 +
include/linux/page-flags.h | 19
include/uapi/linux/kernel-page-flags.h | 1 +
5 files changed, 105 insertions(+)
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8..a4f5a00 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -146,6 +146,9 @@ u64 stable_page_flags(struct page *page)
if (PageBalloon(page))
u |= 1 KPF_BALLOON;
+ if (PageMobile(page))
+ u |= 1 KPF_MOBILE;
+
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index aa8f61c..f693072 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h
@@ -1,6 +1,9 @@
#ifndef _LINUX_COMPACTION_H
#define _LINUX_COMPACTION_H
+#include linux/page-flags.h
+#include linux/pagemap.h
+
/* Return values for compact_zone() and try_to_compact_pages() */
/* compaction didn't start as it was deferred due to past failures */
#define COMPACT_DEFERRED 0
@@ -51,6 +54,70 @@ extern void compaction_defer_reset(struct zone *zone, int
order,
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+static inline bool mobile_page(struct page *page)
+{
+ return page-mapping (PageMobile(page) || PageBalloon(page));
+}
+
+static inline bool isolate_mobilepage(struct page *page, isolate_mode_t mode)
+{
+ bool ret = false;
+
+ /*
+* Avoid burning cycles with pages that are yet under __free_pages(),
+* or just got freed under us.
+*
+* In case we 'win' a race for a mobile page being freed under us and
+* raise its refcount preventing __free_pages() from doing its job
+* the put_page() at the end of this block will take care of
+* release this page, thus avoiding a nasty leakage.
+*/
+ if (unlikely(!get_page_unless_zero(page)))
+ goto out;
+
+ /*
+* As mobile pages are not isolated from LRU lists, concurrent
+* compaction threads can race against page migration functions
+* as well as race against the releasing a page.
+*
+* In order to avoid having an already isolated mobile page
+* being (wrongly) re-isolated while it is under migration,
+* or to avoid attempting to isolate pages being released,
+* lets be sure we have the page lock
+* before proceeding with the mobile page isolation steps.
+*/
+ if (unlikely(!trylock_page(page)))
+ goto out_putpage;
+
+ if (!(mobile_page(page) page-mapping-a_ops-isolatepage))
+ goto out_not_isolated;
+ ret = page-mapping-a_ops-isolatepage(page, mode);
+ if (!ret)
+ goto out_not_isolated;
+ unlock_page(page);
+ return ret;
+
+out_not_isolated:
+ unlock_page(page);
+out_putpage:
+ put_page(page);
+out:
+ return ret;
+}
+
+static inline void putback_mobilepage(struct page *page)
+{
+ /*
+* 'lock_page()' stabilizes the page and prevents races against
+* concurrent isolation threads attempting to re-isolate it.
+*/
+ lock_page(page);
+ if (page-mapping page-mapping-a_ops-putbackpage)
+ page-mapping-a_ops-putbackpage(page);
+ unlock_page(page);
+ /* drop the extra ref count taken for mobile page isolation */
+ put_page(page);
+}
#else
static inline unsigned long try_to_compact_pages(gfp_t gfp_mask,
unsigned int order, int alloc_flags,
@@ -83,6 +150,19 @@ static inline bool compaction_deferred(struct zone *zone,
int order)
return true;
}
+static inline bool mobile_page(struct page *page)
+{
+ return false;
+}
+
+static inline bool isolate_mobilepage(struct page *page, isolate_mode_t mode)
+{
+ return false;
+}
+
+static inline void putback_mobilepage(struct page *page)
+{
+}
#endif /* CONFIG_COMPACTION */
#if defined(CONFIG_COMPACTION) defined(CONFIG_SYSFS) defined(CONFIG_NUMA)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index a0653e5..2cc4b24 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -396,6 +396,8 @@ struct address_space_operations {
*/
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
+ bool
[PATCH 1/4] fs/anon_inodes: new interface to create new inode
From: Gioh Kim guru...@hanmail.net
The anon_inodes has already complete interfaces to create manage
many anonymous inodes but don't have interface to get
new inode. Other sub-modules can create anonymous inode
without creating and mounting it's own pseudo filesystem.
Signed-off-by: Gioh Kim gioh@lge.com
Acked-by: Rafael Aquini aqu...@redhat.com
---
fs/anon_inodes.c| 6 ++
include/linux/anon_inodes.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/fs/anon_inodes.c b/fs/anon_inodes.c
index 80ef38c..1d51f96 100644
--- a/fs/anon_inodes.c
+++ b/fs/anon_inodes.c
@@ -162,6 +162,12 @@ err_put_unused_fd:
}
EXPORT_SYMBOL_GPL(anon_inode_getfd);
+struct inode *anon_inode_new(void)
+{
+ return alloc_anon_inode(anon_inode_mnt-mnt_sb);
+}
+EXPORT_SYMBOL_GPL(anon_inode_new);
+
static int __init anon_inode_init(void)
{
anon_inode_mnt = kern_mount(anon_inode_fs_type);
diff --git a/include/linux/anon_inodes.h b/include/linux/anon_inodes.h
index 8013a45..ddbd67f 100644
--- a/include/linux/anon_inodes.h
+++ b/include/linux/anon_inodes.h
@@ -15,6 +15,7 @@ struct file *anon_inode_getfile(const char *name,
void *priv, int flags);
int anon_inode_getfd(const char *name, const struct file_operations *fops,
void *priv, int flags);
+struct inode *anon_inode_new(void);
#endif /* _LINUX_ANON_INODES_H */
--
2.1.4
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFCv3 0/5] enable migration of driver pages
2015-07-09 오후 10:08에 Daniel Vetter 이(가) 쓴 글: Also there's a bit a lack of gpu drivers from the arm world in upstream, which is probabyl why this patch series doesn't come with a user. Might be better to first upstream the driver before talking about additional infrastructure that it needs. -Daniel I'm not from ARM but I just got the idea of driver page migration during I worked with ARM gpu driver. I'm sure this patch is good for zram and balloon and hope it can be applied to drivers consuming many pages and generating fragmentation, such as GPU or gfx driver. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFCv3 3/5] mm/balloon: apply mobile page migratable into balloon
@@ -124,6 +130,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
struct page *page)
{
__SetPageBalloon(page);
+ page-mapping = balloon-inode-i_mapping;
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
list_add(page-lru, balloon-pages);
@@ -140,6 +147,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
static inline void balloon_page_delete(struct page *page)
{
__ClearPageBalloon(page);
+ page-mapping = NULL;
set_page_private(page, 0);
if (PagePrivate(page)) {
ClearPagePrivate(page);
Order of cleanup here is not the reverse of the order of initialization.
Better make it exactly the reverse.
Also, I have a question: is it enough to lock the page to make changing
the mapping safe? Do all users lock the page too?
I think balloon developers can answer that precisely.
I've just follow this comment:
http://lxr.free-electrons.com/source/include/linux/balloon_compaction.h#L16
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFCv3 0/5] enable migration of driver pages
2015-07-09 오전 7:47에 Dave Airlie 이(가) 쓴 글: Can the various in-kernel GPU drivers benefit from this? If so, wiring up one or more of those would be helpful? I'm sure that other in-kernel GPU drivers can have benefit. It must be helpful. If I was familiar with other in-kernel GPU drivers code, I tried to patch them. It's too bad. I'll bring dri-devel into the loop here. ARM GPU developers please take a look at this stuff, Laurent, Rob, Eric I suppose. I sent a patch, https://lkml.org/lkml/2015/3/24/1182, and my opinion about compaction to ARM GPU developers via Korea ARM branch. I got a reply that they had no time to review it. I hope they're interested to this patch. Daniel Vetter you might have some opinions as well. Dave. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFCv3 0/5] enable migration of driver pages
2015-07-08 오전 7:37에 Andrew Morton 이(가) 쓴 글: On Tue, 7 Jul 2015 13:36:20 +0900 Gioh Kim gioh@lge.com wrote: From: Gioh Kim guru...@hanmail.net Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. ... This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated OK, but what happens if the balloon driver is not used to force compaction? Does your test machine successfully compact pages on demand, so those order-3 allocations now succeed? If any driver that has many pages like the balloon driver is forced to compact, the system can get free high-order pages. I have to show how this patch work with a driver existing in the kernel source, for kernel developers' undestanding. So I selected the balloon driver because it has already compaction and working with kernel compaction. I can show how driver pages is compacted with lru-pages together. Actually balloon driver is not best example to show how this patch compacts pages. The balloon driver compaction is decreasing page consumtion, for instance 1024MB - 512MB. I think it is not compaction precisely. It frees pages. Of course there will be many high-order pages after 512MB is freed. Why are your changes to the GPU driver not included in this patch series? My platform is ARM-based and GPU is ARM-Mali. The driver is not open source. It's too bad that I cannot show effect of this patch with the GPU driver. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFCv3 0/5] enable migration of driver pages
2015-07-08 오전 9:07에 Andrew Morton 이(가) 쓴 글: On Wed, 08 Jul 2015 09:02:59 +0900 Gioh Kim gioh@lge.com wrote: 2015-07-08 __ 7:37___ Andrew Morton ___(___) ___ ___: On Tue, 7 Jul 2015 13:36:20 +0900 Gioh Kim gioh@lge.com wrote: From: Gioh Kim guru...@hanmail.net Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. ... This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated OK, but what happens if the balloon driver is not used to force compaction? Does your test machine successfully compact pages on demand, so those order-3 allocations now succeed? If any driver that has many pages like the balloon driver is forced to compact, the system can get free high-order pages. I have to show how this patch work with a driver existing in the kernel source, for kernel developers' undestanding. So I selected the balloon driver because it has already compaction and working with kernel compaction. I can show how driver pages is compacted with lru-pages together. Actually balloon driver is not best example to show how this patch compacts pages. The balloon driver compaction is decreasing page consumtion, for instance 1024MB - 512MB. I think it is not compaction precisely. It frees pages. Of course there will be many high-order pages after 512MB is freed. Can the various in-kernel GPU drivers benefit from this? If so, wiring up one or more of those would be helpful? I'm sure that other in-kernel GPU drivers can have benefit. It must be helpful. If I was familiar with other in-kernel GPU drivers code, I tried to patch them. It's too bad. Minchan Kim said he had a plan to apply this patch into zram compaction. Many embedded machines use several hundreds MB for zram. The zram can also have benefit with this patch as much as GPU drivers. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv3 4/5] mm: call generic migration callbacks
From: Gioh Kim guru...@hanmail.net
Compaction calls interfaces of mobile page migration
instead of calling balloon migration directly.
Signed-off-by: Gioh Kim gioh@lge.com
---
mm/compaction.c | 8
mm/migrate.c| 19 ++-
2 files changed, 14 insertions(+), 13 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 81bafaf..60e4cbb 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -14,7 +14,7 @@
#include linux/backing-dev.h
#include linux/sysctl.h
#include linux/sysfs.h
-#include linux/balloon_compaction.h
+#include linux/compaction.h
#include linux/page-isolation.h
#include linux/kasan.h
#include internal.h
@@ -714,12 +714,12 @@ isolate_migratepages_block(struct compact_control *cc,
unsigned long low_pfn,
/*
* Check may be lockless but that's ok as we recheck later.
-* It's possible to migrate LRU pages and balloon pages
+* It's possible to migrate LRU pages and mobile pages
* Skip any other type of page
*/
if (!PageLRU(page)) {
- if (unlikely(balloon_page_movable(page))) {
- if (balloon_page_isolate(page, isolate_mode)) {
+ if (unlikely(mobile_page(page))) {
+ if (isolate_mobilepage(page, isolate_mode)) {
/* Successfully isolated */
goto isolate_success;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index c94038e..e22be67 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -35,7 +35,7 @@
#include linux/hugetlb.h
#include linux/hugetlb_cgroup.h
#include linux/gfp.h
-#include linux/balloon_compaction.h
+#include linux/compaction.h
#include linux/mmu_notifier.h
#include asm/tlbflush.h
@@ -76,7 +76,7 @@ int migrate_prep_local(void)
* from where they were once taken off for compaction/migration.
*
* This function shall be used whenever the isolated pageset has been
- * built from lru, balloon, hugetlbfs page. See isolate_migratepages_range()
+ * built from lru, mobile, hugetlbfs page. See isolate_migratepages_range()
* and isolate_huge_page().
*/
void putback_movable_pages(struct list_head *l)
@@ -92,8 +92,8 @@ void putback_movable_pages(struct list_head *l)
list_del(page-lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
- if (unlikely(isolated_balloon_page(page)))
- balloon_page_putback(page);
+ if (unlikely(mobile_page(page)))
+ putback_mobilepage(page);
else
putback_lru_page(page);
}
@@ -844,15 +844,16 @@ static int __unmap_and_move(struct page *page, struct
page *newpage,
}
}
- if (unlikely(isolated_balloon_page(page))) {
+ if (unlikely(mobile_page(page))) {
/*
-* A ballooned page does not need any special attention from
+* A mobile page does not need any special attention from
* physical to virtual reverse mapping procedures.
* Skip any attempt to unmap PTEs or to remap swap cache,
* in order to avoid burning cycles at rmap level, and perform
* the page migration right away (proteced by page lock).
*/
- rc = balloon_page_migrate(page-mapping, newpage, page, mode);
+ rc = page-mapping-a_ops-migratepage(page-mapping,
+ newpage, page, mode);
goto out_unlock;
}
@@ -960,8 +961,8 @@ out:
if (rc != MIGRATEPAGE_SUCCESS put_new_page) {
ClearPageSwapBacked(newpage);
put_new_page(newpage, private);
- } else if (unlikely(__is_movable_balloon_page(newpage))) {
- /* drop our reference, page already in the balloon */
+ } else if (unlikely(mobile_page(newpage))) {
+ /* drop our reference */
put_page(newpage);
} else
putback_lru_page(newpage);
--
2.1.4
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv3 2/5] mm/compaction: enable mobile-page migration
From: Gioh Kim guru...@hanmail.net
Add framework to register callback functions and check page mobility.
There are some modes for page isolation so that isolate interface
has arguments of page address and isolation mode while putback
interface has only page address as argument.
Signed-off-by: Gioh Kim gioh@lge.com
---
fs/proc/page.c | 3 ++
include/linux/compaction.h | 76 ++
include/linux/fs.h | 2 +
include/linux/page-flags.h | 19 +
include/uapi/linux/kernel-page-flags.h | 1 +
5 files changed, 101 insertions(+)
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8..a4f5a00 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -146,6 +146,9 @@ u64 stable_page_flags(struct page *page)
if (PageBalloon(page))
u |= 1 KPF_BALLOON;
+ if (PageMobile(page))
+ u |= 1 KPF_MOBILE;
+
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index aa8f61c..c375a89 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h
@@ -1,6 +1,9 @@
#ifndef _LINUX_COMPACTION_H
#define _LINUX_COMPACTION_H
+#include linux/page-flags.h
+#include linux/pagemap.h
+
/* Return values for compact_zone() and try_to_compact_pages() */
/* compaction didn't start as it was deferred due to past failures */
#define COMPACT_DEFERRED 0
@@ -51,6 +54,66 @@ extern void compaction_defer_reset(struct zone *zone, int
order,
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+static inline bool mobile_page(struct page *page)
+{
+ return page-mapping page-mapping-a_ops
+ (PageMobile(page) || PageBalloon(page));
+}
+
+static inline bool isolate_mobilepage(struct page *page, isolate_mode_t mode)
+{
+ bool ret;
+
+ /*
+* Avoid burning cycles with pages that are yet under __free_pages(),
+* or just got freed under us.
+*
+* In case we 'win' a race for a mobile page being freed under us and
+* raise its refcount preventing __free_pages() from doing its job
+* the put_page() at the end of this block will take care of
+* release this page, thus avoiding a nasty leakage.
+*/
+ if (likely(get_page_unless_zero(page))) {
+ /*
+* As mobile pages are not isolated from LRU lists, concurrent
+* compaction threads can race against page migration functions
+* as well as race against the releasing a page.
+*
+* In order to avoid having an already isolated mobile page
+* being (wrongly) re-isolated while it is under migration,
+* or to avoid attempting to isolate pages being released,
+* lets be sure we have the page lock
+* before proceeding with the mobile page isolation steps.
+*/
+ if (likely(trylock_page(page))) {
+ if (mobile_page(page)
+ page-mapping-a_ops-isolatepage) {
+ ret = page-mapping-a_ops-isolatepage(page,
+ mode);
+ unlock_page(page);
+ return ret;
+ }
+ unlock_page(page);
+ }
+ put_page(page);
+ }
+ return false;
+}
+
+static inline void putback_mobilepage(struct page *page)
+{
+ /*
+* 'lock_page()' stabilizes the page and prevents races against
+* concurrent isolation threads attempting to re-isolate it.
+*/
+ lock_page(page);
+ if (mobile_page(page) page-mapping-a_ops-putbackpage) {
+ page-mapping-a_ops-putbackpage(page);
+ /* drop the extra ref count taken for mobile page isolation */
+ put_page(page);
+ }
+ unlock_page(page);
+}
#else
static inline unsigned long try_to_compact_pages(gfp_t gfp_mask,
unsigned int order, int alloc_flags,
@@ -83,6 +146,19 @@ static inline bool compaction_deferred(struct zone *zone,
int order)
return true;
}
+static inline bool mobile_page(struct page *page)
+{
+ return false;
+}
+
+static inline bool isolate_mobilepage(struct page *page, isolate_mode_t mode)
+{
+ return false;
+}
+
+static inline void putback_mobilepage(struct page *page)
+{
+}
#endif /* CONFIG_COMPACTION */
#if defined(CONFIG_COMPACTION) defined(CONFIG_SYSFS) defined(CONFIG_NUMA)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 35ec87e..33c9aa5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -395,6 +395,8
[RFCv3 5/5] mm: remove direct calling of migration
From: Gioh Kim guru...@hanmail.net
Migration is completely generalized so that migrating mobile page
is processed with lru-pages in move_to_new_page.
Signed-off-by: Gioh Kim gioh@lge.com
---
mm/balloon_compaction.c | 8
mm/migrate.c| 13 -
2 files changed, 21 deletions(-)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 0dd0b0d..9d07ed9 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -170,13 +170,6 @@ int balloon_page_migrate(struct address_space *mapping,
if (!isolated_balloon_page(page))
return rc;
- /*
-* Block others from accessing the 'newpage' when we get around to
-* establishing additional references. We should be the only one
-* holding a reference to the 'newpage' at this point.
-*/
- BUG_ON(!trylock_page(newpage));
-
if (WARN_ON(!__is_movable_balloon_page(page))) {
dump_page(page, not movable balloon page);
unlock_page(newpage);
@@ -186,7 +179,6 @@ int balloon_page_migrate(struct address_space *mapping,
if (balloon balloon-migratepage)
rc = balloon-migratepage(balloon, newpage, page, mode);
- unlock_page(newpage);
return rc;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index e22be67..b82539b 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -844,19 +844,6 @@ static int __unmap_and_move(struct page *page, struct page
*newpage,
}
}
- if (unlikely(mobile_page(page))) {
- /*
-* A mobile page does not need any special attention from
-* physical to virtual reverse mapping procedures.
-* Skip any attempt to unmap PTEs or to remap swap cache,
-* in order to avoid burning cycles at rmap level, and perform
-* the page migration right away (proteced by page lock).
-*/
- rc = page-mapping-a_ops-migratepage(page-mapping,
- newpage, page, mode);
- goto out_unlock;
- }
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
--
2.1.4
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv3 3/5] mm/balloon: apply mobile page migratable into balloon
From: Gioh Kim guru...@hanmail.net
Apply mobile page migration into balloon driver.
The balloong driver has an anonymous inode that manages
address_space_operation for page migration.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c| 3 ++
include/linux/balloon_compaction.h | 15 +++--
mm/balloon_compaction.c| 65 +-
mm/compaction.c| 2 +-
mm/migrate.c | 2 +-
5 files changed, 39 insertions(+), 48 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 82e80e0..ef5b9b5 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -30,6 +30,7 @@
#include linux/balloon_compaction.h
#include linux/oom.h
#include linux/wait.h
+#include linux/anon_inodes.h
/*
* Balloon device works in 4K page units. So each page is pointed to by
@@ -505,6 +506,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
vb-vb_dev_info.migratepage = virtballoon_migratepage;
+ vb-vb_dev_info.inode = anon_inode_new();
+ vb-vb_dev_info.inode-i_mapping-a_ops = balloon_aops;
#endif
err = init_vqs(vb);
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..a9e0bde 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -48,6 +48,7 @@
#include linux/migrate.h
#include linux/gfp.h
#include linux/err.h
+#include linux/fs.h
/*
* Balloon device information descriptor.
@@ -62,6 +63,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct inode *inode;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -73,12 +75,16 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
spin_lock_init(balloon-pages_lock);
INIT_LIST_HEAD(balloon-pages);
balloon-migratepage = NULL;
+ balloon-inode = NULL;
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
+extern const struct address_space_operations balloon_aops;
+extern bool balloon_page_isolate(struct page *page,
+isolate_mode_t mode);
extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
struct page *page, enum migrate_mode mode);
/*
@@ -124,6 +130,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
struct page *page)
{
__SetPageBalloon(page);
+ page-mapping = balloon-inode-i_mapping;
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
list_add(page-lru, balloon-pages);
@@ -140,6 +147,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
static inline void balloon_page_delete(struct page *page)
{
__ClearPageBalloon(page);
+ page-mapping = NULL;
set_page_private(page, 0);
if (PagePrivate(page)) {
ClearPagePrivate(page);
@@ -191,7 +199,8 @@ static inline bool isolated_balloon_page(struct page *page)
return false;
}
-static inline bool balloon_page_isolate(struct page *page)
+static inline bool balloon_page_isolate(struct page *page,
+ isolate_mode_t mode)
{
return false;
}
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index fcad832..0dd0b0d 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -131,43 +131,16 @@ static inline void __putback_balloon_page(struct page
*page)
}
/* __isolate_lru_page() counterpart for a ballooned page */
-bool balloon_page_isolate(struct page *page)
+bool balloon_page_isolate(struct page *page, isolate_mode_t mode)
{
/*
-* Avoid burning cycles with pages that are yet under __free_pages(),
-* or just got freed under us.
-*
-* In case we 'win' a race for a balloon page being freed under us and
-* raise its refcount preventing __free_pages() from doing its job
-* the put_page() at the end of this block will take care of
-* release this page, thus avoiding a nasty leakage.
+* A ballooned page, by default, has PagePrivate set.
+* Prevent concurrent compaction threads from isolating
+* an already isolated balloon page by clearing it.
*/
- if (likely(get_page_unless_zero(page
[RFCv3 0/5] enable migration of driver pages
From: Gioh Kim guru...@hanmail.net Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. This patch set combines 5 patches. 1. patch 1/5: get inode from anon_inodes This patch adds new interface to create inode from anon_inodes. 2. patch 2/5: framework to isolate/migrate/putback page Add isolatepage, putbackpage into address_space_operations and wrapper function to call them 3. patch 3/5: apply the framework into balloon driver The balloon driver is applied into the framework. It gets a inode from anon_inodes and register operations in the inode. Any other drivers can register operations via inode like this to migrate it's pages. 4. patch 4/5: compaction/migration call the generic interfaces Compaction and migration pages call the generic interfaces of the framework, instead of calling balloon migration directly. 5. patch 5/5: remove direct calling of migration of driver pages Non-lru pages are migrated with lru pages by move_to_new_page(). This patch set is tested: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated My thanks to Konstantin Khlebnikov for his reviews of the v2 patch set. Most of the changes were based on his feedback. Changes since v2: - change the name of page type from migratable page into mobile page - get and lock page to isolate page - add wrapper interfaces for page-mapping-a_ops-isolate/putback - leave balloon pages marked as balloon This patch-set is based on v4.1 Gioh Kim (5): fs/anon_inodes: new interface to create new inode mm/compaction: enable mobile-page migration mm/balloon: apply mobile page migratable into balloon mm/compaction: call generic migration callbacks mm: remove direct calling of migration drivers/virtio/virtio_balloon.c| 3 ++ fs/anon_inodes.c | 6 +++ fs/proc/page.c | 3 ++ include/linux/anon_inodes.h| 1 + include/linux/balloon_compaction.h | 15 +-- include/linux/compaction.h | 76 ++ include/linux/fs.h | 2 + include/linux/page-flags.h | 19 + include/uapi/linux/kernel-page-flags.h | 1 + mm/balloon_compaction.c| 71 ++- mm/compaction.c| 8 ++-- mm/migrate.c | 24 +++ 12 files changed, 154 insertions(+), 75 deletions(-) -- 2.1.4 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv3 1/5] fs/anon_inodes: new interface to create new inode
From: Gioh Kim guru...@hanmail.net
The anon_inodes has already complete interfaces to create manage
many anonymous inodes but don't have interface to get
new inode. Other sub-modules can create anonymous inode
without creating and mounting it's own pseudo filesystem.
Signed-off-by: Gioh Kim gioh@lge.com
---
fs/anon_inodes.c| 6 ++
include/linux/anon_inodes.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/fs/anon_inodes.c b/fs/anon_inodes.c
index 80ef38c..1d51f96 100644
--- a/fs/anon_inodes.c
+++ b/fs/anon_inodes.c
@@ -162,6 +162,12 @@ err_put_unused_fd:
}
EXPORT_SYMBOL_GPL(anon_inode_getfd);
+struct inode *anon_inode_new(void)
+{
+ return alloc_anon_inode(anon_inode_mnt-mnt_sb);
+}
+EXPORT_SYMBOL_GPL(anon_inode_new);
+
static int __init anon_inode_init(void)
{
anon_inode_mnt = kern_mount(anon_inode_fs_type);
diff --git a/include/linux/anon_inodes.h b/include/linux/anon_inodes.h
index 8013a45..ddbd67f 100644
--- a/include/linux/anon_inodes.h
+++ b/include/linux/anon_inodes.h
@@ -15,6 +15,7 @@ struct file *anon_inode_getfile(const char *name,
void *priv, int flags);
int anon_inode_getfd(const char *name, const struct file_operations *fops,
void *priv, int flags);
+struct inode *anon_inode_new(void);
#endif /* _LINUX_ANON_INODES_H */
--
2.1.4
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv2 0/5] enable migration of driver pages
Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. The first this patch adds a generic isolate/migrate/putback callbacks for page address-space. The zram and GPU, and any other modules can register its own migration method. The kernel compaction can call the registered migration when it works. Therefore all page in the system can be migrated at once. The 2nd the generic migration callbacks are applied into balloon driver. My gpu driver code is not open so I apply generic migration into balloon to show how it works. I've tested it with qemu enabled by kvm like followings: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated Next kernel compaction code can call generic migration callbacks instead of balloon driver interface. Finally calling migration of balloon driver is removed. This patch-set is based on v4.1 Gioh Kim (5): mm/compaction: enable driver page migration fs/anon_inode: get a new inode mm/balloon: apply driver page migratable into balloon driver mm/compaction: compaction calls generic migration mm: remove direct calling of migration drivers/virtio/virtio_balloon.c| 4 fs/anon_inodes.c | 6 ++ fs/proc/page.c | 3 +++ include/linux/anon_inodes.h| 1 + include/linux/balloon_compaction.h | 33 + include/linux/compaction.h | 11 +++ include/linux/fs.h | 2 ++ include/linux/page-flags.h | 19 +++ include/linux/pagemap.h| 27 +++ include/uapi/linux/kernel-page-flags.h | 2 +- mm/balloon_compaction.c| 25 - mm/compaction.c| 9 + mm/migrate.c | 24 ++-- 13 files changed, 122 insertions(+), 44 deletions(-) -- 1.9.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv2 1/5] mm/compaction: enable driver page migration
Add framework to register callback functions and
check pages migratable.
There are some modes of page isolation so that isolate interface
has an arguments of page address and isolation mode.
Signed-off-by: Gioh Kim gioh@lge.com
---
include/linux/compaction.h | 11 +++
include/linux/fs.h | 2 ++
include/linux/page-flags.h | 19 +++
include/linux/pagemap.h| 27 +++
4 files changed, 59 insertions(+)
diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index aa8f61c..4e91a07 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h
@@ -1,6 +1,9 @@
#ifndef _LINUX_COMPACTION_H
#define _LINUX_COMPACTION_H
+#include linux/pagemap.h
+#include linux/mm.h
+
/* Return values for compact_zone() and try_to_compact_pages() */
/* compaction didn't start as it was deferred due to past failures */
#define COMPACT_DEFERRED 0
@@ -51,6 +54,10 @@ extern void compaction_defer_reset(struct zone *zone, int
order,
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+static inline bool driver_page_migratable(struct page *page)
+{
+ return PageMigratable(page) mapping_migratable(page-mapping);
+}
#else
static inline unsigned long try_to_compact_pages(gfp_t gfp_mask,
unsigned int order, int alloc_flags,
@@ -83,6 +90,10 @@ static inline bool compaction_deferred(struct zone *zone,
int order)
return true;
}
+static inline bool driver_page_migratable(struct page *page)
+{
+ return false
+}
#endif /* CONFIG_COMPACTION */
#if defined(CONFIG_COMPACTION) defined(CONFIG_SYSFS) defined(CONFIG_NUMA)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index a0653e5..2cc4b24 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -396,6 +396,8 @@ struct address_space_operations {
*/
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
+ bool (*isolatepage) (struct page *, isolate_mode_t);
+ void (*putbackpage) (struct page *);
int (*launder_page) (struct page *);
int (*is_partially_uptodate) (struct page *, unsigned long,
unsigned long);
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 91b7f9b..c8a66de 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -649,6 +649,25 @@ static inline void __ClearPageBalloon(struct page *page)
atomic_set(page-_mapcount, -1);
}
+#define PAGE_MIGRATABLE_MAPCOUNT_VALUE (-255)
+
+static inline int PageMigratable(struct page *page)
+{
+ return atomic_read(page-_mapcount) == PAGE_MIGRATABLE_MAPCOUNT_VALUE;
+}
+
+static inline void __SetPageMigratable(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(page-_mapcount) != -1, page);
+ atomic_set(page-_mapcount, PAGE_MIGRATABLE_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageMigratable(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageMigratable(page), page);
+ atomic_set(page-_mapcount, -1);
+}
+
/*
* If network-based swap is enabled, sl*b must keep track of whether pages
* were allocated from pfmemalloc reserves.
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 3e95fb6..a306798 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -25,8 +25,35 @@ enum mapping_flags {
AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */
+ AS_MIGRATABLE = __GFP_BITS_SHIFT + 5,
};
+static inline void mapping_set_migratable(struct address_space *mapping)
+{
+ set_bit(AS_MIGRATABLE, mapping-flags);
+}
+
+static inline void mapping_clear_migratable(struct address_space *mapping)
+{
+ clear_bit(AS_MIGRATABLE, mapping-flags);
+}
+
+static inline int __mapping_ops(struct address_space *mapping)
+{
+ /* migrating page should define all following methods */
+ return mapping-a_ops
+ mapping-a_ops-migratepage
+ mapping-a_ops-isolatepage
+ mapping-a_ops-putbackpage;
+}
+
+static inline int mapping_migratable(struct address_space *mapping)
+{
+ if (mapping __mapping_ops(mapping))
+ return test_bit(AS_MIGRATABLE, mapping-flags);
+ return !!mapping;
+}
+
static inline void mapping_set_error(struct address_space *mapping, int error)
{
if (unlikely(error)) {
--
1.9.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv2 4/5] mm/compaction: compaction calls generic migration
Compaction calls interfaces of driver page migration
instead of calling balloon migration directly.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c | 1 +
mm/compaction.c | 9 +
mm/migrate.c| 21 -
3 files changed, 18 insertions(+), 13 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index c49b553..5e5cbea 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -30,6 +30,7 @@
#include linux/balloon_compaction.h
#include linux/oom.h
#include linux/wait.h
+#include linux/anon_inodes.h
/*
* Balloon device works in 4K page units. So each page is pointed to by
diff --git a/mm/compaction.c b/mm/compaction.c
index 16e1b57..cc5ec81 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -14,7 +14,7 @@
#include linux/backing-dev.h
#include linux/sysctl.h
#include linux/sysfs.h
-#include linux/balloon_compaction.h
+#include linux/compaction.h
#include linux/page-isolation.h
#include linux/kasan.h
#include internal.h
@@ -714,12 +714,13 @@ isolate_migratepages_block(struct compact_control *cc,
unsigned long low_pfn,
/*
* Check may be lockless but that's ok as we recheck later.
-* It's possible to migrate LRU pages and balloon pages
+* It's possible to migrate LRU pages and driver pages
* Skip any other type of page
*/
if (!PageLRU(page)) {
- if (unlikely(balloon_page_movable(page))) {
- if (balloon_page_isolate(page)) {
+ if (unlikely(driver_page_migratable(page))) {
+ if (page-mapping-a_ops-isolatepage(page,
+ isolate_mode)) {
/* Successfully isolated */
goto isolate_success;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 236ee25..a0bc1e4 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -35,7 +35,7 @@
#include linux/hugetlb.h
#include linux/hugetlb_cgroup.h
#include linux/gfp.h
-#include linux/balloon_compaction.h
+#include linux/compaction.h
#include linux/mmu_notifier.h
#include asm/tlbflush.h
@@ -76,7 +76,7 @@ int migrate_prep_local(void)
* from where they were once taken off for compaction/migration.
*
* This function shall be used whenever the isolated pageset has been
- * built from lru, balloon, hugetlbfs page. See isolate_migratepages_range()
+ * built from lru, driver, hugetlbfs page. See isolate_migratepages_range()
* and isolate_huge_page().
*/
void putback_movable_pages(struct list_head *l)
@@ -92,8 +92,8 @@ void putback_movable_pages(struct list_head *l)
list_del(page-lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
- if (unlikely(isolated_balloon_page(page)))
- balloon_page_putback(page);
+ if (unlikely(driver_page_migratable(page)))
+ page-mapping-a_ops-putbackpage(page);
else
putback_lru_page(page);
}
@@ -844,15 +844,18 @@ static int __unmap_and_move(struct page *page, struct
page *newpage,
}
}
- if (unlikely(isolated_balloon_page(page))) {
+ if (unlikely(driver_page_migratable(page))) {
/*
-* A ballooned page does not need any special attention from
+* A driver page does not need any special attention from
* physical to virtual reverse mapping procedures.
* Skip any attempt to unmap PTEs or to remap swap cache,
* in order to avoid burning cycles at rmap level, and perform
* the page migration right away (proteced by page lock).
*/
- rc = balloon_page_migrate(newpage, page, mode);
+ rc = page-mapping-a_ops-migratepage(page-mapping,
+ newpage,
+ page,
+ mode);
goto out_unlock;
}
@@ -962,8 +965,8 @@ out:
if (rc != MIGRATEPAGE_SUCCESS put_new_page) {
ClearPageSwapBacked(newpage);
put_new_page(newpage, private);
- } else if (unlikely(__is_movable_balloon_page(newpage))) {
- /* drop our reference, page already in the balloon */
+ } else if (unlikely(driver_page_migratable(newpage))) {
+ /* drop our reference */
put_page(newpage);
} else
putback_lru_page(newpage);
--
1.9.1
[RFCv2 5/5] mm: remove direct calling of migration
Migration is completely generalized.
Signed-off-by: Gioh Kim gioh@lge.com
---
mm/balloon_compaction.c | 8
mm/migrate.c| 15 ---
2 files changed, 23 deletions(-)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index df72846..a7b7c79 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -206,13 +206,6 @@ int balloon_page_migrate(struct address_space *mapping,
if (!isolated_balloon_page(page))
return rc;
- /*
-* Block others from accessing the 'newpage' when we get around to
-* establishing additional references. We should be the only one
-* holding a reference to the 'newpage' at this point.
-*/
- BUG_ON(!trylock_page(newpage));
-
if (WARN_ON(!__is_movable_balloon_page(page))) {
dump_page(page, not movable balloon page);
unlock_page(newpage);
@@ -222,7 +215,6 @@ int balloon_page_migrate(struct address_space *mapping,
if (balloon balloon-migratepage)
rc = balloon-migratepage(balloon, newpage, page, mode);
- unlock_page(newpage);
return rc;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index a0bc1e4..0b52fa4 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -844,21 +844,6 @@ static int __unmap_and_move(struct page *page, struct page
*newpage,
}
}
- if (unlikely(driver_page_migratable(page))) {
- /*
-* A driver page does not need any special attention from
-* physical to virtual reverse mapping procedures.
-* Skip any attempt to unmap PTEs or to remap swap cache,
-* in order to avoid burning cycles at rmap level, and perform
-* the page migration right away (proteced by page lock).
-*/
- rc = page-mapping-a_ops-migratepage(page-mapping,
- newpage,
- page,
- mode);
- goto out_unlock;
- }
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
--
1.9.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFCv2 3/5] mm/balloon: apply driver page migratable into balloon
Apply driver page migration into balloon driver.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c| 3 +++
fs/proc/page.c | 3 +++
include/linux/balloon_compaction.h | 33 +
include/uapi/linux/kernel-page-flags.h | 2 +-
mm/balloon_compaction.c| 19 +--
5 files changed, 45 insertions(+), 15 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 82e80e0..c49b553 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -505,6 +505,9 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
vb-vb_dev_info.migratepage = virtballoon_migratepage;
+ vb-vb_dev_info.inode = anon_inode_new();
+ vb-vb_dev_info.inode-i_mapping-a_ops = balloon_aops;
+ mapping_set_migratable(vb-vb_dev_info.inode-i_mapping);
#endif
err = init_vqs(vb);
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8..2dc3673 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -146,6 +146,9 @@ u64 stable_page_flags(struct page *page)
if (PageBalloon(page))
u |= 1 KPF_BALLOON;
+ if (PageMigratable(page))
+ u |= 1 KPF_MIGRATABLE;
+
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..e8a3670 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -48,6 +48,7 @@
#include linux/migrate.h
#include linux/gfp.h
#include linux/err.h
+#include linux/fs.h
/*
* Balloon device information descriptor.
@@ -62,6 +63,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct inode *inode;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -73,24 +75,28 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
spin_lock_init(balloon-pages_lock);
INIT_LIST_HEAD(balloon-pages);
balloon-migratepage = NULL;
+ balloon-inode = NULL;
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
+extern const struct address_space_operations balloon_aops;
+extern bool balloon_page_isolate(struct page *page,
+isolate_mode_t mode);
extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
struct page *page, enum migrate_mode mode);
/*
- * __is_movable_balloon_page - helper to perform @page PageBalloon tests
+ * __is_movable_balloon_page - helper to perform @page PageMigratable tests
*/
static inline bool __is_movable_balloon_page(struct page *page)
{
- return PageBalloon(page);
+ return PageMigratable(page);
}
/*
- * balloon_page_movable - test PageBalloon to identify balloon pages
+ * balloon_page_movable - test PageMigratable to identify balloon pages
* and PagePrivate to check that the page is not
* isolated and can be moved by compaction/migration.
*
@@ -99,7 +105,7 @@ static inline bool __is_movable_balloon_page(struct page
*page)
*/
static inline bool balloon_page_movable(struct page *page)
{
- return PageBalloon(page) PagePrivate(page);
+ return PageMigratable(page) PagePrivate(page);
}
/*
@@ -108,7 +114,7 @@ static inline bool balloon_page_movable(struct page *page)
*/
static inline bool isolated_balloon_page(struct page *page)
{
- return PageBalloon(page);
+ return PageMigratable(page);
}
/*
@@ -123,7 +129,8 @@ static inline bool isolated_balloon_page(struct page *page)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
- __SetPageBalloon(page);
+ page-mapping = balloon-inode-i_mapping;
+ __SetPageMigratable(page);
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
list_add(page-lru, balloon-pages);
@@ -139,7 +146,8 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
*/
static inline void balloon_page_delete(struct page *page)
{
- __ClearPageBalloon(page);
+ page-mapping = NULL;
+ __ClearPageMigratable(page);
set_page_private(page, 0);
if (PagePrivate(page)) {
ClearPagePrivate(page);
@@ -166,13 +174,13
[RFCv2 2/5] fs/anon_inodes: get a new inode
A inode is necessary for some drivers that needs special address_space
and address_space_operation for page migration. Each drivers can create
inode with the anon_inodefs.
Signed-off-by: Gioh Kim gioh@lge.com
---
fs/anon_inodes.c| 6 ++
include/linux/anon_inodes.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/fs/anon_inodes.c b/fs/anon_inodes.c
index 80ef38c..1d51f96 100644
--- a/fs/anon_inodes.c
+++ b/fs/anon_inodes.c
@@ -162,6 +162,12 @@ err_put_unused_fd:
}
EXPORT_SYMBOL_GPL(anon_inode_getfd);
+struct inode *anon_inode_new(void)
+{
+ return alloc_anon_inode(anon_inode_mnt-mnt_sb);
+}
+EXPORT_SYMBOL_GPL(anon_inode_new);
+
static int __init anon_inode_init(void)
{
anon_inode_mnt = kern_mount(anon_inode_fs_type);
diff --git a/include/linux/anon_inodes.h b/include/linux/anon_inodes.h
index 8013a45..ddbd67f 100644
--- a/include/linux/anon_inodes.h
+++ b/include/linux/anon_inodes.h
@@ -15,6 +15,7 @@ struct file *anon_inode_getfile(const char *name,
void *priv, int flags);
int anon_inode_getfd(const char *name, const struct file_operations *fops,
void *priv, int flags);
+struct inode *anon_inode_new(void);
#endif /* _LINUX_ANON_INODES_H */
--
1.9.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC 0/4] enable migration of non-LRU pages
2015-06-10 오전 9:08에 Minchan Kim 이(가) 쓴 글: Hello Gioh, On Tue, Jun 02, 2015 at 04:27:40PM +0900, Gioh Kim wrote: Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. The first this patch adds a generic isolate/migrate/putback callbacks for page address-space. The zram and GPU, and any other modules can register its own migration method. The kernel compaction can call the registered migration when it works. Therefore all page in the system can be migrated at once. The 2nd the generic migration callbacks are applied into balloon driver. My gpu driver code is not open so I apply generic migration into balloon to show how it works. I've tested it with qemu enabled by kvm like followings: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated Next kernel compaction code can call generic migration callbacks instead of balloon driver interface. Finally calling migration of balloon driver is removed. I didn't hava a time to review but it surely will help using zram with CMA as well as fragmentation of the system memory via making zram objects movable. I know you are busy. I hope you make time for review. If it lands on mainline, I will work for zram object migration. Thanks! ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC 0/4] enable migration of non-LRU pages
On Tue, Jun 02, 2015 at 04:27:40PM +0900, Gioh Kim wrote: Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. The first this patch adds a generic isolate/migrate/putback callbacks for page address-space. The zram and GPU, and any other modules can register its own migration method. The kernel compaction can call the registered migration when it works. Therefore all page in the system can be migrated at once. The 2nd the generic migration callbacks are applied into balloon driver. My gpu driver code is not open so I apply generic migration into balloon to show how it works. I've tested it with qemu enabled by kvm like followings: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated Next kernel compaction code can call generic migration callbacks instead of balloon driver interface. Finally calling migration of balloon driver is removed. In a glance, ss Konstantin pointed out this set, while it twists chunks around, brings back code we got rid of a while ago because it was messy and racy. Yes, your point is right. I'll take a closer look into your work next week, but for now, I'd say we should not follow this patch of reintroducing long-dead code. BUT as I replied to Konstantin, the code for balloon driver is to show how the generic callbacks can be applied. My point is there are some pages to be migrated which are not LRU pages, but there is no interface for them to migrate. For example gpu driver has many, not mapped to kernel space, pages. Those pages can be migrated when GPU is not working, screen is not refreshed. And zram pages can be migrated also. I'm very sorry that I'm not familiar to balloon driver. If you give me some hints, I might be able to refine code of patch 2~3 in the next spin. Thank you for the feedback. Cheers! -- Rafael Gioh Kim (4): mm/compaction: enable driver page migration mm/balloon: apply migratable-page into balloon driver mm/compaction: apply migratable-page into compaction mm: remove direct migration of migratable-page drivers/virtio/virtio_balloon.c| 2 + fs/proc/page.c | 4 +- include/linux/balloon_compaction.h | 42 +++-- include/linux/compaction.h | 13 +++ include/linux/fs.h | 2 + include/linux/mm.h | 14 +++ include/linux/pagemap.h| 27 ++ include/uapi/linux/kernel-page-flags.h | 2 +- mm/balloon_compaction.c| 67 +- mm/compaction.c| 9 +++-- mm/migrate.c | 25 - 11 files changed, 154 insertions(+), 53 deletions(-) -- 1.9.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majord...@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: a href=mailto:d...@kvack.org; em...@kvack.org /a ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC 2/4] mm/balloon: apply driver page migratable into balloon driver
Apply driver page migration into balloon driver.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c| 2 ++
fs/proc/page.c | 4 +--
include/linux/balloon_compaction.h | 42 ---
include/linux/mm.h | 19 ---
include/uapi/linux/kernel-page-flags.h | 2 +-
mm/balloon_compaction.c| 61 --
6 files changed, 94 insertions(+), 36 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 6a356e3..cdd0038 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -496,6 +496,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
vb-vb_dev_info.migratepage = virtballoon_migratepage;
+ vb-vb_dev_info.mapping = balloon_mapping_alloc(vb-vb_dev_info,
+ balloon_aops);
#endif
err = init_vqs(vb);
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8..e741307 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -143,8 +143,8 @@ u64 stable_page_flags(struct page *page)
if (PageBuddy(page))
u |= 1 KPF_BUDDY;
- if (PageBalloon(page))
- u |= 1 KPF_BALLOON;
+ if (PageMigratable(page))
+ u |= 1 KPF_MIGRATABLE;
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..0989e96 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -48,6 +48,7 @@
#include linux/migrate.h
#include linux/gfp.h
#include linux/err.h
+#include linux/fs.h
/*
* Balloon device information descriptor.
@@ -62,6 +63,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct address_space *mapping;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -73,24 +75,37 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
spin_lock_init(balloon-pages_lock);
INIT_LIST_HEAD(balloon-pages);
balloon-migratepage = NULL;
+ balloon-mapping = NULL;
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
+extern const struct address_space_operations balloon_aops;
+extern bool balloon_page_isolate(struct page *page,
+isolate_mode_t mode);
extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
struct page *page, enum migrate_mode mode);
+extern struct address_space
+*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
+ const struct address_space_operations *a_ops);
+
+static inline void balloon_mapping_free(struct address_space *balloon_mapping)
+{
+ kfree(balloon_mapping);
+}
+
/*
- * __is_movable_balloon_page - helper to perform @page PageBalloon tests
+ * __is_movable_balloon_page - helper to perform @page PageMigratable tests
*/
static inline bool __is_movable_balloon_page(struct page *page)
{
- return PageBalloon(page);
+ return PageMigratable(page);
}
/*
- * balloon_page_movable - test PageBalloon to identify balloon pages
+ * balloon_page_movable - test PageMigratable to identify balloon pages
* and PagePrivate to check that the page is not
* isolated and can be moved by compaction/migration.
*
@@ -99,7 +114,7 @@ static inline bool __is_movable_balloon_page(struct page
*page)
*/
static inline bool balloon_page_movable(struct page *page)
{
- return PageBalloon(page) PagePrivate(page);
+ return PageMigratable(page) PagePrivate(page);
}
/*
@@ -108,7 +123,7 @@ static inline bool balloon_page_movable(struct page *page)
*/
static inline bool isolated_balloon_page(struct page *page)
{
- return PageBalloon(page);
+ return PageMigratable(page);
}
/*
@@ -123,7 +138,8 @@ static inline bool isolated_balloon_page(struct page *page)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
- __SetPageBalloon(page);
+ page-mapping = balloon-mapping;
+ __SetPageMigratable(page);
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
list_add(page-lru, balloon-pages);
@@ -139,7 +155,8 @@ static inline void balloon_page_insert(struct
[RFC 3/4] mm/compaction: compaction calls generic migration
Compaction calls interfaces of driver page migration
instead of calling balloon migration directly.
Signed-off-by: Gioh Kim gioh@lge.com
---
mm/compaction.c | 9 +
mm/migrate.c| 22 +-
2 files changed, 18 insertions(+), 13 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 8c0d945..ca666e2 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -14,7 +14,7 @@
#include linux/backing-dev.h
#include linux/sysctl.h
#include linux/sysfs.h
-#include linux/balloon_compaction.h
+#include linux/compaction.h
#include linux/page-isolation.h
#include linux/kasan.h
#include internal.h
@@ -736,12 +736,13 @@ isolate_migratepages_block(struct compact_control *cc,
unsigned long low_pfn,
/*
* Check may be lockless but that's ok as we recheck later.
-* It's possible to migrate LRU pages and balloon pages
+* It's possible to migrate LRU pages and migratable-pages
* Skip any other type of page
*/
if (!PageLRU(page)) {
- if (unlikely(balloon_page_movable(page))) {
- if (balloon_page_isolate(page)) {
+ if (unlikely(driver_page_migratable(page))) {
+ if (page-mapping-a_ops-isolatepage(page,
+ isolate_mode)) {
/* Successfully isolated */
goto isolate_success;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 85e0426..649b1cd 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -35,7 +35,7 @@
#include linux/hugetlb.h
#include linux/hugetlb_cgroup.h
#include linux/gfp.h
-#include linux/balloon_compaction.h
+#include linux/compaction.h
#include linux/mmu_notifier.h
#include asm/tlbflush.h
@@ -76,7 +76,8 @@ int migrate_prep_local(void)
* from where they were once taken off for compaction/migration.
*
* This function shall be used whenever the isolated pageset has been
- * built from lru, balloon, hugetlbfs page. See isolate_migratepages_range()
+ * built from lru, migratable-page, hugetlbfs page.
+ * See isolate_migratepages_range()
* and isolate_huge_page().
*/
void putback_movable_pages(struct list_head *l)
@@ -92,8 +93,8 @@ void putback_movable_pages(struct list_head *l)
list_del(page-lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
- if (unlikely(isolated_balloon_page(page)))
- balloon_page_putback(page);
+ if (unlikely(driver_page_migratable(page)))
+ page-mapping-a_ops-putbackpage(page);
else
putback_lru_page(page);
}
@@ -843,15 +844,18 @@ static int __unmap_and_move(struct page *page, struct
page *newpage,
}
}
- if (unlikely(isolated_balloon_page(page))) {
+ if (unlikely(driver_page_migratable(page))) {
/*
-* A ballooned page does not need any special attention from
+* A migratable-page does not need any special attention from
* physical to virtual reverse mapping procedures.
* Skip any attempt to unmap PTEs or to remap swap cache,
* in order to avoid burning cycles at rmap level, and perform
* the page migration right away (proteced by page lock).
*/
- rc = balloon_page_migrate(newpage, page, mode);
+ rc = page-mapping-a_ops-migratepage(page-mapping,
+ newpage,
+ page,
+ mode);
goto out_unlock;
}
@@ -948,8 +952,8 @@ out:
if (rc != MIGRATEPAGE_SUCCESS put_new_page) {
ClearPageSwapBacked(newpage);
put_new_page(newpage, private);
- } else if (unlikely(__is_movable_balloon_page(newpage))) {
- /* drop our reference, page already in the balloon */
+ } else if (unlikely(driver_page_migratable(newpage))) {
+ /* drop our reference */
put_page(newpage);
} else
putback_lru_page(newpage);
--
1.9.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC 1/4] mm/compaction: enable driver page migration
Add framework to register callback functions and
check migratable pages.
There are some modes to isolate page so that isolate interface
has arguments of page address and isolation mode.
Signed-off-by: Gioh Kim gioh@lge.com
---
include/linux/compaction.h | 13 +
include/linux/fs.h | 2 ++
include/linux/mm.h | 19 +++
include/linux/pagemap.h| 27 +++
4 files changed, 61 insertions(+)
diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index a014559..1acfa21 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h
@@ -1,6 +1,9 @@
#ifndef _LINUX_COMPACTION_H
#define _LINUX_COMPACTION_H
+#include linux/pagemap.h
+#include linux/mm.h
+
/* Return values for compact_zone() and try_to_compact_pages() */
/* compaction didn't start as it was deferred due to past failures */
#define COMPACT_DEFERRED 0
@@ -50,6 +53,11 @@ extern void compaction_defer_reset(struct zone *zone, int
order,
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+static inline bool driver_page_migratable(struct page *page)
+{
+ return PageMigratable(page) mapping_migratable(page-mapping);
+}
+
#else
static inline unsigned long try_to_compact_pages(gfp_t gfp_mask,
unsigned int order, int alloc_flags,
@@ -82,6 +90,11 @@ static inline bool compaction_deferred(struct zone *zone,
int order)
return true;
}
+static inline bool driver_page_migratable(struct page *page)
+{
+ return false
+}
+
#endif /* CONFIG_COMPACTION */
#if defined(CONFIG_COMPACTION) defined(CONFIG_SYSFS) defined(CONFIG_NUMA)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 52cc449..bdfcadf 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -368,6 +368,8 @@ struct address_space_operations {
*/
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
+ bool (*isolatepage) (struct page *, isolate_mode_t);
+ void (*putbackpage) (struct page *);
int (*launder_page) (struct page *);
int (*is_partially_uptodate) (struct page *, unsigned long,
unsigned long);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 47a9392..422c484 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -618,6 +618,25 @@ static inline void __ClearPageBalloon(struct page *page)
atomic_set(page-_mapcount, -1);
}
+#define PAGE_MIGRATABLE_MAPCOUNT_VALUE (-256)
+
+static inline int PageMigratable(struct page *page)
+{
+ return atomic_read(page-_mapcount) == PAGE_MIGRATABLE_MAPCOUNT_VALUE;
+}
+
+static inline void __SetPageMigratable(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(page-_mapcount) != -1, page);
+ atomic_set(page-_mapcount, PAGE_MIGRATABLE_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageMigratable(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageMigratable(page), page);
+ atomic_set(page-_mapcount, -1);
+}
+
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 4b3736f..e924dfe 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -25,8 +25,35 @@ enum mapping_flags {
AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */
+ AS_MIGRATABLE = __GFP_BITS_SHIFT + 5,
};
+static inline void mapping_set_migratable(struct address_space *mapping)
+{
+ set_bit(AS_MIGRATABLE, mapping-flags);
+}
+
+static inline void mapping_clear_migratable(struct address_space *mapping)
+{
+ clear_bit(AS_MIGRATABLE, mapping-flags);
+}
+
+static inline int __mapping_ops(struct address_space *mapping)
+{
+ /* migrating page should define all following methods */
+ return mapping-a_ops
+ mapping-a_ops-migratepage
+ mapping-a_ops-isolatepage
+ mapping-a_ops-putbackpage;
+}
+
+static inline int mapping_migratable(struct address_space *mapping)
+{
+ if (mapping __mapping_ops(mapping))
+ return test_bit(AS_MIGRATABLE, mapping-flags);
+ return !!mapping;
+}
+
static inline void mapping_set_error(struct address_space *mapping, int error)
{
if (unlikely(error)) {
--
1.9.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC 4/4] mm: remove direct calling of migration
Migration is completely generalized.
Signed-off-by: Gioh Kim gioh@lge.com
---
mm/balloon_compaction.c | 8
mm/migrate.c| 15 ---
2 files changed, 23 deletions(-)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index f98a500..d29270aa 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -206,13 +206,6 @@ int balloon_page_migrate(struct address_space *mapping,
if (!isolated_balloon_page(page))
return rc;
- /*
-* Block others from accessing the 'newpage' when we get around to
-* establishing additional references. We should be the only one
-* holding a reference to the 'newpage' at this point.
-*/
- BUG_ON(!trylock_page(newpage));
-
if (WARN_ON(!__is_movable_balloon_page(page))) {
dump_page(page, not movable balloon page);
unlock_page(newpage);
@@ -222,7 +215,6 @@ int balloon_page_migrate(struct address_space *mapping,
if (balloon balloon-migratepage)
rc = balloon-migratepage(balloon, newpage, page, mode);
- unlock_page(newpage);
return rc;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 649b1cd..ca47b3e 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -844,21 +844,6 @@ static int __unmap_and_move(struct page *page, struct page
*newpage,
}
}
- if (unlikely(driver_page_migratable(page))) {
- /*
-* A migratable-page does not need any special attention from
-* physical to virtual reverse mapping procedures.
-* Skip any attempt to unmap PTEs or to remap swap cache,
-* in order to avoid burning cycles at rmap level, and perform
-* the page migration right away (proteced by page lock).
-*/
- rc = page-mapping-a_ops-migratepage(page-mapping,
- newpage,
- page,
- mode);
- goto out_unlock;
- }
-
/*
* Corner case handling:
* 1. When a new swap-cache page is read into, it is added to the LRU
--
1.9.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[RFC 0/4] enable migration of non-LRU pages
Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It has memory size 512MB ~ 1024MB. 30% ~ 40% memory is consumed for graphic processing and 20~30 memory is reserved for zram. I found that many pages of GPU driver and zram are non-movable pages. So I reported Minchan Kim, the maintainer of zram, and he made the internal compaction logic of zram. And I made the internal compaction of GPU driver. They reduced some fragmentation but they are not enough effective. They are activated by its own interface, /sys, so they are not cooperative with kernel compaction. If there is too much fragmentation and kernel starts to compaction, zram and GPU driver cannot work with the kernel compaction. The first this patch adds a generic isolate/migrate/putback callbacks for page address-space. The zram and GPU, and any other modules can register its own migration method. The kernel compaction can call the registered migration when it works. Therefore all page in the system can be migrated at once. The 2nd the generic migration callbacks are applied into balloon driver. My gpu driver code is not open so I apply generic migration into balloon to show how it works. I've tested it with qemu enabled by kvm like followings: - turn on Ubuntu 14.04 with 1G memory on qemu. - do kernel building - after several seconds check more than 512MB is used with free command - command balloon 512 in qemu monitor - check hundreds MB of pages are migrated Next kernel compaction code can call generic migration callbacks instead of balloon driver interface. Finally calling migration of balloon driver is removed. Gioh Kim (4): mm/compaction: enable driver page migration mm/balloon: apply migratable-page into balloon driver mm/compaction: apply migratable-page into compaction mm: remove direct migration of migratable-page drivers/virtio/virtio_balloon.c| 2 + fs/proc/page.c | 4 +- include/linux/balloon_compaction.h | 42 +++-- include/linux/compaction.h | 13 +++ include/linux/fs.h | 2 + include/linux/mm.h | 14 +++ include/linux/pagemap.h| 27 ++ include/uapi/linux/kernel-page-flags.h | 2 +- mm/balloon_compaction.c| 67 +- mm/compaction.c| 9 +++-- mm/migrate.c | 25 - 11 files changed, 154 insertions(+), 53 deletions(-) -- 1.9.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC 2/4] mm/balloon: apply driver page migratable into balloon driver
On Tue, Jun 2, 2015 at 10:27 AM, Gioh Kim gioh@lge.com wrote:
Apply driver page migration into balloon driver.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c| 2 ++
fs/proc/page.c | 4 +--
include/linux/balloon_compaction.h | 42 ---
include/linux/mm.h | 19 ---
include/uapi/linux/kernel-page-flags.h | 2 +-
mm/balloon_compaction.c| 61 --
6 files changed, 94 insertions(+), 36 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 6a356e3..cdd0038 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -496,6 +496,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
vb-vb_dev_info.migratepage = virtballoon_migratepage;
+ vb-vb_dev_info.mapping = balloon_mapping_alloc(vb-vb_dev_info,
+ balloon_aops);
#endif
err = init_vqs(vb);
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8..e741307 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -143,8 +143,8 @@ u64 stable_page_flags(struct page *page)
if (PageBuddy(page))
u |= 1 KPF_BUDDY;
- if (PageBalloon(page))
- u |= 1 KPF_BALLOON;
+ if (PageMigratable(page))
+ u |= 1 KPF_MIGRATABLE;
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..0989e96 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -48,6 +48,7 @@
#include linux/migrate.h
#include linux/gfp.h
#include linux/err.h
+#include linux/fs.h
/*
* Balloon device information descriptor.
@@ -62,6 +63,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct address_space *mapping;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -73,24 +75,37 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
spin_lock_init(balloon-pages_lock);
INIT_LIST_HEAD(balloon-pages);
balloon-migratepage = NULL;
+ balloon-mapping = NULL;
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
+extern const struct address_space_operations balloon_aops;
+extern bool balloon_page_isolate(struct page *page,
+isolate_mode_t mode);
extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
struct page *page, enum migrate_mode mode);
+extern struct address_space
+*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
+ const struct address_space_operations *a_ops);
+
+static inline void balloon_mapping_free(struct address_space *balloon_mapping)
+{
+ kfree(balloon_mapping);
+}
+
/*
- * __is_movable_balloon_page - helper to perform @page PageBalloon tests
+ * __is_movable_balloon_page - helper to perform @page PageMigratable tests
*/
static inline bool __is_movable_balloon_page(struct page *page)
{
- return PageBalloon(page);
+ return PageMigratable(page);
}
/*
- * balloon_page_movable - test PageBalloon to identify balloon pages
+ * balloon_page_movable - test PageMigratable to identify balloon pages
* and PagePrivate to check that the page is not
* isolated and can be moved by compaction/migration.
*
@@ -99,7 +114,7 @@ static inline bool __is_movable_balloon_page(struct page
*page)
*/
static inline bool balloon_page_movable(struct page *page)
{
- return PageBalloon(page) PagePrivate(page);
+ return PageMigratable(page) PagePrivate(page);
}
/*
@@ -108,7 +123,7 @@ static inline bool balloon_page_movable(struct page *page)
*/
static inline bool isolated_balloon_page(struct page *page)
{
- return PageBalloon(page);
+ return PageMigratable(page);
}
/*
@@ -123,7 +138,8 @@ static inline bool isolated_balloon_page(struct page *page)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
- __SetPageBalloon(page);
+ page-mapping = balloon-mapping;
+ __SetPageMigratable(page);
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon
Re: [PATCH] add generic callbacks into compaction
2015-04-07 오전 9:59에 Minchan Kim 이(가) 쓴 글:
Hello Gioh,
I wanted to have such feature for zsmalloc.
Thanks for the work.
On Wed, Apr 01, 2015 at 08:11:30AM +0900, Gioh Kim wrote:
I sent a patch about page allocation for less fragmentation.
http://permalink.gmane.org/gmane.linux.kernel.mm/130599
It proposes a page allocator allocates pages in the same pageblock
for the drivers to move their unmovable pages. Some drivers which comsumes many
pages
and increases system fragmentation use the allocator to move their pages to
decrease fragmentation.
I think I can try another approach.
There is a compaction code for balloon pages.
But the compaction code cannot migrate pages of other drivers.
If there is a generic migration framework applicable to any drivers,
drivers can register their migration functions.
And the compaction can migrate movable pages and also driver's pages.
I'm not familiar with virtualization so I couldn't test this patch yet.
But if mm developers agree with this approach, I will complete this patch.
I would do appreciate any feedback.
Could you separate introducing migrate core patchset and balloon patchset for
using the feature?
Sure.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c|2 ++
include/linux/balloon_compaction.h | 23 +---
include/linux/fs.h |3 ++
include/linux/pagemap.h| 26 ++
mm/balloon_compaction.c| 68 ++--
mm/compaction.c|7 ++--
mm/migrate.c | 24 ++---
7 files changed, 129 insertions(+), 24 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 0413157..cd9b8e4 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -486,6 +486,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
+ vb-vb_dev_info.mapping = balloon_mapping_alloc(vb-vb_dev_info,
+ balloon_aops);
vb-vb_dev_info.migratepage = virtballoon_migratepage;
#endif
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..0af32b3 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -62,6 +62,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct address_space *mapping;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -76,10 +77,22 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
-extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode);
+extern const struct address_space_operations balloon_aops;
+extern int balloon_page_isolate(struct page *page);
+extern int balloon_page_putback(struct page *page);
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
+ struct page *page,
+ enum migrate_mode mode);
+
+extern struct address_space
+*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
+ const struct address_space_operations *a_ops);
+
+static inline void balloon_mapping_free(struct address_space *balloon_mapping)
+{
+ kfree(balloon_mapping);
+}
/*
* __is_movable_balloon_page - helper to perform @page PageBalloon tests
@@ -123,6 +136,7 @@ static inline bool isolated_balloon_page(struct page *page)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
+ page-mapping = balloon-mapping;
__SetPageBalloon(page);
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
@@ -139,6 +153,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
*/
static inline void balloon_page_delete(struct page *page)
{
+ page-mapping = NULL;
__ClearPageBalloon(page);
set_page_private(page, 0);
if (PagePrivate(page)) {
diff --git a/include/linux/fs.h b/include/linux/fs.h
index b4d71b5..de463b9 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -368,6 +368,9 @@ struct address_space_operations {
*/
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
+ int (*isolatepage)(struct page
[PATCH] add generic callbacks into compaction
I sent a patch about page allocation for less fragmentation.
http://permalink.gmane.org/gmane.linux.kernel.mm/130599
It proposes a page allocator allocates pages in the same pageblock
for the drivers to move their unmovable pages. Some drivers which comsumes many
pages
and increases system fragmentation use the allocator to move their pages to
decrease fragmentation.
I think I can try another approach.
There is a compaction code for balloon pages.
But the compaction code cannot migrate pages of other drivers.
If there is a generic migration framework applicable to any drivers,
drivers can register their migration functions.
And the compaction can migrate movable pages and also driver's pages.
I'm not familiar with virtualization so I couldn't test this patch yet.
But if mm developers agree with this approach, I will complete this patch.
I would do appreciate any feedback.
Signed-off-by: Gioh Kim gioh@lge.com
---
drivers/virtio/virtio_balloon.c|2 ++
include/linux/balloon_compaction.h | 23 +---
include/linux/fs.h |3 ++
include/linux/pagemap.h| 26 ++
mm/balloon_compaction.c| 68 ++--
mm/compaction.c|7 ++--
mm/migrate.c | 24 ++---
7 files changed, 129 insertions(+), 24 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 0413157..cd9b8e4 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -486,6 +486,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
balloon_devinfo_init(vb-vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
+ vb-vb_dev_info.mapping = balloon_mapping_alloc(vb-vb_dev_info,
+ balloon_aops);
vb-vb_dev_info.migratepage = virtballoon_migratepage;
#endif
diff --git a/include/linux/balloon_compaction.h
b/include/linux/balloon_compaction.h
index 9b0a15d..0af32b3 100644
--- a/include/linux/balloon_compaction.h
+++ b/include/linux/balloon_compaction.h
@@ -62,6 +62,7 @@ struct balloon_dev_info {
struct list_head pages; /* Pages enqueued handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
+ struct address_space *mapping;
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
@@ -76,10 +77,22 @@ static inline void balloon_devinfo_init(struct
balloon_dev_info *balloon)
}
#ifdef CONFIG_BALLOON_COMPACTION
-extern bool balloon_page_isolate(struct page *page);
-extern void balloon_page_putback(struct page *page);
-extern int balloon_page_migrate(struct page *newpage,
- struct page *page, enum migrate_mode mode);
+extern const struct address_space_operations balloon_aops;
+extern int balloon_page_isolate(struct page *page);
+extern int balloon_page_putback(struct page *page);
+extern int balloon_page_migrate(struct address_space *mapping,
+ struct page *newpage,
+ struct page *page,
+ enum migrate_mode mode);
+
+extern struct address_space
+*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
+ const struct address_space_operations *a_ops);
+
+static inline void balloon_mapping_free(struct address_space *balloon_mapping)
+{
+ kfree(balloon_mapping);
+}
/*
* __is_movable_balloon_page - helper to perform @page PageBalloon tests
@@ -123,6 +136,7 @@ static inline bool isolated_balloon_page(struct page *page)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
+ page-mapping = balloon-mapping;
__SetPageBalloon(page);
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
@@ -139,6 +153,7 @@ static inline void balloon_page_insert(struct
balloon_dev_info *balloon,
*/
static inline void balloon_page_delete(struct page *page)
{
+ page-mapping = NULL;
__ClearPageBalloon(page);
set_page_private(page, 0);
if (PagePrivate(page)) {
diff --git a/include/linux/fs.h b/include/linux/fs.h
index b4d71b5..de463b9 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -368,6 +368,9 @@ struct address_space_operations {
*/
int (*migratepage) (struct address_space *,
struct page *, struct page *, enum migrate_mode);
+ int (*isolatepage)(struct page *);
+ int (*putbackpage)(struct page *);
+
int (*launder_page) (struct page *);
int (*is_partially_uptodate) (struct page *, unsigned long,
unsigned long);
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 4b3736f..715b5b2 100644
