Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Tue, Sep 23, 2014 at 12:22 AM, Rafael Aquini wrote:
> On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
>> On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini wrote:
>> > On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
>> >> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
>> >> wrote:
>> >> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
>> >> > wrote:
>> >> >
>> >> >> >
>> >> >> > So I'm going to send "fix for
>> >> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> >> >> > separately, but it has no changelog at all.
>> >> >>
>> >> >> Probably it would be better if you drop everything except actually
>> >> >> fixes and stresstest. This is gone too far, now balloon won't compile
>> >> >> in the middle of patchset. Just tell me and I'll redo the rest.
>> >> >
>> >> > I think it's best if I drop everything:
>> >> >
>> >> > mm-balloon_compaction-ignore-anonymous-pages.patch
>> >> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
>> >> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
>> >> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
>> >> > mm-introduce-common-page-state-for-ballooned-memory.patch
>> >> > mm-balloon_compaction-use-common-page-ballooning.patch
>> >> > mm-balloon_compaction-general-cleanup.patch
>> >> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>> >> >
>> >> > Please go through it and send out a new version?
>> >> >
>> >> >
>> >>
>> >> I've found yet another bug in this code. It seems here is a nest.
>> >> balloon_page_dequeue can race with balloon_page_isolate:
>> >> balloon_page_isolate can remove page from list between
>> >> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
>> >> balloon_page_dequeue runs under mutex_lock(>balloon_lock);
>> >> both of them lock page using trylock_page so race is tight but it is
>> >> not impossible.
>> > Plausible to happen if stress testing compaction simultaneously with
>> > freezing/unloading the balloon driver. As you noted, it's quite tight
>> > despite not impossible. Nice catch.
>> >
>> >
>> >> Probably it's really easier to rewrite it than to fix bugs one by one =/
>> > I'm not against a rewrite, but I don't think that rewriting the code to
>> > get rid
>> > of such bugs changes the fact we still have to address them in the actual
>> > placed
>> > code as we go on finding them. That's why I thought your inital changeset
>> > fine,
>> > with patches for stable going first and code overhaul for next following
>> > them up.
>> >
>> > For this race you spotted, I think a simple change like the following
>> > might be enough (not-tested)
>>
>> This locking scheme is too fragile and uncommon.
>>
>
> page_lock and refcounting was what I had at my disposal to sort these
> races out since I haven't thoutgh on a special page->_mapcount when
> designing this feature. It's the way other page races are sorted out.
> Not arguing it's not fragile, but it's the way code is layed out since
> some time, so we must check the feasibility of a total overhaul for
> stable branches.
>
>
>> What about this:
>>
>> * special page->_mapcount marks ballooned pages
>> * page->private points to balloon (directly, without intermediate mapping)
>> * flag PagePrivate means page currently in balloon page list (i.e. not
>> isolated, like PageLRU for normal pages)
>> * lock_page protects all of them
>>
>> balloon_page_dequeue() will delete page from balloon list only if it's
>> not isolated, also it always clears page->private and balloon mark.
>> put-back rechecks mark after locking the page and releases it as
>> normal page if mark is gone.
>>
>
> I have already agreed with you here, since the changes above are mostly from
> your original overhaul proposal. It's a much better approach for that
> balloon code, no doubts. Thanks for doing it. Only thing we need to take
> care here is about its requirement on changing the semantics for those
> interfaces might turn the changes unfeasible for old stable branches. If we
> can ignore this mentioned fact entirely, I don't see why not going with
> your idea all branches across, otherwise I think we should overhaul the
> code for -next, and send pontual fixes for stable.
>
Ok, I'll try to implement this approach in suitable for stable branches way.
Keeping several different versions is overkill for this code.
>
> Cheers,
> -- Rafael
>
>> >
>> > diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
>> > index 6e45a50..fd3a497 100644
>> > --- a/mm/balloon_compaction.c
>> > +++ b/mm/balloon_compaction.c
>> > @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
>> > balloon_dev_info *b_dev_info)
>> > * to be released by the balloon driver.
>> > */
>> > if (trylock_page(page)) {
>> > + /*
>> > +* Skip dequeue attempt for this page to a
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
> On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini wrote:
> > On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
> >> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
> >> wrote:
> >> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
> >> > wrote:
> >> >
> >> >> >
> >> >> > So I'm going to send "fix for
> >> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> >> >> > separately, but it has no changelog at all.
> >> >>
> >> >> Probably it would be better if you drop everything except actually
> >> >> fixes and stresstest. This is gone too far, now balloon won't compile
> >> >> in the middle of patchset. Just tell me and I'll redo the rest.
> >> >
> >> > I think it's best if I drop everything:
> >> >
> >> > mm-balloon_compaction-ignore-anonymous-pages.patch
> >> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> >> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> >> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> >> > mm-introduce-common-page-state-for-ballooned-memory.patch
> >> > mm-balloon_compaction-use-common-page-ballooning.patch
> >> > mm-balloon_compaction-general-cleanup.patch
> >> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
> >> >
> >> > Please go through it and send out a new version?
> >> >
> >> >
> >>
> >> I've found yet another bug in this code. It seems here is a nest.
> >> balloon_page_dequeue can race with balloon_page_isolate:
> >> balloon_page_isolate can remove page from list between
> >> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
> >> balloon_page_dequeue runs under mutex_lock(>balloon_lock);
> >> both of them lock page using trylock_page so race is tight but it is
> >> not impossible.
> > Plausible to happen if stress testing compaction simultaneously with
> > freezing/unloading the balloon driver. As you noted, it's quite tight
> > despite not impossible. Nice catch.
> >
> >
> >> Probably it's really easier to rewrite it than to fix bugs one by one =/
> > I'm not against a rewrite, but I don't think that rewriting the code to get
> > rid
> > of such bugs changes the fact we still have to address them in the actual
> > placed
> > code as we go on finding them. That's why I thought your inital changeset
> > fine,
> > with patches for stable going first and code overhaul for next following
> > them up.
> >
> > For this race you spotted, I think a simple change like the following
> > might be enough (not-tested)
>
> This locking scheme is too fragile and uncommon.
>
page_lock and refcounting was what I had at my disposal to sort these
races out since I haven't thoutgh on a special page->_mapcount when
designing this feature. It's the way other page races are sorted out.
Not arguing it's not fragile, but it's the way code is layed out since
some time, so we must check the feasibility of a total overhaul for
stable branches.
> What about this:
>
> * special page->_mapcount marks ballooned pages
> * page->private points to balloon (directly, without intermediate mapping)
> * flag PagePrivate means page currently in balloon page list (i.e. not
> isolated, like PageLRU for normal pages)
> * lock_page protects all of them
>
> balloon_page_dequeue() will delete page from balloon list only if it's
> not isolated, also it always clears page->private and balloon mark.
> put-back rechecks mark after locking the page and releases it as
> normal page if mark is gone.
>
I have already agreed with you here, since the changes above are mostly from
your original overhaul proposal. It's a much better approach for that
balloon code, no doubts. Thanks for doing it. Only thing we need to take
care here is about its requirement on changing the semantics for those
interfaces might turn the changes unfeasible for old stable branches. If we
can ignore this mentioned fact entirely, I don't see why not going with
your idea all branches across, otherwise I think we should overhaul the
code for -next, and send pontual fixes for stable.
Cheers,
-- Rafael
> >
> > diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> > index 6e45a50..fd3a497 100644
> > --- a/mm/balloon_compaction.c
> > +++ b/mm/balloon_compaction.c
> > @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
> > balloon_dev_info *b_dev_info)
> > * to be released by the balloon driver.
> > */
> > if (trylock_page(page)) {
> > + /*
> > +* Skip dequeue attempt for this page to a later
> > round
> > +* if balloon_page_isolate() has sucessfully
> > isolated
> > +* it just before we got the page lock here.
> > +*/
> > + if (page_count(page) != 1) {
> > +
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini wrote:
> On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
>> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
>> wrote:
>> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
>> > wrote:
>> >
>> >> >
>> >> > So I'm going to send "fix for
>> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
>> >> > separately, but it has no changelog at all.
>> >>
>> >> Probably it would be better if you drop everything except actually
>> >> fixes and stresstest. This is gone too far, now balloon won't compile
>> >> in the middle of patchset. Just tell me and I'll redo the rest.
>> >
>> > I think it's best if I drop everything:
>> >
>> > mm-balloon_compaction-ignore-anonymous-pages.patch
>> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
>> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
>> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
>> > mm-introduce-common-page-state-for-ballooned-memory.patch
>> > mm-balloon_compaction-use-common-page-ballooning.patch
>> > mm-balloon_compaction-general-cleanup.patch
>> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
>> >
>> > Please go through it and send out a new version?
>> >
>> >
>>
>> I've found yet another bug in this code. It seems here is a nest.
>> balloon_page_dequeue can race with balloon_page_isolate:
>> balloon_page_isolate can remove page from list between
>> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
>> balloon_page_dequeue runs under mutex_lock(>balloon_lock);
>> both of them lock page using trylock_page so race is tight but it is
>> not impossible.
> Plausible to happen if stress testing compaction simultaneously with
> freezing/unloading the balloon driver. As you noted, it's quite tight
> despite not impossible. Nice catch.
>
>
>> Probably it's really easier to rewrite it than to fix bugs one by one =/
> I'm not against a rewrite, but I don't think that rewriting the code to get
> rid
> of such bugs changes the fact we still have to address them in the actual
> placed
> code as we go on finding them. That's why I thought your inital changeset
> fine,
> with patches for stable going first and code overhaul for next following them
> up.
>
> For this race you spotted, I think a simple change like the following
> might be enough (not-tested)
This locking scheme is too fragile and uncommon.
What about this:
* special page->_mapcount marks ballooned pages
* page->private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not
isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's
not isolated, also it always clears page->private and balloon mark.
put-back rechecks mark after locking the page and releases it as
normal page if mark is gone.
>
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index 6e45a50..fd3a497 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
> balloon_dev_info *b_dev_info)
> * to be released by the balloon driver.
> */
> if (trylock_page(page)) {
> + /*
> +* Skip dequeue attempt for this page to a later round
> +* if balloon_page_isolate() has sucessfully isolated
> +* it just before we got the page lock here.
> +*/
> + if (page_count(page) != 1) {
> + unlock_page(page);
> + continue
> + }
> +
> spin_lock_irqsave(_dev_info->pages_lock, flags);
> /*
> * Raise the page refcount here to prevent any
> * wrong
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
> On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
> wrote:
> > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
> > wrote:
> >
> >> >
> >> > So I'm going to send "fix for
> >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> >> > separately, but it has no changelog at all.
> >>
> >> Probably it would be better if you drop everything except actually
> >> fixes and stresstest. This is gone too far, now balloon won't compile
> >> in the middle of patchset. Just tell me and I'll redo the rest.
> >
> > I think it's best if I drop everything:
> >
> > mm-balloon_compaction-ignore-anonymous-pages.patch
> > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
> > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
> > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
> > mm-introduce-common-page-state-for-ballooned-memory.patch
> > mm-balloon_compaction-use-common-page-ballooning.patch
> > mm-balloon_compaction-general-cleanup.patch
> > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
> >
> > Please go through it and send out a new version?
> >
> >
>
> I've found yet another bug in this code. It seems here is a nest.
> balloon_page_dequeue can race with balloon_page_isolate:
> balloon_page_isolate can remove page from list between
> llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
> balloon_page_dequeue runs under mutex_lock(>balloon_lock);
> both of them lock page using trylock_page so race is tight but it is
> not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
> Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid
of such bugs changes the fact we still have to address them in the actual placed
code as we go on finding them. That's why I thought your inital changeset fine,
with patches for stable going first and code overhaul for next following them
up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+* Skip dequeue attempt for this page to a later round
+* if balloon_page_isolate() has sucessfully isolated
+* it just before we got the page lock here.
+*/
+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(_dev_info->pages_lock, flags);
/*
* Raise the page refcount here to prevent any
* wrong
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton wrote: > On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov > wrote: > >> > >> > So I'm going to send "fix for >> > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus >> > separately, but it has no changelog at all. >> >> Probably it would be better if you drop everything except actually >> fixes and stresstest. This is gone too far, now balloon won't compile >> in the middle of patchset. Just tell me and I'll redo the rest. > > I think it's best if I drop everything: > > mm-balloon_compaction-ignore-anonymous-pages.patch > mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch > mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch > selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch > mm-introduce-common-page-state-for-ballooned-memory.patch > mm-balloon_compaction-use-common-page-ballooning.patch > mm-balloon_compaction-general-cleanup.patch > mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch > > Please go through it and send out a new version? > > I've found yet another bug in this code. It seems here is a nest. balloon_page_dequeue can race with balloon_page_isolate: balloon_page_isolate can remove page from list between llist_for_each_entry_safe and trylock_page in balloon_page_dequeue. balloon_page_dequeue runs under mutex_lock(>balloon_lock); both of them lock page using trylock_page so race is tight but it is not impossible. Probably it's really easier to rewrite it than to fix bugs one by one =/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton a...@linux-foundation.org wrote: On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov koc...@gmail.com wrote: So I'm going to send fix for mm-balloon_compaction-use-common-page-ballooning-v2 to Linus separately, but it has no changelog at all. Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest. I think it's best if I drop everything: mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch Please go through it and send out a new version? I've found yet another bug in this code. It seems here is a nest. balloon_page_dequeue can race with balloon_page_isolate: balloon_page_isolate can remove page from list between llist_for_each_entry_safe and trylock_page in balloon_page_dequeue. balloon_page_dequeue runs under mutex_lock(vb-balloon_lock); both of them lock page using trylock_page so race is tight but it is not impossible. Probably it's really easier to rewrite it than to fix bugs one by one =/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov koc...@gmail.com
wrote:
So I'm going to send fix for
mm-balloon_compaction-use-common-page-ballooning-v2 to Linus
separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch
mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
mm-introduce-common-page-state-for-ballooned-memory.patch
mm-balloon_compaction-use-common-page-ballooning.patch
mm-balloon_compaction-general-cleanup.patch
mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeue can race with balloon_page_isolate:
balloon_page_isolate can remove page from list between
llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
balloon_page_dequeue runs under mutex_lock(vb-balloon_lock);
both of them lock page using trylock_page so race is tight but it is
not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid
of such bugs changes the fact we still have to address them in the actual placed
code as we go on finding them. That's why I thought your inital changeset fine,
with patches for stable going first and code overhaul for next following them
up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+* Skip dequeue attempt for this page to a later round
+* if balloon_page_isolate() has sucessfully isolated
+* it just before we got the page lock here.
+*/
+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(b_dev_info-pages_lock, flags);
/*
* Raise the page refcount here to prevent any
* wrong
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini aqu...@redhat.com wrote:
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
koc...@gmail.com wrote:
So I'm going to send fix for
mm-balloon_compaction-use-common-page-ballooning-v2 to Linus
separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch
mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
mm-introduce-common-page-state-for-ballooned-memory.patch
mm-balloon_compaction-use-common-page-ballooning.patch
mm-balloon_compaction-general-cleanup.patch
mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeue can race with balloon_page_isolate:
balloon_page_isolate can remove page from list between
llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
balloon_page_dequeue runs under mutex_lock(vb-balloon_lock);
both of them lock page using trylock_page so race is tight but it is
not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get
rid
of such bugs changes the fact we still have to address them in the actual
placed
code as we go on finding them. That's why I thought your inital changeset
fine,
with patches for stable going first and code overhaul for next following them
up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
This locking scheme is too fragile and uncommon.
What about this:
* special page-_mapcount marks ballooned pages
* page-private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not
isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's
not isolated, also it always clears page-private and balloon mark.
put-back rechecks mark after locking the page and releases it as
normal page if mark is gone.
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+* Skip dequeue attempt for this page to a later round
+* if balloon_page_isolate() has sucessfully isolated
+* it just before we got the page lock here.
+*/
+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(b_dev_info-pages_lock, flags);
/*
* Raise the page refcount here to prevent any
* wrong
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini aqu...@redhat.com wrote:
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
koc...@gmail.com wrote:
So I'm going to send fix for
mm-balloon_compaction-use-common-page-ballooning-v2 to Linus
separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch
mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
mm-introduce-common-page-state-for-ballooned-memory.patch
mm-balloon_compaction-use-common-page-ballooning.patch
mm-balloon_compaction-general-cleanup.patch
mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeue can race with balloon_page_isolate:
balloon_page_isolate can remove page from list between
llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
balloon_page_dequeue runs under mutex_lock(vb-balloon_lock);
both of them lock page using trylock_page so race is tight but it is
not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get
rid
of such bugs changes the fact we still have to address them in the actual
placed
code as we go on finding them. That's why I thought your inital changeset
fine,
with patches for stable going first and code overhaul for next following
them up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
This locking scheme is too fragile and uncommon.
page_lock and refcounting was what I had at my disposal to sort these
races out since I haven't thoutgh on a special page-_mapcount when
designing this feature. It's the way other page races are sorted out.
Not arguing it's not fragile, but it's the way code is layed out since
some time, so we must check the feasibility of a total overhaul for
stable branches.
What about this:
* special page-_mapcount marks ballooned pages
* page-private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not
isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's
not isolated, also it always clears page-private and balloon mark.
put-back rechecks mark after locking the page and releases it as
normal page if mark is gone.
I have already agreed with you here, since the changes above are mostly from
your original overhaul proposal. It's a much better approach for that
balloon code, no doubts. Thanks for doing it. Only thing we need to take
care here is about its requirement on changing the semantics for those
interfaces might turn the changes unfeasible for old stable branches. If we
can ignore this mentioned fact entirely, I don't see why not going with
your idea all branches across, otherwise I think we should overhaul the
code for -next, and send pontual fixes for stable.
Cheers,
-- Rafael
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+* Skip dequeue attempt for this page to a later
round
+* if balloon_page_isolate() has sucessfully
isolated
+* it just before we got the page lock here.
+*/
+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(b_dev_info-pages_lock, flags);
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Tue, Sep 23, 2014 at 12:22 AM, Rafael Aquini aqu...@redhat.com wrote:
On Tue, Sep 23, 2014 at 12:06:11AM +0400, Konstantin Khlebnikov wrote:
On Mon, Sep 22, 2014 at 11:22 PM, Rafael Aquini aqu...@redhat.com wrote:
On Mon, Sep 22, 2014 at 10:40:34PM +0400, Konstantin Khlebnikov wrote:
On Sat, Sep 20, 2014 at 10:23 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov
koc...@gmail.com wrote:
So I'm going to send fix for
mm-balloon_compaction-use-common-page-ballooning-v2 to Linus
separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch
mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch
mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch
selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch
mm-introduce-common-page-state-for-ballooned-memory.patch
mm-balloon_compaction-use-common-page-ballooning.patch
mm-balloon_compaction-general-cleanup.patch
mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeue can race with balloon_page_isolate:
balloon_page_isolate can remove page from list between
llist_for_each_entry_safe and trylock_page in balloon_page_dequeue.
balloon_page_dequeue runs under mutex_lock(vb-balloon_lock);
both of them lock page using trylock_page so race is tight but it is
not impossible.
Plausible to happen if stress testing compaction simultaneously with
freezing/unloading the balloon driver. As you noted, it's quite tight
despite not impossible. Nice catch.
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to
get rid
of such bugs changes the fact we still have to address them in the actual
placed
code as we go on finding them. That's why I thought your inital changeset
fine,
with patches for stable going first and code overhaul for next following
them up.
For this race you spotted, I think a simple change like the following
might be enough (not-tested)
This locking scheme is too fragile and uncommon.
page_lock and refcounting was what I had at my disposal to sort these
races out since I haven't thoutgh on a special page-_mapcount when
designing this feature. It's the way other page races are sorted out.
Not arguing it's not fragile, but it's the way code is layed out since
some time, so we must check the feasibility of a total overhaul for
stable branches.
What about this:
* special page-_mapcount marks ballooned pages
* page-private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not
isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's
not isolated, also it always clears page-private and balloon mark.
put-back rechecks mark after locking the page and releases it as
normal page if mark is gone.
I have already agreed with you here, since the changes above are mostly from
your original overhaul proposal. It's a much better approach for that
balloon code, no doubts. Thanks for doing it. Only thing we need to take
care here is about its requirement on changing the semantics for those
interfaces might turn the changes unfeasible for old stable branches. If we
can ignore this mentioned fact entirely, I don't see why not going with
your idea all branches across, otherwise I think we should overhaul the
code for -next, and send pontual fixes for stable.
Ok, I'll try to implement this approach in suitable for stable branches way.
Keeping several different versions is overkill for this code.
Cheers,
-- Rafael
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 6e45a50..fd3a497 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+* Skip dequeue attempt for this page to a later
round
+* if balloon_page_isolate() has sucessfully
isolated
+* it just before we got the page lock here.
+*/
+ if (page_count(page) != 1) {
+
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov wrote: > > > > So I'm going to send "fix for > > mm-balloon_compaction-use-common-page-ballooning-v2" to Linus > > separately, but it has no changelog at all. > > Probably it would be better if you drop everything except actually > fixes and stresstest. This is gone too far, now balloon won't compile > in the middle of patchset. Just tell me and I'll redo the rest. I think it's best if I drop everything: mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch Please go through it and send out a new version? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov koc...@gmail.com wrote: So I'm going to send fix for mm-balloon_compaction-use-common-page-ballooning-v2 to Linus separately, but it has no changelog at all. Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest. I think it's best if I drop everything: mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch Please go through it and send out a new version? -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 20, 2014 at 1:35 AM, Andrew Morton
wrote:
> On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov
> wrote:
>
>> On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
>> wrote:
>> > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov
>> > wrote:
>> >
>> >> >
>> >> > Did we really need to put the BalloonPages count into per-zone vmstat,
>> >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
>> >> > important?
>> >>
>> >> Balloon grabs random pages, their distribution among numa nodes might
>> >> be important.
>> >> But I know nobody who uses numa-aware vm together with ballooning.
>> >>
>> >> Probably it's better to drop per-zone vmstat and line from meminfo,
>> >> global vmstat counter should be enough.
>> >
>> > Yes, the less we add the better - we can always add stuff later if
>> > there is a demonstrated need.
>>
>> Ok. (I guess incremental patches are more convenient for you)
>> Here is two fixes which remove redundant per-zone counters and adds
>> three vmstat counters: "balloon_inflate", "balloon_deflate" and
>> "balloon_migrate".
>>
>> This statistic seems more useful than current state snapshot.
>> Size of balloon is just a difference between "inflate" and "deflate".
>
> This is a complete mess.
nod
>
> Your second patch (which is actually the first one) called "fix for
> mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a
> fix for
> mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
> has a changelog.
>
> I eventually worked out that your first patch (whcih is actually the
> second one) called "fix for
> mm-balloon_compaction-use-common-page-ballooning-v2" assumes that
> mm-balloon_compaction-general-cleanup.patch has been applied. Does it
> actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
> I can't tell, because the "general cleanup" is in the way.
Oops I did it again.
>
> So I'm going to send "fix for
> mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
> separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
>
> Please always write changelogs. Please never send more than one patch
> in a single email. Please be *consistent* in filenames, patch titles,
> email subjects, etc.
That patch is supposed to be merged into patch as a fix.
>
> Please send me a changelog for the below patch.
Ok, sure.
>
From: Konstantin Khlebnikov
Subject: mm/balloon_compaction: use vmstat counters
This is fix for "mm/balloon_compaction: use common page ballooning".
it reverts per-zone balloon counters and replaces them with vmstat counters:
"balloon_inflate", "balloon_deflate" and "balloon_migrate".
Per-zone balloon counters have been reverted after discussion but reverting
them from balloon_compaction conflicts with massive cleanup in this code.
Thus this change ends up as a separate patch. Sorry for the mess.
Signed-off-by: Konstantin Khlebnikov
Signed-off-by: Andrew Morton
> ---
>
> drivers/virtio/virtio_balloon.c|1 +
> include/linux/balloon_compaction.h |2 --
> mm/balloon_compaction.c|2 ++
> 3 files changed, 3 insertions(+), 2 deletions(-)
>
> diff -puN
> drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> drivers/virtio/virtio_balloon.c
> ---
> a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/drivers/virtio/virtio_balloon.c
> @@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
> /* balloon's page migration 1st step -- inflate "newpage" */
> spin_lock_irqsave(_dev_info->pages_lock, flags);
> balloon_page_insert(vb_dev_info, newpage);
> + __count_vm_event(BALLOON_MIGRATE);
> vb_dev_info->isolated_pages--;
> spin_unlock_irqrestore(_dev_info->pages_lock, flags);
> vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> diff -puN
> include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> include/linux/balloon_compaction.h
> ---
> a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
> +++ a/include/linux/balloon_compaction.h
> @@ -87,7 +87,6 @@ static inline void
> balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
> {
> __SetPageBalloon(page);
> - inc_zone_page_state(page, NR_BALLOON_PAGES);
> set_page_private(page, (unsigned long)b_dev_info);
> list_add(>lru, _dev_info->pages);
> }
> @@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
> static inline void balloon_page_delete(struct page *page, bool isolated)
> {
> __ClearPageBalloon(page);
> - dec_zone_page_state(page, NR_BALLOON_PAGES);
> set_page_private(page, 0);
> if (!isolated)
>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov
wrote:
> On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
> wrote:
> > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov
> > wrote:
> >
> >> >
> >> > Did we really need to put the BalloonPages count into per-zone vmstat,
> >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> >> > important?
> >>
> >> Balloon grabs random pages, their distribution among numa nodes might
> >> be important.
> >> But I know nobody who uses numa-aware vm together with ballooning.
> >>
> >> Probably it's better to drop per-zone vmstat and line from meminfo,
> >> global vmstat counter should be enough.
> >
> > Yes, the less we add the better - we can always add stuff later if
> > there is a demonstrated need.
>
> Ok. (I guess incremental patches are more convenient for you)
> Here is two fixes which remove redundant per-zone counters and adds
> three vmstat counters: "balloon_inflate", "balloon_deflate" and
> "balloon_migrate".
>
> This statistic seems more useful than current state snapshot.
> Size of balloon is just a difference between "inflate" and "deflate".
This is a complete mess.
Your second patch (which is actually the first one) called "fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a
fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
has a changelog.
I eventually worked out that your first patch (whcih is actually the
second one) called "fix for
mm-balloon_compaction-use-common-page-ballooning-v2" assumes that
mm-balloon_compaction-general-cleanup.patch has been applied. Does it
actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
I can't tell, because the "general cleanup" is in the way.
So I'm going to send "fix for
mm-balloon_compaction-use-common-page-ballooning-v2" to Linus
separately, but it has no changelog at all.
Please always write changelogs. Please never send more than one patch
in a single email. Please be *consistent* in filenames, patch titles,
email subjects, etc.
Please send me a changelog for the below patch.
From: Konstantin Khlebnikov
Subject: mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
Signed-off-by: Konstantin Khlebnikov
Signed-off-by: Andrew Morton
---
drivers/virtio/virtio_balloon.c|1 +
include/linux/balloon_compaction.h |2 --
mm/balloon_compaction.c|2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
diff -puN
drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
drivers/virtio/virtio_balloon.c
---
a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/drivers/virtio/virtio_balloon.c
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(_dev_info->pages_lock, flags);
balloon_page_insert(vb_dev_info, newpage);
+ __count_vm_event(BALLOON_MIGRATE);
vb_dev_info->isolated_pages--;
spin_unlock_irqrestore(_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
diff -puN
include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
include/linux/balloon_compaction.h
---
a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/include/linux/balloon_compaction.h
@@ -87,7 +87,6 @@ static inline void
balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, (unsigned long)b_dev_info);
list_add(>lru, _dev_info->pages);
}
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, 0);
if (!isolated)
list_del(>lru);
diff -puN
mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
mm/balloon_compaction.c
---
a/mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/mm/balloon_compaction.c
@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct
BUG_ON(!trylock_page(page));
spin_lock_irqsave(_dev_info->pages_lock, flags);
balloon_page_insert(b_dev_info, page);
+ __count_vm_event(BALLOON_INFLATE);
spin_unlock_irqrestore(_dev_info->pages_lock, flags);
unlock_page(page);
return page;
@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct
if (trylock_page(page)) {
spin_lock_irqsave(_dev_info->pages_lock, flags);
balloon_page_delete(page, false);
+ __count_vm_event(BALLOON_DEFLATE);
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov koc...@gmail.com
wrote:
On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov koc...@gmail.com
wrote:
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Balloon grabs random pages, their distribution among numa nodes might
be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo,
global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if
there is a demonstrated need.
Ok. (I guess incremental patches are more convenient for you)
Here is two fixes which remove redundant per-zone counters and adds
three vmstat counters: balloon_inflate, balloon_deflate and
balloon_migrate.
This statistic seems more useful than current state snapshot.
Size of balloon is just a difference between inflate and deflate.
This is a complete mess.
Your second patch (which is actually the first one) called fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2 is indeed a
fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
has a changelog.
I eventually worked out that your first patch (whcih is actually the
second one) called fix for
mm-balloon_compaction-use-common-page-ballooning-v2 assumes that
mm-balloon_compaction-general-cleanup.patch has been applied. Does it
actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
I can't tell, because the general cleanup is in the way.
So I'm going to send fix for
mm-balloon_compaction-use-common-page-ballooning-v2 to Linus
separately, but it has no changelog at all.
Please always write changelogs. Please never send more than one patch
in a single email. Please be *consistent* in filenames, patch titles,
email subjects, etc.
Please send me a changelog for the below patch.
From: Konstantin Khlebnikov koc...@gmail.com
Subject: mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
Signed-off-by: Konstantin Khlebnikov koc...@gmail.com
Signed-off-by: Andrew Morton a...@linux-foundation.org
---
drivers/virtio/virtio_balloon.c|1 +
include/linux/balloon_compaction.h |2 --
mm/balloon_compaction.c|2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
diff -puN
drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
drivers/virtio/virtio_balloon.c
---
a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/drivers/virtio/virtio_balloon.c
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
/* balloon's page migration 1st step -- inflate newpage */
spin_lock_irqsave(vb_dev_info-pages_lock, flags);
balloon_page_insert(vb_dev_info, newpage);
+ __count_vm_event(BALLOON_MIGRATE);
vb_dev_info-isolated_pages--;
spin_unlock_irqrestore(vb_dev_info-pages_lock, flags);
vb-num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
diff -puN
include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
include/linux/balloon_compaction.h
---
a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/include/linux/balloon_compaction.h
@@ -87,7 +87,6 @@ static inline void
balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, (unsigned long)b_dev_info);
list_add(page-lru, b_dev_info-pages);
}
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, 0);
if (!isolated)
list_del(page-lru);
diff -puN
mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
mm/balloon_compaction.c
---
a/mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/mm/balloon_compaction.c
@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(struct
BUG_ON(!trylock_page(page));
spin_lock_irqsave(b_dev_info-pages_lock, flags);
balloon_page_insert(b_dev_info, page);
+ __count_vm_event(BALLOON_INFLATE);
spin_unlock_irqrestore(b_dev_info-pages_lock, flags);
unlock_page(page);
return page;
@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(struct
if (trylock_page(page)) {
spin_lock_irqsave(b_dev_info-pages_lock, flags);
balloon_page_delete(page, false);
+
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 20, 2014 at 1:35 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov koc...@gmail.com
wrote:
On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov
koc...@gmail.com wrote:
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Balloon grabs random pages, their distribution among numa nodes might
be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo,
global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if
there is a demonstrated need.
Ok. (I guess incremental patches are more convenient for you)
Here is two fixes which remove redundant per-zone counters and adds
three vmstat counters: balloon_inflate, balloon_deflate and
balloon_migrate.
This statistic seems more useful than current state snapshot.
Size of balloon is just a difference between inflate and deflate.
This is a complete mess.
nod
Your second patch (which is actually the first one) called fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2 is indeed a
fix for
mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and
has a changelog.
I eventually worked out that your first patch (whcih is actually the
second one) called fix for
mm-balloon_compaction-use-common-page-ballooning-v2 assumes that
mm-balloon_compaction-general-cleanup.patch has been applied. Does it
actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch?
I can't tell, because the general cleanup is in the way.
Oops I did it again.
So I'm going to send fix for
mm-balloon_compaction-use-common-page-ballooning-v2 to Linus
separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually
fixes and stresstest. This is gone too far, now balloon won't compile
in the middle of patchset. Just tell me and I'll redo the rest.
Please always write changelogs. Please never send more than one patch
in a single email. Please be *consistent* in filenames, patch titles,
email subjects, etc.
That patch is supposed to be merged into patch as a fix.
Please send me a changelog for the below patch.
Ok, sure.
From: Konstantin Khlebnikov koc...@gmail.com
Subject: mm/balloon_compaction: use vmstat counters
This is fix for mm/balloon_compaction: use common page ballooning.
it reverts per-zone balloon counters and replaces them with vmstat counters:
balloon_inflate, balloon_deflate and balloon_migrate.
Per-zone balloon counters have been reverted after discussion but reverting
them from balloon_compaction conflicts with massive cleanup in this code.
Thus this change ends up as a separate patch. Sorry for the mess.
Signed-off-by: Konstantin Khlebnikov koc...@gmail.com
Signed-off-by: Andrew Morton a...@linux-foundation.org
---
drivers/virtio/virtio_balloon.c|1 +
include/linux/balloon_compaction.h |2 --
mm/balloon_compaction.c|2 ++
3 files changed, 3 insertions(+), 2 deletions(-)
diff -puN
drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
drivers/virtio/virtio_balloon.c
---
a/drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/drivers/virtio/virtio_balloon.c
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc
/* balloon's page migration 1st step -- inflate newpage */
spin_lock_irqsave(vb_dev_info-pages_lock, flags);
balloon_page_insert(vb_dev_info, newpage);
+ __count_vm_event(BALLOON_MIGRATE);
vb_dev_info-isolated_pages--;
spin_unlock_irqrestore(vb_dev_info-pages_lock, flags);
vb-num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
diff -puN
include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
include/linux/balloon_compaction.h
---
a/include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
+++ a/include/linux/balloon_compaction.h
@@ -87,7 +87,6 @@ static inline void
balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page)
{
__SetPageBalloon(page);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, (unsigned long)b_dev_info);
list_add(page-lru, b_dev_info-pages);
}
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_i
static inline void balloon_page_delete(struct page *page, bool isolated)
{
__ClearPageBalloon(page);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
set_page_private(page, 0);
if (!isolated)
list_del(page-lru);
diff -puN
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On 09/13/2014 01:26 AM, Konstantin Khlebnikov wrote: >> Did we really need to put the BalloonPages count into per-zone vmstat, >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so >> > important? > Balloon grabs random pages, their distribution among numa nodes might > be important. > But I know nobody who uses numa-aware vm together with ballooning. *cough*. me? Obviously there's no need to keep that per-zone vmstat just for me, but right now NUMA on KVM works just fine and does a good job of catching NUMA issues without having to run on actual metal. Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton wrote: > On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov > wrote: > >> > >> > Did we really need to put the BalloonPages count into per-zone vmstat, >> > global vmstat and /proc/meminfo? Seems a bit overkillish - why so >> > important? >> >> Balloon grabs random pages, their distribution among numa nodes might >> be important. >> But I know nobody who uses numa-aware vm together with ballooning. >> >> Probably it's better to drop per-zone vmstat and line from meminfo, >> global vmstat counter should be enough. > > Yes, the less we add the better - we can always add stuff later if > there is a demonstrated need. Ok. (I guess incremental patches are more convenient for you) Here is two fixes which remove redundant per-zone counters and adds three vmstat counters: "balloon_inflate", "balloon_deflate" and "balloon_migrate". This statistic seems more useful than current state snapshot. Size of balloon is just a difference between "inflate" and "deflate". > >> > >> > Consuming another page flag is a big deal. We keep on nearly running >> > out and one day we'll run out for real. page-flags-layout.h is >> > incomprehensible. How many flags do we have left (worst-case) with this >> > change? Is there no other way? Needs extraordinary justification, >> > please. >> >> PageBalloon is not a page flags, it's like PageBuddy -- special state >> of _mapcount (-256 in this case). >> The same was in v1 and is written in the comment above. > > oop sorry, I got confused about KPF_BALLOON. mm-balloon_compaction-use-common-page-ballooning-use-vmstat-counters Description: Binary data mm-introduce-common-page-state-for-ballooned-memory-vmstat-counters Description: Binary data
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 13, 2014 at 9:42 AM, Andrew Morton a...@linux-foundation.org wrote: On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov koc...@gmail.com wrote: Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and /proc/meminfo? Seems a bit overkillish - why so important? Balloon grabs random pages, their distribution among numa nodes might be important. But I know nobody who uses numa-aware vm together with ballooning. Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough. Yes, the less we add the better - we can always add stuff later if there is a demonstrated need. Ok. (I guess incremental patches are more convenient for you) Here is two fixes which remove redundant per-zone counters and adds three vmstat counters: balloon_inflate, balloon_deflate and balloon_migrate. This statistic seems more useful than current state snapshot. Size of balloon is just a difference between inflate and deflate. Consuming another page flag is a big deal. We keep on nearly running out and one day we'll run out for real. page-flags-layout.h is incomprehensible. How many flags do we have left (worst-case) with this change? Is there no other way? Needs extraordinary justification, please. PageBalloon is not a page flags, it's like PageBuddy -- special state of _mapcount (-256 in this case). The same was in v1 and is written in the comment above. oop sorry, I got confused about KPF_BALLOON. mm-balloon_compaction-use-common-page-ballooning-use-vmstat-counters Description: Binary data mm-introduce-common-page-state-for-ballooned-memory-vmstat-counters Description: Binary data
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On 09/13/2014 01:26 AM, Konstantin Khlebnikov wrote: Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and /proc/meminfo? Seems a bit overkillish - why so important? Balloon grabs random pages, their distribution among numa nodes might be important. But I know nobody who uses numa-aware vm together with ballooning. *cough*. me? Obviously there's no need to keep that per-zone vmstat just for me, but right now NUMA on KVM works just fine and does a good job of catching NUMA issues without having to run on actual metal. Thanks, Sasha -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov wrote: > > > > Did we really need to put the BalloonPages count into per-zone vmstat, > > global vmstat and /proc/meminfo? Seems a bit overkillish - why so > > important? > > Balloon grabs random pages, their distribution among numa nodes might > be important. > But I know nobody who uses numa-aware vm together with ballooning. > > Probably it's better to drop per-zone vmstat and line from meminfo, > global vmstat counter should be enough. Yes, the less we add the better - we can always add stuff later if there is a demonstrated need. > > > > Consuming another page flag is a big deal. We keep on nearly running > > out and one day we'll run out for real. page-flags-layout.h is > > incomprehensible. How many flags do we have left (worst-case) with this > > change? Is there no other way? Needs extraordinary justification, > > please. > > PageBalloon is not a page flags, it's like PageBuddy -- special state > of _mapcount (-256 in this case). > The same was in v1 and is written in the comment above. oop sorry, I got confused about KPF_BALLOON. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 13, 2014 at 3:51 AM, Andrew Morton
wrote:
> On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov
> wrote:
>
>> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
>> Like PageBuddy() PageBalloon() looks like page-flag but actually this is
>> special
>> state of page->_mapcount counter. There is no conflict because ballooned
>> pages
>> cannot be mapped and cannot be in buddy allocator.
>>
>> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown
>> them
>> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon
>> into
>> userspace via /proc/kpageflags as KPF_BALLOON.
>>
>> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
>> ballooning driver which wants use this feature.
>
> The delta from the (fixed) v1 is below.
>
> What's up with those Kconfig/Makefile changes? We're now including a
> pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
> changes were not changelogged?
It's been moved into next patch, and not mentioned in changes. Sorry.
>
> Did we really need to put the BalloonPages count into per-zone vmstat,
> global vmstat and /proc/meminfo? Seems a bit overkillish - why so
> important?
Balloon grabs random pages, their distribution among numa nodes might
be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo,
global vmstat counter should be enough.
>
> Consuming another page flag is a big deal. We keep on nearly running
> out and one day we'll run out for real. page-flags-layout.h is
> incomprehensible. How many flags do we have left (worst-case) with this
> change? Is there no other way? Needs extraordinary justification,
> please.
PageBalloon is not a page flags, it's like PageBuddy -- special state
of _mapcount (-256 in this case).
The same was in v1 and is written in the comment above.
>
> drivers/virtio/Kconfig |1 -
> include/linux/mm.h | 14 --
> mm/Makefile |3 +--
> mm/balloon_compaction.c | 16
> 4 files changed, 13 insertions(+), 21 deletions(-)
>
> diff -puN
> drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> drivers/virtio/Kconfig
> ---
> a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/drivers/virtio/Kconfig
> @@ -25,7 +25,6 @@ config VIRTIO_PCI
> config VIRTIO_BALLOON
> tristate "Virtio balloon driver"
> depends on VIRTIO
> - select MEMORY_BALLOON
> ---help---
> This driver supports increasing and decreasing the amount
> of memory within a KVM guest.
> diff -puN
> include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> include/linux/mm.h
> ---
> a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/include/linux/mm.h
> @@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
> return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
> atomic_read(>_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
> }
> -void __SetPageBalloon(struct page *page);
> -void __ClearPageBalloon(struct page *page);
> +
> +static inline void __SetPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(atomic_read(>_mapcount) != -1, page);
> + atomic_set(>_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
> +}
> +
> +static inline void __ClearPageBalloon(struct page *page)
> +{
> + VM_BUG_ON_PAGE(!PageBalloon(page), page);
> + atomic_set(>_mapcount, -1);
> +}
>
> void put_page(struct page *page);
> void put_pages_list(struct list_head *pages);
> diff -puN
> mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> mm/Makefile
> --- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/mm/Makefile
> @@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
>readahead.o swap.o truncate.o vmscan.o shmem.o \
>util.o mmzone.o vmstat.o backing-dev.o \
>mm_init.o mmu_context.o percpu.o slab_common.o \
> - compaction.o vmacache.o \
> + compaction.o balloon_compaction.o vmacache.o \
>interval_tree.o list_lru.o workingset.o \
>iov_iter.o $(mmu-y)
>
> @@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD)+= zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> obj-$(CONFIG_CMA) += cma.o
> -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
> diff -puN
> mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> mm/balloon_compaction.c
> ---
> a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
> +++ a/mm/balloon_compaction.c
> @@ -10,22 +10,6 @@
> #include
> #include
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov
wrote:
> This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
> Like PageBuddy() PageBalloon() looks like page-flag but actually this is
> special
> state of page->_mapcount counter. There is no conflict because ballooned pages
> cannot be mapped and cannot be in buddy allocator.
>
> Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown
> them
> in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
> userspace via /proc/kpageflags as KPF_BALLOON.
>
> All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
> ballooning driver which wants use this feature.
The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a
pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
changes were not changelogged?
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Consuming another page flag is a big deal. We keep on nearly running
out and one day we'll run out for real. page-flags-layout.h is
incomprehensible. How many flags do we have left (worst-case) with this
change? Is there no other way? Needs extraordinary justification,
please.
drivers/virtio/Kconfig |1 -
include/linux/mm.h | 14 --
mm/Makefile |3 +--
mm/balloon_compaction.c | 16
4 files changed, 13 insertions(+), 21 deletions(-)
diff -puN
drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
drivers/virtio/Kconfig
---
a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/drivers/virtio/Kconfig
@@ -25,7 +25,6 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
- select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff -puN
include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
include/linux/mm.h
---
a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/include/linux/mm.h
@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
return IS_ENABLED(CONFIG_MEMORY_BALLOON) &&
atomic_read(>_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
}
-void __SetPageBalloon(struct page *page);
-void __ClearPageBalloon(struct page *page);
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(>_mapcount) != -1, page);
+ atomic_set(>_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(>_mapcount, -1);
+}
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff -puN
mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
mm/Makefile
--- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o vmacache.o \
+ compaction.o balloon_compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD)+= zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
-obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff -puN
mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
mm/balloon_compaction.c
---
a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/balloon_compaction.c
@@ -10,22 +10,6 @@
#include
#include
-void __SetPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(atomic_read(>_mapcount) != -1, page);
- atomic_set(>_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__SetPageBalloon);
-
-void __ClearPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(!PageBalloon(page), page);
- atomic_set(>_mapcount, -1);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__ClearPageBalloon);
-
/*
* balloon_devinfo_alloc - allocates a balloon device information descriptor.
* @balloon_dev_descriptor: pointer to reference the balloon device which
_
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov koc...@gmail.com
wrote:
This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
Like PageBuddy() PageBalloon() looks like page-flag but actually this is
special
state of page-_mapcount counter. There is no conflict because ballooned pages
cannot be mapped and cannot be in buddy allocator.
Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown
them
in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into
userspace via /proc/kpageflags as KPF_BALLOON.
All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
ballooning driver which wants use this feature.
The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a
pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
changes were not changelogged?
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Consuming another page flag is a big deal. We keep on nearly running
out and one day we'll run out for real. page-flags-layout.h is
incomprehensible. How many flags do we have left (worst-case) with this
change? Is there no other way? Needs extraordinary justification,
please.
drivers/virtio/Kconfig |1 -
include/linux/mm.h | 14 --
mm/Makefile |3 +--
mm/balloon_compaction.c | 16
4 files changed, 13 insertions(+), 21 deletions(-)
diff -puN
drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
drivers/virtio/Kconfig
---
a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/drivers/virtio/Kconfig
@@ -25,7 +25,6 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate Virtio balloon driver
depends on VIRTIO
- select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff -puN
include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
include/linux/mm.h
---
a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/include/linux/mm.h
@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
return IS_ENABLED(CONFIG_MEMORY_BALLOON)
atomic_read(page-_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
}
-void __SetPageBalloon(struct page *page);
-void __ClearPageBalloon(struct page *page);
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(page-_mapcount) != -1, page);
+ atomic_set(page-_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(page-_mapcount, -1);
+}
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff -puN
mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
mm/Makefile
--- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o vmacache.o \
+ compaction.o balloon_compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD)+= zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
-obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff -puN
mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
mm/balloon_compaction.c
---
a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/balloon_compaction.c
@@ -10,22 +10,6 @@
#include linux/export.h
#include linux/balloon_compaction.h
-void __SetPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(atomic_read(page-_mapcount) != -1, page);
- atomic_set(page-_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
- inc_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__SetPageBalloon);
-
-void __ClearPageBalloon(struct page *page)
-{
- VM_BUG_ON_PAGE(!PageBalloon(page), page);
- atomic_set(page-_mapcount, -1);
- dec_zone_page_state(page, NR_BALLOON_PAGES);
-}
-EXPORT_SYMBOL(__ClearPageBalloon);
-
/*
* balloon_devinfo_alloc - allocates a balloon device information descriptor.
* @balloon_dev_descriptor: pointer to reference the balloon device which
_
--
To unsubscribe from this
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Sep 13, 2014 at 3:51 AM, Andrew Morton
a...@linux-foundation.org wrote:
On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov koc...@gmail.com
wrote:
This patch adds page state PageBallon() and functions __Set/ClearPageBalloon.
Like PageBuddy() PageBalloon() looks like page-flag but actually this is
special
state of page-_mapcount counter. There is no conflict because ballooned
pages
cannot be mapped and cannot be in buddy allocator.
Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown
them
in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon
into
userspace via /proc/kpageflags as KPF_BALLOON.
All new code is under CONFIG_MEMORY_BALLOON, it should be selected by
ballooning driver which wants use this feature.
The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a
pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These
changes were not changelogged?
It's been moved into next patch, and not mentioned in changes. Sorry.
Did we really need to put the BalloonPages count into per-zone vmstat,
global vmstat and /proc/meminfo? Seems a bit overkillish - why so
important?
Balloon grabs random pages, their distribution among numa nodes might
be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo,
global vmstat counter should be enough.
Consuming another page flag is a big deal. We keep on nearly running
out and one day we'll run out for real. page-flags-layout.h is
incomprehensible. How many flags do we have left (worst-case) with this
change? Is there no other way? Needs extraordinary justification,
please.
PageBalloon is not a page flags, it's like PageBuddy -- special state
of _mapcount (-256 in this case).
The same was in v1 and is written in the comment above.
drivers/virtio/Kconfig |1 -
include/linux/mm.h | 14 --
mm/Makefile |3 +--
mm/balloon_compaction.c | 16
4 files changed, 13 insertions(+), 21 deletions(-)
diff -puN
drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
drivers/virtio/Kconfig
---
a/drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/drivers/virtio/Kconfig
@@ -25,7 +25,6 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate Virtio balloon driver
depends on VIRTIO
- select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.
diff -puN
include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
include/linux/mm.h
---
a/include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/include/linux/mm.h
@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pag
return IS_ENABLED(CONFIG_MEMORY_BALLOON)
atomic_read(page-_mapcount) == PAGE_BALLOON_MAPCOUNT_VALUE;
}
-void __SetPageBalloon(struct page *page);
-void __ClearPageBalloon(struct page *page);
+
+static inline void __SetPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(atomic_read(page-_mapcount) != -1, page);
+ atomic_set(page-_mapcount, PAGE_BALLOON_MAPCOUNT_VALUE);
+}
+
+static inline void __ClearPageBalloon(struct page *page)
+{
+ VM_BUG_ON_PAGE(!PageBalloon(page), page);
+ atomic_set(page-_mapcount, -1);
+}
void put_page(struct page *page);
void put_pages_list(struct list_head *pages);
diff -puN
mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
mm/Makefile
--- a/mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.
readahead.o swap.o truncate.o vmscan.o shmem.o \
util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o vmacache.o \
+ compaction.o balloon_compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
iov_iter.o $(mmu-y)
@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD)+= zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
-obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff -puN
mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
mm/balloon_compaction.c
---
a/mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2
+++ a/mm/balloon_compaction.c
@@ -10,22 +10,6 @@
#include linux/export.h
#include linux/balloon_compaction.h
-void __SetPageBalloon(struct
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov koc...@gmail.com wrote: Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and /proc/meminfo? Seems a bit overkillish - why so important? Balloon grabs random pages, their distribution among numa nodes might be important. But I know nobody who uses numa-aware vm together with ballooning. Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough. Yes, the less we add the better - we can always add stuff later if there is a demonstrated need. Consuming another page flag is a big deal. We keep on nearly running out and one day we'll run out for real. page-flags-layout.h is incomprehensible. How many flags do we have left (worst-case) with this change? Is there no other way? Needs extraordinary justification, please. PageBalloon is not a page flags, it's like PageBuddy -- special state of _mapcount (-256 in this case). The same was in v1 and is written in the comment above. oop sorry, I got confused about KPF_BALLOON. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Aug 30, 2014 at 08:41:20PM +0400, Konstantin Khlebnikov wrote: > From: Konstantin Khlebnikov > > This patch adds page state PageBallon() and functions __Set/ClearPageBalloon. > Like PageBuddy() PageBalloon() looks like page-flag but actually this is > special > state of page->_mapcount counter. There is no conflict because ballooned pages > cannot be mapped and cannot be in buddy allocator. > > Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown > them > in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into > userspace via /proc/kpageflags as KPF_BALLOON. > > All new code is under CONFIG_MEMORY_BALLOON, it should be selected by > ballooning driver which wants use this feature. > > Signed-off-by: Konstantin Khlebnikov > --- > Documentation/filesystems/proc.txt |2 ++ > drivers/base/node.c| 16 ++-- > fs/proc/meminfo.c |6 ++ > fs/proc/page.c |3 +++ > include/linux/mm.h | 20 > include/linux/mmzone.h |3 +++ > include/uapi/linux/kernel-page-flags.h |1 + > mm/Kconfig |5 + > mm/vmstat.c|8 +++- > tools/vm/page-types.c |1 + > 10 files changed, 58 insertions(+), 7 deletions(-) > > diff --git a/Documentation/filesystems/proc.txt > b/Documentation/filesystems/proc.txt > index eb8a10e..154a345 100644 > --- a/Documentation/filesystems/proc.txt > +++ b/Documentation/filesystems/proc.txt > @@ -796,6 +796,7 @@ VmallocTotal: 112216 kB > VmallocUsed: 428 kB > VmallocChunk: 111088 kB > AnonHugePages: 49152 kB > +BalloonPages:0 kB > > MemTotal: Total usable ram (i.e. physical ram minus a few reserved >bits and the kernel binary code) > @@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available > for starting new > Writeback: Memory which is actively being written back to the disk > AnonPages: Non-file backed pages mapped into userspace page tables > AnonHugePages: Non-file backed huge pages mapped into userspace page tables > +BalloonPages: Memory which was ballooned, not included into MemTotal >Mapped: files which have been mmaped, such as libraries > Slab: in-kernel data structures cache > SReclaimable: Part of Slab, that might be reclaimed, such as caches > diff --git a/drivers/base/node.c b/drivers/base/node.c > index c6d3ae0..59e565c 100644 > --- a/drivers/base/node.c > +++ b/drivers/base/node.c > @@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev, > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > "Node %d AnonHugePages: %8lu kB\n" > #endif > +#ifdef CONFIG_MEMORY_BALLOON > +"Node %d BalloonPages: %8lu kB\n" > +#endif > , > nid, K(node_page_state(nid, NR_FILE_DIRTY)), > nid, K(node_page_state(nid, NR_WRITEBACK)), > @@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev, > nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) + > node_page_state(nid, NR_SLAB_UNRECLAIMABLE)), > nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)), > -#ifdef CONFIG_TRANSPARENT_HUGEPAGE > nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)) > - , nid, > - K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) * > - HPAGE_PMD_NR)); > -#else > -nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); > +#ifdef CONFIG_TRANSPARENT_HUGEPAGE > +,nid, K(node_page_state(nid, > + NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) > +#endif > +#ifdef CONFIG_MEMORY_BALLOON > +,nid, K(node_page_state(nid, NR_BALLOON_PAGES)) > #endif > +); > n += hugetlb_report_node_meminfo(nid, buf + n); > return n; > } > diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c > index aa1eee0..f897fbf 100644 > --- a/fs/proc/meminfo.c > +++ b/fs/proc/meminfo.c > @@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > "AnonHugePages: %8lu kB\n" > #endif > +#ifdef CONFIG_MEMORY_BALLOON > + "BalloonPages: %8lu kB\n" > +#endif > , > K(i.totalram), > K(i.freeram), > @@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) > ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) * > HPAGE_PMD_NR) > #endif > +#ifdef CONFIG_MEMORY_BALLOON > + ,K(global_page_state(NR_BALLOON_PAGES)) > +#endif > ); > > hugetlb_report_meminfo(m); > diff --git a/fs/proc/page.c
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
On Sat, Aug 30, 2014 at 08:41:20PM +0400, Konstantin Khlebnikov wrote: From: Konstantin Khlebnikov k.khlebni...@samsung.com This patch adds page state PageBallon() and functions __Set/ClearPageBalloon. Like PageBuddy() PageBalloon() looks like page-flag but actually this is special state of page-_mapcount counter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator. Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into userspace via /proc/kpageflags as KPF_BALLOON. All new code is under CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature. Signed-off-by: Konstantin Khlebnikov k.khlebni...@samsung.com --- Documentation/filesystems/proc.txt |2 ++ drivers/base/node.c| 16 ++-- fs/proc/meminfo.c |6 ++ fs/proc/page.c |3 +++ include/linux/mm.h | 20 include/linux/mmzone.h |3 +++ include/uapi/linux/kernel-page-flags.h |1 + mm/Kconfig |5 + mm/vmstat.c|8 +++- tools/vm/page-types.c |1 + 10 files changed, 58 insertions(+), 7 deletions(-) diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index eb8a10e..154a345 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -796,6 +796,7 @@ VmallocTotal: 112216 kB VmallocUsed: 428 kB VmallocChunk: 111088 kB AnonHugePages: 49152 kB +BalloonPages:0 kB MemTotal: Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code) @@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new Writeback: Memory which is actively being written back to the disk AnonPages: Non-file backed pages mapped into userspace page tables AnonHugePages: Non-file backed huge pages mapped into userspace page tables +BalloonPages: Memory which was ballooned, not included into MemTotal Mapped: files which have been mmaped, such as libraries Slab: in-kernel data structures cache SReclaimable: Part of Slab, that might be reclaimed, such as caches diff --git a/drivers/base/node.c b/drivers/base/node.c index c6d3ae0..59e565c 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE Node %d AnonHugePages: %8lu kB\n #endif +#ifdef CONFIG_MEMORY_BALLOON +Node %d BalloonPages: %8lu kB\n +#endif , nid, K(node_page_state(nid, NR_FILE_DIRTY)), nid, K(node_page_state(nid, NR_WRITEBACK)), @@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) + node_page_state(nid, NR_SLAB_UNRECLAIMABLE)), nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)), -#ifdef CONFIG_TRANSPARENT_HUGEPAGE nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)) - , nid, - K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) * - HPAGE_PMD_NR)); -#else -nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +,nid, K(node_page_state(nid, + NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) +#endif +#ifdef CONFIG_MEMORY_BALLOON +,nid, K(node_page_state(nid, NR_BALLOON_PAGES)) #endif +); n += hugetlb_report_node_meminfo(nid, buf + n); return n; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index aa1eee0..f897fbf 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE AnonHugePages: %8lu kB\n #endif +#ifdef CONFIG_MEMORY_BALLOON + BalloonPages: %8lu kB\n +#endif , K(i.totalram), K(i.freeram), @@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) #endif +#ifdef CONFIG_MEMORY_BALLOON + ,K(global_page_state(NR_BALLOON_PAGES)) +#endif ); hugetlb_report_meminfo(m); diff --git a/fs/proc/page.c b/fs/proc/page.c index e647c55..1e3187d 100644 --- a/fs/proc/page.c +++
[PATCH v2 4/6] mm: introduce common page state for ballooned memory
From: Konstantin Khlebnikov This patch adds page state PageBallon() and functions __Set/ClearPageBalloon. Like PageBuddy() PageBalloon() looks like page-flag but actually this is special state of page->_mapcount counter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator. Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into userspace via /proc/kpageflags as KPF_BALLOON. All new code is under CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature. Signed-off-by: Konstantin Khlebnikov --- Documentation/filesystems/proc.txt |2 ++ drivers/base/node.c| 16 ++-- fs/proc/meminfo.c |6 ++ fs/proc/page.c |3 +++ include/linux/mm.h | 20 include/linux/mmzone.h |3 +++ include/uapi/linux/kernel-page-flags.h |1 + mm/Kconfig |5 + mm/vmstat.c|8 +++- tools/vm/page-types.c |1 + 10 files changed, 58 insertions(+), 7 deletions(-) diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index eb8a10e..154a345 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -796,6 +796,7 @@ VmallocTotal: 112216 kB VmallocUsed: 428 kB VmallocChunk: 111088 kB AnonHugePages: 49152 kB +BalloonPages:0 kB MemTotal: Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code) @@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new Writeback: Memory which is actively being written back to the disk AnonPages: Non-file backed pages mapped into userspace page tables AnonHugePages: Non-file backed huge pages mapped into userspace page tables +BalloonPages: Memory which was ballooned, not included into MemTotal Mapped: files which have been mmaped, such as libraries Slab: in-kernel data structures cache SReclaimable: Part of Slab, that might be reclaimed, such as caches diff --git a/drivers/base/node.c b/drivers/base/node.c index c6d3ae0..59e565c 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE "Node %d AnonHugePages: %8lu kB\n" #endif +#ifdef CONFIG_MEMORY_BALLOON + "Node %d BalloonPages: %8lu kB\n" +#endif , nid, K(node_page_state(nid, NR_FILE_DIRTY)), nid, K(node_page_state(nid, NR_WRITEBACK)), @@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) + node_page_state(nid, NR_SLAB_UNRECLAIMABLE)), nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)), -#ifdef CONFIG_TRANSPARENT_HUGEPAGE nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)) - , nid, - K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) * - HPAGE_PMD_NR)); -#else - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ,nid, K(node_page_state(nid, + NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) +#endif +#ifdef CONFIG_MEMORY_BALLOON + ,nid, K(node_page_state(nid, NR_BALLOON_PAGES)) #endif + ); n += hugetlb_report_node_meminfo(nid, buf + n); return n; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index aa1eee0..f897fbf 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE "AnonHugePages: %8lu kB\n" #endif +#ifdef CONFIG_MEMORY_BALLOON + "BalloonPages: %8lu kB\n" +#endif , K(i.totalram), K(i.freeram), @@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) #endif +#ifdef CONFIG_MEMORY_BALLOON + ,K(global_page_state(NR_BALLOON_PAGES)) +#endif ); hugetlb_report_meminfo(m); diff --git a/fs/proc/page.c b/fs/proc/page.c index e647c55..1e3187d 100644 --- a/fs/proc/page.c +++ b/fs/proc/page.c @@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page) if (PageBuddy(page)) u |= 1 << KPF_BUDDY; + if
[PATCH v2 4/6] mm: introduce common page state for ballooned memory
From: Konstantin Khlebnikov k.khlebni...@samsung.com This patch adds page state PageBallon() and functions __Set/ClearPageBalloon. Like PageBuddy() PageBalloon() looks like page-flag but actually this is special state of page-_mapcount counter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator. Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into userspace via /proc/kpageflags as KPF_BALLOON. All new code is under CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature. Signed-off-by: Konstantin Khlebnikov k.khlebni...@samsung.com --- Documentation/filesystems/proc.txt |2 ++ drivers/base/node.c| 16 ++-- fs/proc/meminfo.c |6 ++ fs/proc/page.c |3 +++ include/linux/mm.h | 20 include/linux/mmzone.h |3 +++ include/uapi/linux/kernel-page-flags.h |1 + mm/Kconfig |5 + mm/vmstat.c|8 +++- tools/vm/page-types.c |1 + 10 files changed, 58 insertions(+), 7 deletions(-) diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index eb8a10e..154a345 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -796,6 +796,7 @@ VmallocTotal: 112216 kB VmallocUsed: 428 kB VmallocChunk: 111088 kB AnonHugePages: 49152 kB +BalloonPages:0 kB MemTotal: Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code) @@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new Writeback: Memory which is actively being written back to the disk AnonPages: Non-file backed pages mapped into userspace page tables AnonHugePages: Non-file backed huge pages mapped into userspace page tables +BalloonPages: Memory which was ballooned, not included into MemTotal Mapped: files which have been mmaped, such as libraries Slab: in-kernel data structures cache SReclaimable: Part of Slab, that might be reclaimed, such as caches diff --git a/drivers/base/node.c b/drivers/base/node.c index c6d3ae0..59e565c 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE Node %d AnonHugePages: %8lu kB\n #endif +#ifdef CONFIG_MEMORY_BALLOON + Node %d BalloonPages: %8lu kB\n +#endif , nid, K(node_page_state(nid, NR_FILE_DIRTY)), nid, K(node_page_state(nid, NR_WRITEBACK)), @@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE) + node_page_state(nid, NR_SLAB_UNRECLAIMABLE)), nid, K(node_page_state(nid, NR_SLAB_RECLAIMABLE)), -#ifdef CONFIG_TRANSPARENT_HUGEPAGE nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE)) - , nid, - K(node_page_state(nid, NR_ANON_TRANSPARENT_HUGEPAGES) * - HPAGE_PMD_NR)); -#else - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ,nid, K(node_page_state(nid, + NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) +#endif +#ifdef CONFIG_MEMORY_BALLOON + ,nid, K(node_page_state(nid, NR_BALLOON_PAGES)) #endif + ); n += hugetlb_report_node_meminfo(nid, buf + n); return n; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index aa1eee0..f897fbf 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE AnonHugePages: %8lu kB\n #endif +#ifdef CONFIG_MEMORY_BALLOON + BalloonPages: %8lu kB\n +#endif , K(i.totalram), K(i.freeram), @@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR) #endif +#ifdef CONFIG_MEMORY_BALLOON + ,K(global_page_state(NR_BALLOON_PAGES)) +#endif ); hugetlb_report_meminfo(m); diff --git a/fs/proc/page.c b/fs/proc/page.c index e647c55..1e3187d 100644 --- a/fs/proc/page.c +++ b/fs/proc/page.c @@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page) if (PageBuddy(page)) u |= 1
