Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/22/2018 02:27 PM, Oleksandr Andrushchenko wrote:
> On 05/22/2018 09:02 PM, Boris Ostrovsky wrote:
>> On 05/22/2018 11:00 AM, Oleksandr Andrushchenko wrote:
>>> On 05/22/2018 05:33 PM, Boris Ostrovsky wrote:
 On 05/22/2018 01:55 AM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 11:36 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:
>>> On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:
 On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:
>>> On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:
 On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko
> 
 A commit message would be useful.
>>> Sure, v1 will have it
> Signed-off-by: Oleksandr Andrushchenko
> 
>
>    for (i = 0; i < nr_pages; i++) {
> -    page = alloc_page(gfp);
> -    if (page == NULL) {
> -    nr_pages = i;
> -    state = BP_EAGAIN;
> -    break;
> +    if (ext_pages) {
> +    page = ext_pages[i];
> +    } else {
> +    page = alloc_page(gfp);
> +    if (page == NULL) {
> +    nr_pages = i;
> +    state = BP_EAGAIN;
> +    break;
> +    }
>    }
>    scrub_page(page);
>    list_add(>lru, );
> @@ -529,7 +565,7 @@ static enum bp_state
> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>    i = 0;
>    list_for_each_entry_safe(page, tmp, , lru) {
>    /* XENMEM_decrease_reservation requires a
> GFN */
> -    frame_list[i++] = xen_page_to_gfn(page);
> +    frames[i++] = xen_page_to_gfn(page);
>      #ifdef CONFIG_XEN_HAVE_PVMMU
>    /*
> @@ -552,18 +588,22 @@ static enum bp_state
> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>    #endif
>    list_del(>lru);
>    -    balloon_append(page);
> +    if (!ext_pages)
> +    balloon_append(page);
 So what you are proposing is not really ballooning. You are
 just
 piggybacking on existing interfaces, aren't you?
>>> Sort of. Basically I need to
>>> {increase|decrease}_reservation, not
>>> actually
>>> allocating ballooned pages.
>>> Do you think I can simply EXPORT_SYMBOL for
>>> {increase|decrease}_reservation?
>>> Any other suggestion?
>> I am actually wondering how much of that code you end up
>> reusing.
>> You
>> pretty much create new code paths in both routines and common
>> code
>> ends
>> up being essentially the hypercall.
> Well, I hoped that it would be easier to maintain if I modify
> existing
> code
> to support both use-cases, but I am also ok to create new
> routines if
> this
> seems to be reasonable - please let me know
>>   So the question is --- would it make
>> sense to do all of this separately from the balloon driver?
> This can be done, but which driver will host this code then?
> If we
> move from
> the balloon driver, then this could go to either gntdev or
> grant-table.
> What's your preference?
 A separate module?
 Is there any use for this feature outside of your zero-copy DRM
 driver?
>>> Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.
>>>
>>> At the time I tried to upstream zcopy driver it was discussed and
>>> decided that
>>> it would be better if I remove all DRM specific code and move it to
>>> Xen drivers.
>>> Thus, this RFC.
>>>
>>> But it can also be implemented as a dedicated Xen dma-buf driver
>>> which
>>> will have all the
>>> code from this RFC + a bit more (char/misc device handling at
>>> least).
>>> This will also require a dedicated user-space library, just like
>>> libxengnttab.so
>>> for gntdev (now I have all new IOCTLs covered there).
>>>
>>> If the idea of a dedicated Xen dma-buf driver seems to be more
>>> attractive we
>>> can work toward this solution. BTW, I do support this idea, but
>>> was not
>>> sure if Xen 

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

On 05/22/2018 09:02 PM, Boris Ostrovsky wrote:

On 05/22/2018 11:00 AM, Oleksandr Andrushchenko wrote:

On 05/22/2018 05:33 PM, Boris Ostrovsky wrote:

On 05/22/2018 01:55 AM, Oleksandr Andrushchenko wrote:

On 05/21/2018 11:36 PM, Boris Ostrovsky wrote:

On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:

On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko


A commit message would be useful.

Sure, v1 will have it

Signed-off-by: Oleksandr Andrushchenko


   for (i = 0; i < nr_pages; i++) {
-    page = alloc_page(gfp);
-    if (page == NULL) {
-    nr_pages = i;
-    state = BP_EAGAIN;
-    break;
+    if (ext_pages) {
+    page = ext_pages[i];
+    } else {
+    page = alloc_page(gfp);
+    if (page == NULL) {
+    nr_pages = i;
+    state = BP_EAGAIN;
+    break;
+    }
   }
   scrub_page(page);
   list_add(>lru, );
@@ -529,7 +565,7 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   i = 0;
   list_for_each_entry_safe(page, tmp, , lru) {
   /* XENMEM_decrease_reservation requires a GFN */
-    frame_list[i++] = xen_page_to_gfn(page);
+    frames[i++] = xen_page_to_gfn(page);
     #ifdef CONFIG_XEN_HAVE_PVMMU
   /*
@@ -552,18 +588,22 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   #endif
   list_del(>lru);
   -    balloon_append(page);
+    if (!ext_pages)
+    balloon_append(page);

So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

Sort of. Basically I need to {increase|decrease}_reservation, not
actually
allocating ballooned pages.
Do you think I can simply EXPORT_SYMBOL for
{increase|decrease}_reservation?
Any other suggestion?

I am actually wondering how much of that code you end up reusing.
You
pretty much create new code paths in both routines and common code
ends
up being essentially the hypercall.

Well, I hoped that it would be easier to maintain if I modify
existing
code
to support both use-cases, but I am also ok to create new
routines if
this
seems to be reasonable - please let me know

  So the question is --- would it make
sense to do all of this separately from the balloon driver?

This can be done, but which driver will host this code then? If we
move from
the balloon driver, then this could go to either gntdev or
grant-table.
What's your preference?

A separate module?
Is there any use for this feature outside of your zero-copy DRM
driver?

Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.

At the time I tried to upstream zcopy driver it was discussed and
decided that
it would be better if I remove all DRM specific code and move it to
Xen drivers.
Thus, this RFC.

But it can also be implemented as a dedicated Xen dma-buf driver
which
will have all the
code from this RFC + a bit more (char/misc device handling at least).
This will also require a dedicated user-space library, just like
libxengnttab.so
for gntdev (now I have all new IOCTLs covered there).

If the idea of a dedicated Xen dma-buf driver seems to be more
attractive we
can work toward this solution. BTW, I do support this idea, but
was not
sure if Xen community accepts yet another driver which duplicates
quite some code
of the existing gntdev/balloon/grant-table. And now after this RFC I
hope that all cons
and pros of both dedicated driver and gntdev/balloon/grant-table
extension are
clearly seen and we can make a decision.

IIRC the objection for a separate module was in the context of gntdev
was discussion, because (among other things) people didn't want to
have
yet another file in /dev/xen/

Here we are talking about (a new) balloon-like module which doesn't
create any new user-visible interfaces. And as for duplicating code
---
as I said, I am not convinced there is much of duplication.

I might even argue that we should add a new config option for this
module.

I am not quite sure I am fully following you here: so, you suggest
that we have balloon.c unchanged, but instead create a new
module (namely a file under the same folder as balloon.c, e.g.
dma-buf-reservation.c) and move those {increase|decrease}_reservation
routines (specific to dma-buf) to that new file? And make it selectable
via Kconfig? If so, then how about the changes to grant-table and
gntdev?
Those will look inconsistent then.

Inconsistent with what? The changes to grant code will also be under the
new config option.

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/22/2018 11:00 AM, Oleksandr Andrushchenko wrote:
> On 05/22/2018 05:33 PM, Boris Ostrovsky wrote:
>> On 05/22/2018 01:55 AM, Oleksandr Andrushchenko wrote:
>>> On 05/21/2018 11:36 PM, Boris Ostrovsky wrote:
 On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:
>>> On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:
 On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:
> On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:
>> On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko
>>> 
>> A commit message would be useful.
> Sure, v1 will have it
>>> Signed-off-by: Oleksandr Andrushchenko
>>> 
>>>
>>>   for (i = 0; i < nr_pages; i++) {
>>> -    page = alloc_page(gfp);
>>> -    if (page == NULL) {
>>> -    nr_pages = i;
>>> -    state = BP_EAGAIN;
>>> -    break;
>>> +    if (ext_pages) {
>>> +    page = ext_pages[i];
>>> +    } else {
>>> +    page = alloc_page(gfp);
>>> +    if (page == NULL) {
>>> +    nr_pages = i;
>>> +    state = BP_EAGAIN;
>>> +    break;
>>> +    }
>>>   }
>>>   scrub_page(page);
>>>   list_add(>lru, );
>>> @@ -529,7 +565,7 @@ static enum bp_state
>>> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>>>   i = 0;
>>>   list_for_each_entry_safe(page, tmp, , lru) {
>>>   /* XENMEM_decrease_reservation requires a GFN */
>>> -    frame_list[i++] = xen_page_to_gfn(page);
>>> +    frames[i++] = xen_page_to_gfn(page);
>>>     #ifdef CONFIG_XEN_HAVE_PVMMU
>>>   /*
>>> @@ -552,18 +588,22 @@ static enum bp_state
>>> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>>>   #endif
>>>   list_del(>lru);
>>>   -    balloon_append(page);
>>> +    if (!ext_pages)
>>> +    balloon_append(page);
>> So what you are proposing is not really ballooning. You are just
>> piggybacking on existing interfaces, aren't you?
> Sort of. Basically I need to {increase|decrease}_reservation, not
> actually
> allocating ballooned pages.
> Do you think I can simply EXPORT_SYMBOL for
> {increase|decrease}_reservation?
> Any other suggestion?
 I am actually wondering how much of that code you end up reusing.
 You
 pretty much create new code paths in both routines and common code
 ends
 up being essentially the hypercall.
>>> Well, I hoped that it would be easier to maintain if I modify
>>> existing
>>> code
>>> to support both use-cases, but I am also ok to create new
>>> routines if
>>> this
>>> seems to be reasonable - please let me know
  So the question is --- would it make
 sense to do all of this separately from the balloon driver?
>>> This can be done, but which driver will host this code then? If we
>>> move from
>>> the balloon driver, then this could go to either gntdev or
>>> grant-table.
>>> What's your preference?
>> A separate module?
>> Is there any use for this feature outside of your zero-copy DRM
>> driver?
> Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.
>
> At the time I tried to upstream zcopy driver it was discussed and
> decided that
> it would be better if I remove all DRM specific code and move it to
> Xen drivers.
> Thus, this RFC.
>
> But it can also be implemented as a dedicated Xen dma-buf driver
> which
> will have all the
> code from this RFC + a bit more (char/misc device handling at least).
> This will also require a dedicated user-space library, just like
> libxengnttab.so
> for gntdev (now I have all new IOCTLs covered there).
>
> If the idea of a dedicated Xen dma-buf driver seems to be more
> attractive we
> can work toward this solution. BTW, I do support this idea, but
> was not
> sure if Xen community accepts yet another driver which duplicates
> quite some code
> of the existing gntdev/balloon/grant-table. And now after this RFC I
> hope that all cons
> and pros of both dedicated driver and gntdev/balloon/grant-table
> extension are
> clearly seen and we can make a decision.
 IIRC the objection for a separate module was in the context of gntdev
 was discussion, 

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

On 05/22/2018 05:33 PM, Boris Ostrovsky wrote:

On 05/22/2018 01:55 AM, Oleksandr Andrushchenko wrote:

On 05/21/2018 11:36 PM, Boris Ostrovsky wrote:

On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:

On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

A commit message would be useful.

Sure, v1 will have it

Signed-off-by: Oleksandr Andrushchenko


  for (i = 0; i < nr_pages; i++) {
-    page = alloc_page(gfp);
-    if (page == NULL) {
-    nr_pages = i;
-    state = BP_EAGAIN;
-    break;
+    if (ext_pages) {
+    page = ext_pages[i];
+    } else {
+    page = alloc_page(gfp);
+    if (page == NULL) {
+    nr_pages = i;
+    state = BP_EAGAIN;
+    break;
+    }
  }
  scrub_page(page);
  list_add(>lru, );
@@ -529,7 +565,7 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
  i = 0;
  list_for_each_entry_safe(page, tmp, , lru) {
  /* XENMEM_decrease_reservation requires a GFN */
-    frame_list[i++] = xen_page_to_gfn(page);
+    frames[i++] = xen_page_to_gfn(page);
    #ifdef CONFIG_XEN_HAVE_PVMMU
  /*
@@ -552,18 +588,22 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
  #endif
  list_del(>lru);
  -    balloon_append(page);
+    if (!ext_pages)
+    balloon_append(page);

So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

Sort of. Basically I need to {increase|decrease}_reservation, not
actually
allocating ballooned pages.
Do you think I can simply EXPORT_SYMBOL for
{increase|decrease}_reservation?
Any other suggestion?

I am actually wondering how much of that code you end up reusing.
You
pretty much create new code paths in both routines and common code
ends
up being essentially the hypercall.

Well, I hoped that it would be easier to maintain if I modify
existing
code
to support both use-cases, but I am also ok to create new routines if
this
seems to be reasonable - please let me know

     So the question is --- would it make
sense to do all of this separately from the balloon driver?

This can be done, but which driver will host this code then? If we
move from
the balloon driver, then this could go to either gntdev or
grant-table.
What's your preference?

A separate module?
Is there any use for this feature outside of your zero-copy DRM
driver?

Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.

At the time I tried to upstream zcopy driver it was discussed and
decided that
it would be better if I remove all DRM specific code and move it to
Xen drivers.
Thus, this RFC.

But it can also be implemented as a dedicated Xen dma-buf driver which
will have all the
code from this RFC + a bit more (char/misc device handling at least).
This will also require a dedicated user-space library, just like
libxengnttab.so
for gntdev (now I have all new IOCTLs covered there).

If the idea of a dedicated Xen dma-buf driver seems to be more
attractive we
can work toward this solution. BTW, I do support this idea, but was not
sure if Xen community accepts yet another driver which duplicates
quite some code
of the existing gntdev/balloon/grant-table. And now after this RFC I
hope that all cons
and pros of both dedicated driver and gntdev/balloon/grant-table
extension are
clearly seen and we can make a decision.

IIRC the objection for a separate module was in the context of gntdev
was discussion, because (among other things) people didn't want to have
yet another file in /dev/xen/

Here we are talking about (a new) balloon-like module which doesn't
create any new user-visible interfaces. And as for duplicating code ---
as I said, I am not convinced there is much of duplication.

I might even argue that we should add a new config option for this
module.

I am not quite sure I am fully following you here: so, you suggest
that we have balloon.c unchanged, but instead create a new
module (namely a file under the same folder as balloon.c, e.g.
dma-buf-reservation.c) and move those {increase|decrease}_reservation
routines (specific to dma-buf) to that new file? And make it selectable
via Kconfig? If so, then how about the changes to grant-table and gntdev?
Those will look inconsistent then.

Inconsistent with what? The changes to grant code will also be under the
new config option.

Ah, ok.

Option 1. We will have Kconfig option which will cover dma-buf
changes in balloon, grant-table and gntdev. 

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/22/2018 01:55 AM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 11:36 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:
>>> On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:
 On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:
>>> On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:
 On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
 A commit message would be useful.
>>> Sure, v1 will have it
> Signed-off-by: Oleksandr Andrushchenko
> 
>
>  for (i = 0; i < nr_pages; i++) {
> -    page = alloc_page(gfp);
> -    if (page == NULL) {
> -    nr_pages = i;
> -    state = BP_EAGAIN;
> -    break;
> +    if (ext_pages) {
> +    page = ext_pages[i];
> +    } else {
> +    page = alloc_page(gfp);
> +    if (page == NULL) {
> +    nr_pages = i;
> +    state = BP_EAGAIN;
> +    break;
> +    }
>  }
>  scrub_page(page);
>  list_add(>lru, );
> @@ -529,7 +565,7 @@ static enum bp_state
> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>  i = 0;
>  list_for_each_entry_safe(page, tmp, , lru) {
>  /* XENMEM_decrease_reservation requires a GFN */
> -    frame_list[i++] = xen_page_to_gfn(page);
> +    frames[i++] = xen_page_to_gfn(page);
>    #ifdef CONFIG_XEN_HAVE_PVMMU
>  /*
> @@ -552,18 +588,22 @@ static enum bp_state
> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>  #endif
>  list_del(>lru);
>  -    balloon_append(page);
> +    if (!ext_pages)
> +    balloon_append(page);
 So what you are proposing is not really ballooning. You are just
 piggybacking on existing interfaces, aren't you?
>>> Sort of. Basically I need to {increase|decrease}_reservation, not
>>> actually
>>> allocating ballooned pages.
>>> Do you think I can simply EXPORT_SYMBOL for
>>> {increase|decrease}_reservation?
>>> Any other suggestion?
>> I am actually wondering how much of that code you end up reusing.
>> You
>> pretty much create new code paths in both routines and common code
>> ends
>> up being essentially the hypercall.
> Well, I hoped that it would be easier to maintain if I modify
> existing
> code
> to support both use-cases, but I am also ok to create new routines if
> this
> seems to be reasonable - please let me know
>>     So the question is --- would it make
>> sense to do all of this separately from the balloon driver?
> This can be done, but which driver will host this code then? If we
> move from
> the balloon driver, then this could go to either gntdev or
> grant-table.
> What's your preference?
 A separate module?
 Is there any use for this feature outside of your zero-copy DRM
 driver?
>>> Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.
>>>
>>> At the time I tried to upstream zcopy driver it was discussed and
>>> decided that
>>> it would be better if I remove all DRM specific code and move it to
>>> Xen drivers.
>>> Thus, this RFC.
>>>
>>> But it can also be implemented as a dedicated Xen dma-buf driver which
>>> will have all the
>>> code from this RFC + a bit more (char/misc device handling at least).
>>> This will also require a dedicated user-space library, just like
>>> libxengnttab.so
>>> for gntdev (now I have all new IOCTLs covered there).
>>>
>>> If the idea of a dedicated Xen dma-buf driver seems to be more
>>> attractive we
>>> can work toward this solution. BTW, I do support this idea, but was not
>>> sure if Xen community accepts yet another driver which duplicates
>>> quite some code
>>> of the existing gntdev/balloon/grant-table. And now after this RFC I
>>> hope that all cons
>>> and pros of both dedicated driver and gntdev/balloon/grant-table
>>> extension are
>>> clearly seen and we can make a decision.
>>
>> IIRC the objection for a separate module was in the context of gntdev
>> was discussion, because (among other things) people didn't want to have
>> yet another file in /dev/xen/
>>
>> Here we are talking about (a new) balloon-like module which doesn't
>> create any new user-visible interfaces. And as for duplicating code ---
>> as I said, I am not convinced there is much of duplication.
>>
>> I might even argue that we should add a new 

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

On 05/21/2018 11:36 PM, Boris Ostrovsky wrote:

On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:

On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

A commit message would be useful.

Sure, v1 will have it

Signed-off-by: Oleksandr Andrushchenko


     for (i = 0; i < nr_pages; i++) {
-    page = alloc_page(gfp);
-    if (page == NULL) {
-    nr_pages = i;
-    state = BP_EAGAIN;
-    break;
+    if (ext_pages) {
+    page = ext_pages[i];
+    } else {
+    page = alloc_page(gfp);
+    if (page == NULL) {
+    nr_pages = i;
+    state = BP_EAGAIN;
+    break;
+    }
     }
     scrub_page(page);
     list_add(>lru, );
@@ -529,7 +565,7 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
     i = 0;
     list_for_each_entry_safe(page, tmp, , lru) {
     /* XENMEM_decrease_reservation requires a GFN */
-    frame_list[i++] = xen_page_to_gfn(page);
+    frames[i++] = xen_page_to_gfn(page);
       #ifdef CONFIG_XEN_HAVE_PVMMU
     /*
@@ -552,18 +588,22 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
     #endif
     list_del(>lru);
     -    balloon_append(page);
+    if (!ext_pages)
+    balloon_append(page);

So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

Sort of. Basically I need to {increase|decrease}_reservation, not
actually
allocating ballooned pages.
Do you think I can simply EXPORT_SYMBOL for
{increase|decrease}_reservation?
Any other suggestion?

I am actually wondering how much of that code you end up reusing. You
pretty much create new code paths in both routines and common code
ends
up being essentially the hypercall.

Well, I hoped that it would be easier to maintain if I modify existing
code
to support both use-cases, but I am also ok to create new routines if
this
seems to be reasonable - please let me know

    So the question is --- would it make
sense to do all of this separately from the balloon driver?

This can be done, but which driver will host this code then? If we
move from
the balloon driver, then this could go to either gntdev or grant-table.
What's your preference?

A separate module?
Is there any use for this feature outside of your zero-copy DRM driver?

Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.

At the time I tried to upstream zcopy driver it was discussed and
decided that
it would be better if I remove all DRM specific code and move it to
Xen drivers.
Thus, this RFC.

But it can also be implemented as a dedicated Xen dma-buf driver which
will have all the
code from this RFC + a bit more (char/misc device handling at least).
This will also require a dedicated user-space library, just like
libxengnttab.so
for gntdev (now I have all new IOCTLs covered there).

If the idea of a dedicated Xen dma-buf driver seems to be more
attractive we
can work toward this solution. BTW, I do support this idea, but was not
sure if Xen community accepts yet another driver which duplicates
quite some code
of the existing gntdev/balloon/grant-table. And now after this RFC I
hope that all cons
and pros of both dedicated driver and gntdev/balloon/grant-table
extension are
clearly seen and we can make a decision.


IIRC the objection for a separate module was in the context of gntdev
was discussion, because (among other things) people didn't want to have
yet another file in /dev/xen/

Here we are talking about (a new) balloon-like module which doesn't
create any new user-visible interfaces. And as for duplicating code ---
as I said, I am not convinced there is much of duplication.

I might even argue that we should add a new config option for this module.

I am not quite sure I am fully following you here: so, you suggest
that we have balloon.c unchanged, but instead create a new
module (namely a file under the same folder as balloon.c, e.g.
dma-buf-reservation.c) and move those {increase|decrease}_reservation
routines (specific to dma-buf) to that new file? And make it selectable
via Kconfig? If so, then how about the changes to grant-table and gntdev?
Those will look inconsistent then.

If you suggest a new kernel driver module:
IMO, there is nothing bad if we create a dedicated kernel module
(driver) for Xen dma-buf handling selectable under Kconfig option.
Yes, this will create a yet another device under /dev/xen,
but most people will never see it if we set Kconfig to default to "n".
And then we'll need 

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/21/2018 03:13 PM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:
>>> On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:
 On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:
> On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:
>> On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko 
>> A commit message would be useful.
> Sure, v1 will have it
>>> Signed-off-by: Oleksandr Andrushchenko
>>> 
>>>
>>>     for (i = 0; i < nr_pages; i++) {
>>> -    page = alloc_page(gfp);
>>> -    if (page == NULL) {
>>> -    nr_pages = i;
>>> -    state = BP_EAGAIN;
>>> -    break;
>>> +    if (ext_pages) {
>>> +    page = ext_pages[i];
>>> +    } else {
>>> +    page = alloc_page(gfp);
>>> +    if (page == NULL) {
>>> +    nr_pages = i;
>>> +    state = BP_EAGAIN;
>>> +    break;
>>> +    }
>>>     }
>>>     scrub_page(page);
>>>     list_add(>lru, );
>>> @@ -529,7 +565,7 @@ static enum bp_state
>>> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>>>     i = 0;
>>>     list_for_each_entry_safe(page, tmp, , lru) {
>>>     /* XENMEM_decrease_reservation requires a GFN */
>>> -    frame_list[i++] = xen_page_to_gfn(page);
>>> +    frames[i++] = xen_page_to_gfn(page);
>>>       #ifdef CONFIG_XEN_HAVE_PVMMU
>>>     /*
>>> @@ -552,18 +588,22 @@ static enum bp_state
>>> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>>>     #endif
>>>     list_del(>lru);
>>>     -    balloon_append(page);
>>> +    if (!ext_pages)
>>> +    balloon_append(page);
>> So what you are proposing is not really ballooning. You are just
>> piggybacking on existing interfaces, aren't you?
> Sort of. Basically I need to {increase|decrease}_reservation, not
> actually
> allocating ballooned pages.
> Do you think I can simply EXPORT_SYMBOL for
> {increase|decrease}_reservation?
> Any other suggestion?
 I am actually wondering how much of that code you end up reusing. You
 pretty much create new code paths in both routines and common code
 ends
 up being essentially the hypercall.
>>> Well, I hoped that it would be easier to maintain if I modify existing
>>> code
>>> to support both use-cases, but I am also ok to create new routines if
>>> this
>>> seems to be reasonable - please let me know
    So the question is --- would it make
 sense to do all of this separately from the balloon driver?
>>> This can be done, but which driver will host this code then? If we
>>> move from
>>> the balloon driver, then this could go to either gntdev or grant-table.
>>> What's your preference?
>> A separate module?
>
>> Is there any use for this feature outside of your zero-copy DRM driver?
> Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.
>
> At the time I tried to upstream zcopy driver it was discussed and
> decided that
> it would be better if I remove all DRM specific code and move it to
> Xen drivers.
> Thus, this RFC.
>
> But it can also be implemented as a dedicated Xen dma-buf driver which
> will have all the
> code from this RFC + a bit more (char/misc device handling at least).
> This will also require a dedicated user-space library, just like
> libxengnttab.so
> for gntdev (now I have all new IOCTLs covered there).
>
> If the idea of a dedicated Xen dma-buf driver seems to be more
> attractive we
> can work toward this solution. BTW, I do support this idea, but was not
> sure if Xen community accepts yet another driver which duplicates
> quite some code
> of the existing gntdev/balloon/grant-table. And now after this RFC I
> hope that all cons
> and pros of both dedicated driver and gntdev/balloon/grant-table
> extension are
> clearly seen and we can make a decision.


IIRC the objection for a separate module was in the context of gntdev
was discussion, because (among other things) people didn't want to have
yet another file in /dev/xen/

Here we are talking about (a new) balloon-like module which doesn't
create any new user-visible interfaces. And as for duplicating code ---
as I said, I am not convinced there is much of duplication.

I might even argue that we should add a new config option for this module.


-boris

>
>>
>> -boris
> Thank you,
> Oleksandr
> [1]
> https://lists.freedesktop.org/archives/dri-devel/2018-April/173163.html

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

On 05/21/2018 09:53 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:

On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:

On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

A commit message would be useful.

Sure, v1 will have it

Signed-off-by: Oleksandr Andrushchenko


    for (i = 0; i < nr_pages; i++) {
-    page = alloc_page(gfp);
-    if (page == NULL) {
-    nr_pages = i;
-    state = BP_EAGAIN;
-    break;
+    if (ext_pages) {
+    page = ext_pages[i];
+    } else {
+    page = alloc_page(gfp);
+    if (page == NULL) {
+    nr_pages = i;
+    state = BP_EAGAIN;
+    break;
+    }
    }
    scrub_page(page);
    list_add(>lru, );
@@ -529,7 +565,7 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
    i = 0;
    list_for_each_entry_safe(page, tmp, , lru) {
    /* XENMEM_decrease_reservation requires a GFN */
-    frame_list[i++] = xen_page_to_gfn(page);
+    frames[i++] = xen_page_to_gfn(page);
      #ifdef CONFIG_XEN_HAVE_PVMMU
    /*
@@ -552,18 +588,22 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
    #endif
    list_del(>lru);
    -    balloon_append(page);
+    if (!ext_pages)
+    balloon_append(page);

So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

Sort of. Basically I need to {increase|decrease}_reservation, not
actually
allocating ballooned pages.
Do you think I can simply EXPORT_SYMBOL for
{increase|decrease}_reservation?
Any other suggestion?

I am actually wondering how much of that code you end up reusing. You
pretty much create new code paths in both routines and common code ends
up being essentially the hypercall.

Well, I hoped that it would be easier to maintain if I modify existing
code
to support both use-cases, but I am also ok to create new routines if
this
seems to be reasonable - please let me know

   So the question is --- would it make
sense to do all of this separately from the balloon driver?

This can be done, but which driver will host this code then? If we
move from
the balloon driver, then this could go to either gntdev or grant-table.
What's your preference?

A separate module?



Is there any use for this feature outside of your zero-copy DRM driver?

Intel's hyper dma-buf (Dongwon/Matt CC'ed), V4L/GPU at least.

At the time I tried to upstream zcopy driver it was discussed and 
decided that
it would be better if I remove all DRM specific code and move it to Xen 
drivers.

Thus, this RFC.

But it can also be implemented as a dedicated Xen dma-buf driver which 
will have all the

code from this RFC + a bit more (char/misc device handling at least).
This will also require a dedicated user-space library, just like 
libxengnttab.so

for gntdev (now I have all new IOCTLs covered there).

If the idea of a dedicated Xen dma-buf driver seems to be more attractive we
can work toward this solution. BTW, I do support this idea, but was not
sure if Xen community accepts yet another driver which duplicates quite 
some code
of the existing gntdev/balloon/grant-table. And now after this RFC I 
hope that all cons
and pros of both dedicated driver and gntdev/balloon/grant-table 
extension are

clearly seen and we can make a decision.



-boris

Thank you,
Oleksandr
[1] https://lists.freedesktop.org/archives/dri-devel/2018-April/173163.html

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/21/2018 01:32 PM, Oleksandr Andrushchenko wrote:
> On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:
>> On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:
>>> On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:
 On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 
 A commit message would be useful.
>>> Sure, v1 will have it
> Signed-off-by: Oleksandr Andrushchenko
> 
>
>    for (i = 0; i < nr_pages; i++) {
> -    page = alloc_page(gfp);
> -    if (page == NULL) {
> -    nr_pages = i;
> -    state = BP_EAGAIN;
> -    break;
> +    if (ext_pages) {
> +    page = ext_pages[i];
> +    } else {
> +    page = alloc_page(gfp);
> +    if (page == NULL) {
> +    nr_pages = i;
> +    state = BP_EAGAIN;
> +    break;
> +    }
>    }
>    scrub_page(page);
>    list_add(>lru, );
> @@ -529,7 +565,7 @@ static enum bp_state
> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>    i = 0;
>    list_for_each_entry_safe(page, tmp, , lru) {
>    /* XENMEM_decrease_reservation requires a GFN */
> -    frame_list[i++] = xen_page_to_gfn(page);
> +    frames[i++] = xen_page_to_gfn(page);
>      #ifdef CONFIG_XEN_HAVE_PVMMU
>    /*
> @@ -552,18 +588,22 @@ static enum bp_state
> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>    #endif
>    list_del(>lru);
>    -    balloon_append(page);
> +    if (!ext_pages)
> +    balloon_append(page);
 So what you are proposing is not really ballooning. You are just
 piggybacking on existing interfaces, aren't you?
>>> Sort of. Basically I need to {increase|decrease}_reservation, not
>>> actually
>>> allocating ballooned pages.
>>> Do you think I can simply EXPORT_SYMBOL for
>>> {increase|decrease}_reservation?
>>> Any other suggestion?
>>
>> I am actually wondering how much of that code you end up reusing. You
>> pretty much create new code paths in both routines and common code ends
>> up being essentially the hypercall.
> Well, I hoped that it would be easier to maintain if I modify existing
> code
> to support both use-cases, but I am also ok to create new routines if
> this
> seems to be reasonable - please let me know
>>   So the question is --- would it make
>> sense to do all of this separately from the balloon driver?
> This can be done, but which driver will host this code then? If we
> move from
> the balloon driver, then this could go to either gntdev or grant-table.
> What's your preference?

A separate module?

Is there any use for this feature outside of your zero-copy DRM driver?

-boris

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

On 05/21/2018 07:35 PM, Boris Ostrovsky wrote:

On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:

On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

A commit message would be useful.

Sure, v1 will have it

Signed-off-by: Oleksandr Andrushchenko


   for (i = 0; i < nr_pages; i++) {
-    page = alloc_page(gfp);
-    if (page == NULL) {
-    nr_pages = i;
-    state = BP_EAGAIN;
-    break;
+    if (ext_pages) {
+    page = ext_pages[i];
+    } else {
+    page = alloc_page(gfp);
+    if (page == NULL) {
+    nr_pages = i;
+    state = BP_EAGAIN;
+    break;
+    }
   }
   scrub_page(page);
   list_add(>lru, );
@@ -529,7 +565,7 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   i = 0;
   list_for_each_entry_safe(page, tmp, , lru) {
   /* XENMEM_decrease_reservation requires a GFN */
-    frame_list[i++] = xen_page_to_gfn(page);
+    frames[i++] = xen_page_to_gfn(page);
     #ifdef CONFIG_XEN_HAVE_PVMMU
   /*
@@ -552,18 +588,22 @@ static enum bp_state
decrease_reservation(unsigned long nr_pages, gfp_t gfp)
   #endif
   list_del(>lru);
   -    balloon_append(page);
+    if (!ext_pages)
+    balloon_append(page);

So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

Sort of. Basically I need to {increase|decrease}_reservation, not
actually
allocating ballooned pages.
Do you think I can simply EXPORT_SYMBOL for
{increase|decrease}_reservation?
Any other suggestion?


I am actually wondering how much of that code you end up reusing. You
pretty much create new code paths in both routines and common code ends
up being essentially the hypercall.

Well, I hoped that it would be easier to maintain if I modify existing code
to support both use-cases, but I am also ok to create new routines if this
seems to be reasonable - please let me know

  So the question is --- would it make
sense to do all of this separately from the balloon driver?

This can be done, but which driver will host this code then? If we move from
the balloon driver, then this could go to either gntdev or grant-table.
What's your preference?


-boris

Thank you,
Oleksandr

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/21/2018 01:40 AM, Oleksandr Andrushchenko wrote:
> On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:
>> On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
>>> From: Oleksandr Andrushchenko 
>>
>> A commit message would be useful.
> Sure, v1 will have it
>>
>>> Signed-off-by: Oleksandr Andrushchenko
>>> 
>>>
>>>   for (i = 0; i < nr_pages; i++) {
>>> -    page = alloc_page(gfp);
>>> -    if (page == NULL) {
>>> -    nr_pages = i;
>>> -    state = BP_EAGAIN;
>>> -    break;
>>> +    if (ext_pages) {
>>> +    page = ext_pages[i];
>>> +    } else {
>>> +    page = alloc_page(gfp);
>>> +    if (page == NULL) {
>>> +    nr_pages = i;
>>> +    state = BP_EAGAIN;
>>> +    break;
>>> +    }
>>>   }
>>>   scrub_page(page);
>>>   list_add(>lru, );
>>> @@ -529,7 +565,7 @@ static enum bp_state
>>> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>>>   i = 0;
>>>   list_for_each_entry_safe(page, tmp, , lru) {
>>>   /* XENMEM_decrease_reservation requires a GFN */
>>> -    frame_list[i++] = xen_page_to_gfn(page);
>>> +    frames[i++] = xen_page_to_gfn(page);
>>>     #ifdef CONFIG_XEN_HAVE_PVMMU
>>>   /*
>>> @@ -552,18 +588,22 @@ static enum bp_state
>>> decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>>>   #endif
>>>   list_del(>lru);
>>>   -    balloon_append(page);
>>> +    if (!ext_pages)
>>> +    balloon_append(page);
>>
>> So what you are proposing is not really ballooning. You are just
>> piggybacking on existing interfaces, aren't you?
> Sort of. Basically I need to {increase|decrease}_reservation, not
> actually
> allocating ballooned pages.
> Do you think I can simply EXPORT_SYMBOL for
> {increase|decrease}_reservation?
> Any other suggestion?


I am actually wondering how much of that code you end up reusing. You
pretty much create new code paths in both routines and common code ends
up being essentially the hypercall. So the question is --- would it make
sense to do all of this separately from the balloon driver?


-boris

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

On 05/19/2018 01:04 AM, Boris Ostrovsky wrote:

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 


A commit message would be useful.

Sure, v1 will have it



Signed-off-by: Oleksandr Andrushchenko 

for (i = 0; i < nr_pages; i++) {
-   page = alloc_page(gfp);
-   if (page == NULL) {
-   nr_pages = i;
-   state = BP_EAGAIN;
-   break;
+   if (ext_pages) {
+   page = ext_pages[i];
+   } else {
+   page = alloc_page(gfp);
+   if (page == NULL) {
+   nr_pages = i;
+   state = BP_EAGAIN;
+   break;
+   }
}
scrub_page(page);
list_add(>lru, );
@@ -529,7 +565,7 @@ static enum bp_state decrease_reservation(unsigned long 
nr_pages, gfp_t gfp)
i = 0;
list_for_each_entry_safe(page, tmp, , lru) {
/* XENMEM_decrease_reservation requires a GFN */
-   frame_list[i++] = xen_page_to_gfn(page);
+   frames[i++] = xen_page_to_gfn(page);
  
  #ifdef CONFIG_XEN_HAVE_PVMMU

/*
@@ -552,18 +588,22 @@ static enum bp_state decrease_reservation(unsigned long 
nr_pages, gfp_t gfp)
  #endif
list_del(>lru);
  
-		balloon_append(page);

+   if (!ext_pages)
+   balloon_append(page);


So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

Sort of. Basically I need to {increase|decrease}_reservation, not actually
allocating ballooned pages.
Do you think I can simply EXPORT_SYMBOL for {increase|decrease}_reservation?
Any other suggestion?

-boris



Thank you,
Oleksandr

Re: [Xen-devel] [RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Boris Ostrovsky]

On 05/17/2018 04:26 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko 


A commit message would be useful.


>
> Signed-off-by: Oleksandr Andrushchenko 
>
>   for (i = 0; i < nr_pages; i++) {
> - page = alloc_page(gfp);
> - if (page == NULL) {
> - nr_pages = i;
> - state = BP_EAGAIN;
> - break;
> + if (ext_pages) {
> + page = ext_pages[i];
> + } else {
> + page = alloc_page(gfp);
> + if (page == NULL) {
> + nr_pages = i;
> + state = BP_EAGAIN;
> + break;
> + }
>   }
>   scrub_page(page);
>   list_add(>lru, );
> @@ -529,7 +565,7 @@ static enum bp_state decrease_reservation(unsigned long 
> nr_pages, gfp_t gfp)
>   i = 0;
>   list_for_each_entry_safe(page, tmp, , lru) {
>   /* XENMEM_decrease_reservation requires a GFN */
> - frame_list[i++] = xen_page_to_gfn(page);
> + frames[i++] = xen_page_to_gfn(page);
>  
>  #ifdef CONFIG_XEN_HAVE_PVMMU
>   /*
> @@ -552,18 +588,22 @@ static enum bp_state decrease_reservation(unsigned long 
> nr_pages, gfp_t gfp)
>  #endif
>   list_del(>lru);
>  
> - balloon_append(page);
> + if (!ext_pages)
> + balloon_append(page);


So what you are proposing is not really ballooning. You are just
piggybacking on existing interfaces, aren't you?

-boris

[Xen-devel][RFC 1/3] xen/balloon: Allow allocating DMA buffers

[Oleksandr Andrushchenko]

From: Oleksandr Andrushchenko 

Signed-off-by: Oleksandr Andrushchenko 
---
 drivers/xen/balloon.c | 214 +++---
 drivers/xen/xen-balloon.c |   2 +
 include/xen/balloon.h |  11 +-
 3 files changed, 188 insertions(+), 39 deletions(-)

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index e4db19e88ab1..e3a145aa9f29 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -415,8 +415,10 @@ static bool balloon_is_inflated(void)
return balloon_stats.balloon_low || balloon_stats.balloon_high;
 }
 
-static enum bp_state increase_reservation(unsigned long nr_pages)
+static enum bp_state increase_reservation(unsigned long nr_pages,
+ struct page **ext_pages)
 {
+   enum bp_state ret = BP_DONE;
int rc;
unsigned long i;
struct page   *page;
@@ -425,32 +427,49 @@ static enum bp_state increase_reservation(unsigned long 
nr_pages)
.extent_order = EXTENT_ORDER,
.domid= DOMID_SELF
};
+   xen_pfn_t *frames;
 
-   if (nr_pages > ARRAY_SIZE(frame_list))
-   nr_pages = ARRAY_SIZE(frame_list);
+   if (nr_pages > ARRAY_SIZE(frame_list)) {
+   frames = kcalloc(nr_pages, sizeof(xen_pfn_t), GFP_KERNEL);
+   if (!frames)
+   return BP_ECANCELED;
+   } else {
+   frames = frame_list;
+   }
 
-   page = list_first_entry_or_null(_pages, struct page, lru);
-   for (i = 0; i < nr_pages; i++) {
-   if (!page) {
-   nr_pages = i;
-   break;
-   }
+   /* XENMEM_populate_physmap requires a PFN based on Xen
+* granularity.
+*/
+   if (ext_pages) {
+   for (i = 0; i < nr_pages; i++)
+   frames[i] = page_to_xen_pfn(ext_pages[i]);
+   } else {
+   page = list_first_entry_or_null(_pages,
+   struct page, lru);
+   for (i = 0; i < nr_pages; i++) {
+   if (!page) {
+   nr_pages = i;
+   break;
+   }
 
-   /* XENMEM_populate_physmap requires a PFN based on Xen
-* granularity.
-*/
-   frame_list[i] = page_to_xen_pfn(page);
-   page = balloon_next_page(page);
+   frames[i] = page_to_xen_pfn(page);
+   page = balloon_next_page(page);
+   }
}
 
-   set_xen_guest_handle(reservation.extent_start, frame_list);
+   set_xen_guest_handle(reservation.extent_start, frames);
reservation.nr_extents = nr_pages;
rc = HYPERVISOR_memory_op(XENMEM_populate_physmap, );
-   if (rc <= 0)
-   return BP_EAGAIN;
+   if (rc <= 0) {
+   ret = BP_EAGAIN;
+   goto out;
+   }
 
for (i = 0; i < rc; i++) {
-   page = balloon_retrieve(false);
+   if (ext_pages)
+   page = ext_pages[i];
+   else
+   page = balloon_retrieve(false);
BUG_ON(page == NULL);
 
 #ifdef CONFIG_XEN_HAVE_PVMMU
@@ -463,14 +482,14 @@ static enum bp_state increase_reservation(unsigned long 
nr_pages)
if (!xen_feature(XENFEAT_auto_translated_physmap)) {
unsigned long pfn = page_to_pfn(page);
 
-   set_phys_to_machine(pfn, frame_list[i]);
+   set_phys_to_machine(pfn, frames[i]);
 
/* Link back into the page tables if not highmem. */
if (!PageHighMem(page)) {
int ret;
ret = HYPERVISOR_update_va_mapping(
(unsigned long)__va(pfn << 
PAGE_SHIFT),
-   mfn_pte(frame_list[i], 
PAGE_KERNEL),
+   mfn_pte(frames[i], PAGE_KERNEL),
0);
BUG_ON(ret);
}
@@ -478,15 +497,22 @@ static enum bp_state increase_reservation(unsigned long 
nr_pages)
 #endif
 
/* Relinquish the page back to the allocator. */
-   __free_reserved_page(page);
+   if (!ext_pages)
+   __free_reserved_page(page);
}
 
-   balloon_stats.current_pages += rc;
+   if (!ext_pages)
+   balloon_stats.current_pages += rc;
 
-   return BP_DONE;
+out:
+   if (frames != frame_list)
+   kfree(frames);
+
+   return ret;
 }
 
-static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
+static enum bp_state