Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
On 11/04/2017 07:28 PM, Tetsuo Handa wrote:
Wei Wang wrote:
On 11/03/2017 07:25 PM, Tetsuo Handa wrote:
If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not
use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked
on vb->balloon_lock mutex.
OK. Since the preload() doesn't need too much memory (< 4K in total),
how about GFP_NOWAIT here?
Maybe GFP_NOWAIT | __GFP_NOWARN ?
Sounds good to me. I also plan to move "xb_set_page()" under mutex_lock,
that is,
fill_balloon()
{
...
mutex_lock(>balloon_lock);
vb->num_pfns = 0;
while ((page = balloon_page_pop())) {
==>xb_set_page(..,page,..);
balloon_page_enqueue(>vb_dev_info, page);
...
}
As explained in the xbitmap patch, we need the lock to avoid concurrent
access to the bitmap.
Best,
Wei
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
On 11/04/2017 07:28 PM, Tetsuo Handa wrote:
Wei Wang wrote:
On 11/03/2017 07:25 PM, Tetsuo Handa wrote:
If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not
use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked
on vb->balloon_lock mutex.
OK. Since the preload() doesn't need too much memory (< 4K in total),
how about GFP_NOWAIT here?
Maybe GFP_NOWAIT | __GFP_NOWARN ?
Sounds good to me. I also plan to move "xb_set_page()" under mutex_lock,
that is,
fill_balloon()
{
...
mutex_lock(>balloon_lock);
vb->num_pfns = 0;
while ((page = balloon_page_pop())) {
==>xb_set_page(..,page,..);
balloon_page_enqueue(>vb_dev_info, page);
...
}
As explained in the xbitmap patch, we need the lock to avoid concurrent
access to the bitmap.
Best,
Wei
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
Wei Wang wrote:
> On 11/03/2017 07:25 PM, Tetsuo Handa wrote:
> >> @@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon
> >> *vb, size_t num)
> >>
> >>num_allocated_pages = vb->num_pfns;
> >>/* Did we get any? */
> >> - if (vb->num_pfns != 0)
> >> - tell_host(vb, vb->inflate_vq);
> >> + if (vb->num_pfns) {
> >> + if (use_sg)
> >> + tell_host_sgs(vb, vb->inflate_vq, pfn_min, pfn_max);
> > Please describe why tell_host_sgs() can work without __GFP_DIRECT_RECLAIM
> > allocation,
> > for tell_host_sgs() is called with vb->balloon_lock mutex held.
>
> Essentially,
> tell_host_sgs()-->send_balloon_page_sg()-->add_one_sg()-->virtqueue_add_inbuf(
>
> , , num=1 ,,GFP_KERNEL)
> won't need any memory allocation, because we always add one sg (i.e.
> num=1) each time. That memory
> allocation option is only used when multiple sgs are added (i.e. num >
> 1) and the implementation inside virtqueue_add_inbuf
> need allocation of indirect descriptor table.
>
> We could also add some comments above the function to explain a little
> about this if necessary.
Yes, please do so.
Or maybe replace GFP_KERNEL with GFP_NOWAIT or 0. Though Michael might remove
that GFP
argument (
http://lkml.kernel.org/r/201710022344.jii17368.hqtlomjoosf...@i-love.sakura.ne.jp
).
> > If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must
> > not
> > use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be
> > blocked
> > on vb->balloon_lock mutex.
>
> OK. Since the preload() doesn't need too much memory (< 4K in total),
> how about GFP_NOWAIT here?
Maybe GFP_NOWAIT | __GFP_NOWARN ?
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
Wei Wang wrote:
> On 11/03/2017 07:25 PM, Tetsuo Handa wrote:
> >> @@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon
> >> *vb, size_t num)
> >>
> >>num_allocated_pages = vb->num_pfns;
> >>/* Did we get any? */
> >> - if (vb->num_pfns != 0)
> >> - tell_host(vb, vb->inflate_vq);
> >> + if (vb->num_pfns) {
> >> + if (use_sg)
> >> + tell_host_sgs(vb, vb->inflate_vq, pfn_min, pfn_max);
> > Please describe why tell_host_sgs() can work without __GFP_DIRECT_RECLAIM
> > allocation,
> > for tell_host_sgs() is called with vb->balloon_lock mutex held.
>
> Essentially,
> tell_host_sgs()-->send_balloon_page_sg()-->add_one_sg()-->virtqueue_add_inbuf(
>
> , , num=1 ,,GFP_KERNEL)
> won't need any memory allocation, because we always add one sg (i.e.
> num=1) each time. That memory
> allocation option is only used when multiple sgs are added (i.e. num >
> 1) and the implementation inside virtqueue_add_inbuf
> need allocation of indirect descriptor table.
>
> We could also add some comments above the function to explain a little
> about this if necessary.
Yes, please do so.
Or maybe replace GFP_KERNEL with GFP_NOWAIT or 0. Though Michael might remove
that GFP
argument (
http://lkml.kernel.org/r/201710022344.jii17368.hqtlomjoosf...@i-love.sakura.ne.jp
).
> > If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must
> > not
> > use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be
> > blocked
> > on vb->balloon_lock mutex.
>
> OK. Since the preload() doesn't need too much memory (< 4K in total),
> how about GFP_NOWAIT here?
Maybe GFP_NOWAIT | __GFP_NOWARN ?
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
On 11/03/2017 07:25 PM, Tetsuo Handa wrote:
Wei Wang wrote:
@@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
size_t num)
break;
}
+ if (use_sg && xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
Right, thanks, will add __free_page(page) here.
@@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
size_t num)
num_allocated_pages = vb->num_pfns;
/* Did we get any? */
- if (vb->num_pfns != 0)
- tell_host(vb, vb->inflate_vq);
+ if (vb->num_pfns) {
+ if (use_sg)
+ tell_host_sgs(vb, vb->inflate_vq, pfn_min, pfn_max);
Please describe why tell_host_sgs() can work without __GFP_DIRECT_RECLAIM
allocation,
for tell_host_sgs() is called with vb->balloon_lock mutex held.
Essentially,
tell_host_sgs()-->send_balloon_page_sg()-->add_one_sg()-->virtqueue_add_inbuf(
, , num=1 ,,GFP_KERNEL)
won't need any memory allocation, because we always add one sg (i.e.
num=1) each time. That memory
allocation option is only used when multiple sgs are added (i.e. num >
1) and the implementation inside virtqueue_add_inbuf
need allocation of indirect descriptor table.
We could also add some comments above the function to explain a little
about this if necessary.
@@ -223,7 +353,13 @@ static unsigned leak_balloon(struct virtio_balloon *vb,
size_t num)
page = balloon_page_dequeue(vb_dev_info);
if (!page)
break;
- set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
+ if (use_sg) {
+ if (xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
Yes, will make it:
if (xb_set_page(vb, page, _min, _max) < 0) {
balloon_page_enqueue(..., page);
break;
}
If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not
use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked
on vb->balloon_lock mutex.
OK. Since the preload() doesn't need too much memory (< 4K in total),
how about GFP_NOWAIT here?
Best,
Wei
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
On 11/03/2017 07:25 PM, Tetsuo Handa wrote:
Wei Wang wrote:
@@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
size_t num)
break;
}
+ if (use_sg && xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
Right, thanks, will add __free_page(page) here.
@@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
size_t num)
num_allocated_pages = vb->num_pfns;
/* Did we get any? */
- if (vb->num_pfns != 0)
- tell_host(vb, vb->inflate_vq);
+ if (vb->num_pfns) {
+ if (use_sg)
+ tell_host_sgs(vb, vb->inflate_vq, pfn_min, pfn_max);
Please describe why tell_host_sgs() can work without __GFP_DIRECT_RECLAIM
allocation,
for tell_host_sgs() is called with vb->balloon_lock mutex held.
Essentially,
tell_host_sgs()-->send_balloon_page_sg()-->add_one_sg()-->virtqueue_add_inbuf(
, , num=1 ,,GFP_KERNEL)
won't need any memory allocation, because we always add one sg (i.e.
num=1) each time. That memory
allocation option is only used when multiple sgs are added (i.e. num >
1) and the implementation inside virtqueue_add_inbuf
need allocation of indirect descriptor table.
We could also add some comments above the function to explain a little
about this if necessary.
@@ -223,7 +353,13 @@ static unsigned leak_balloon(struct virtio_balloon *vb,
size_t num)
page = balloon_page_dequeue(vb_dev_info);
if (!page)
break;
- set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
+ if (use_sg) {
+ if (xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
Yes, will make it:
if (xb_set_page(vb, page, _min, _max) < 0) {
balloon_page_enqueue(..., page);
break;
}
If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not
use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked
on vb->balloon_lock mutex.
OK. Since the preload() doesn't need too much memory (< 4K in total),
how about GFP_NOWAIT here?
Best,
Wei
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
Wei Wang wrote:
> @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
> size_t num)
> break;
> }
>
> + if (use_sg && xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
> + break;
> balloon_page_push(, page);
> }
>
> @@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
> size_t num)
>
> num_allocated_pages = vb->num_pfns;
> /* Did we get any? */
> - if (vb->num_pfns != 0)
> - tell_host(vb, vb->inflate_vq);
> + if (vb->num_pfns) {
> + if (use_sg)
> + tell_host_sgs(vb, vb->inflate_vq, pfn_min, pfn_max);
Please describe why tell_host_sgs() can work without __GFP_DIRECT_RECLAIM
allocation,
for tell_host_sgs() is called with vb->balloon_lock mutex held.
> + else
> + tell_host(vb, vb->inflate_vq);
> + }
> mutex_unlock(>balloon_lock);
>
> return num_allocated_pages;
> @@ -223,7 +353,13 @@ static unsigned leak_balloon(struct virtio_balloon *vb,
> size_t num)
> page = balloon_page_dequeue(vb_dev_info);
> if (!page)
> break;
> - set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
> + if (use_sg) {
> + if (xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not
use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked
on vb->balloon_lock mutex.
> + break;
> + } else {
> + set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
> + }
> +
> list_add(>lru, );
> vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
> }
Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG
Wei Wang wrote:
> @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
> size_t num)
> break;
> }
>
> + if (use_sg && xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
> + break;
> balloon_page_push(, page);
> }
>
> @@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon *vb,
> size_t num)
>
> num_allocated_pages = vb->num_pfns;
> /* Did we get any? */
> - if (vb->num_pfns != 0)
> - tell_host(vb, vb->inflate_vq);
> + if (vb->num_pfns) {
> + if (use_sg)
> + tell_host_sgs(vb, vb->inflate_vq, pfn_min, pfn_max);
Please describe why tell_host_sgs() can work without __GFP_DIRECT_RECLAIM
allocation,
for tell_host_sgs() is called with vb->balloon_lock mutex held.
> + else
> + tell_host(vb, vb->inflate_vq);
> + }
> mutex_unlock(>balloon_lock);
>
> return num_allocated_pages;
> @@ -223,7 +353,13 @@ static unsigned leak_balloon(struct virtio_balloon *vb,
> size_t num)
> page = balloon_page_dequeue(vb_dev_info);
> if (!page)
> break;
> - set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
> + if (use_sg) {
> + if (xb_set_page(vb, page, _min, _max) < 0)
Isn't this leaking "page" ?
If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not
use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked
on vb->balloon_lock mutex.
> + break;
> + } else {
> + set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
> + }
> +
> list_add(>lru, );
> vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
> }
