Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/18/2017 06:18 AM, Matthew Wilcox wrote: On Sun, Dec 17, 2017 at 01:47:21PM +, Wang, Wei W wrote: On Saturday, December 16, 2017 3:22 AM, Matthew Wilcox wrote: On Fri, Dec 15, 2017 at 10:49:15AM -0800, Matthew Wilcox wrote: - xbit_clear() can't return an error. Neither can

Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/16/2017 07:28 PM, Tetsuo Handa wrote: Wei Wang wrote: On 12/16/2017 02:42 AM, Matthew Wilcox wrote: On Tue, Dec 12, 2017 at 07:55:55PM +0800, Wei Wang wrote: +int xb_preload_and_set_bit(struct xb *xb, unsigned long bit, gfp_t gfp); I'm struggling to understand when one would use

Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/15/2017 12:29 AM, Tetsuo Handa wrote: Wei Wang wrote: I used the example of xb_clear_bit_range(), and xb_find_next_bit() is the same fundamentally. Please let me know if anywhere still looks fuzzy. I don't think it is the same for xb_find_next_bit() with set == 0

Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/16/2017 02:42 AM, Matthew Wilcox wrote: On Tue, Dec 12, 2017 at 07:55:55PM +0800, Wei Wang wrote: +int xb_preload_and_set_bit(struct xb *xb, unsigned long bit, gfp_t gfp); I'm struggling to understand when one would use this. The xb_ API requires you to handle your own locking

Re: [PATCH v19 1/7] xbitmap: Introduce xbitmap

On 12/15/2017 09:24 PM, Matthew Wilcox wrote: On Fri, Dec 15, 2017 at 07:05:07PM +0800, kbuild test robot wrote: 21 struct radix_tree_node *node; 22 void **slot; ^^^ missing __rcu annotation here. Wei, could you fold that change into

Re: [virtio-dev] Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/14/2017 11:47 AM, Wei Wang wrote: On 12/13/2017 10:16 PM, Tetsuo Handa wrote: if (set) ret = find_next_bit(, BITS_PER_LONG, ebit); else ret = find_next_zero_bit

Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/13/2017 10:16 PM, Tetsuo Handa wrote: Wei Wang wrote: On 12/12/2017 09:20 PM, Tetsuo Handa wrote: Wei Wang wrote: +void xb_clear_bit_range(struct xb *xb, unsigned long start, unsigned long end) +{ + struct radix_tree_root *root = >xbrt; + struct radix_tree_node *n

Re: [PATCH v19 3/7] xbitmap: add more operations

On 12/12/2017 09:20 PM, Tetsuo Handa wrote: Wei Wang wrote: +void xb_clear_bit_range(struct xb *xb, unsigned long start, unsigned long end) +{ + struct radix_tree_root *root = >xbrt; + struct radix_tree_node *node; + void **slot; + struct ida_bitmap *bit

Re: [PATCH v18 10/10] virtio-balloon: don't report free pages when page poisoning is enabled

On 12/11/2017 09:24 PM, Michael S. Tsirkin wrote: On Mon, Dec 11, 2017 at 02:38:45PM +0800, Wei Wang wrote: On 12/01/2017 11:49 PM, Michael S. Tsirkin wrote: On Wed, Nov 29, 2017 at 09:55:26PM +0800, Wei Wang wrote: The guest free pages should not be discarded by the live migration thread

[PATCH v19 7/7] virtio-balloon: don't report free pages when page poisoning is enabled

adds a config field, poison_val. Guest writes to the config field to tell the host about the poisoning value. The value will be 0 in the following cases: 1) PAGE_POISONING_NO_SANITY is enabled; 2) page poisoning is disabled; or 3) PAGE_POISONING_ZERO is enabled. Signed-off-by: Wei Wang <wei.

[PATCH v19 6/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

the reporting in advance by sending the stop cmd id to the guest via the configuration register. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@kernel.org

[PATCH v19 5/7] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v19 1/7] xbitmap: Introduce xbitmap

st_bit(). Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura

[PATCH v19 4/7] virtio-balloon: VIRTIO_BALLOON_F_SG

-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> --- drivers/virtio/virtio_balloon.c | 232

[PATCH v19 3/7] xbitmap: add more operations

to help searching. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker.

[PATCH v19 2/7] xbitmap: potential improvement

x_tree_preload() after kmalloc() and returns a boolean to indicate the success or failure. Also, add the __must_check annotation to xb_preload for prudence purpose. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@

[PATCH v19 0/7] Virtio-balloon Enhancement

CHUNKS and VIRTIO_BALLOON_F_MISC_VQ, which were mixed together in the previous implementation; 2) Simpler function to get the free page block. v7->v8: 1) Use only one chunk format, instead of two. 2) re-write the virtio-balloon implementation patch. 3) commit changes 4) patch re-org Matthew Wilcox (1):

Re: [PATCH v18 10/10] virtio-balloon: don't report free pages when page poisoning is enabled

On 12/01/2017 11:49 PM, Michael S. Tsirkin wrote: On Wed, Nov 29, 2017 at 09:55:26PM +0800, Wei Wang wrote: The guest free pages should not be discarded by the live migration thread when page poisoning is enabled with PAGE_POISONING_NO_SANITY=n, because skipping the transfer of such poisoned

Re: [PATCH v18 05/10] xbitmap: add more operations

On 12/03/2017 09:50 AM, Tetsuo Handa wrote: Matthew Wilcox wrote: On Fri, Dec 01, 2017 at 03:09:08PM +, Wang, Wei W wrote: On Friday, December 1, 2017 9:02 PM, Tetsuo Handa wrote: If start == end is legal, for (; start < end; start = (start | (IDA_BITMAP_BITS - 1)) + 1) { makes this

Re: [PATCH v18 10/10] virtio-balloon: don't report free pages when page poisoning is enabled

On 12/01/2017 11:49 PM, Michael S. Tsirkin wrote: On Wed, Nov 29, 2017 at 09:55:26PM +0800, Wei Wang wrote: The guest free pages should not be discarded by the live migration thread when page poisoning is enabled with PAGE_POISONING_NO_SANITY=n, because skipping the transfer of such poisoned

Re: [PATCH v18 07/10] virtio-balloon: VIRTIO_BALLOON_F_SG

On 12/01/2017 11:38 PM, Michael S. Tsirkin wrote: On Wed, Nov 29, 2017 at 09:55:23PM +0800, Wei Wang wrote: +static void send_one_desc(struct virtio_balloon *vb, + struct virtqueue *vq, + uint64_t addr, + uint32_t len

Re: [PATCH v18 06/10] virtio_ring: add a new API, virtqueue_add_one_desc

On 12/01/2017 03:38 AM, Michael S. Tsirkin wrote: On Wed, Nov 29, 2017 at 09:55:22PM +0800, Wei Wang wrote: Current virtqueue_add API implementation is based on the scatterlist struct, which uses kaddr. This is inadequate to all the use case of vring. For example: - Some usages don't use IOMMU

Re: [PATCH v18 05/10] xbitmap: add more operations

On 11/30/2017 06:34 PM, Tetsuo Handa wrote: Wei Wang wrote: + * @start: the start of the bit range, inclusive + * @end: the end of the bit range, inclusive + * + * This function is used to clear a bit in the xbitmap. If all the bits of the + * bitmap are 0, the bitmap will be freed. + */ +void

[PATCH v18 08/10] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v18 10/10] virtio-balloon: don't report free pages when page poisoning is enabled

skips the reporting of free pages in the above case. Reported-by: Michael S. Tsirkin <m...@redhat.com> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michal Hocko <mho...@suse.com> --- drivers/virtio/virtio_balloon.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) di

[PATCH v18 07/10] virtio-balloon: VIRTIO_BALLOON_F_SG

. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> --- drivers/virtio/virtio_balloon.c | 230 ++

[PATCH v18 09/10] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

the reporting in advance by sending the stop cmd id to the guest via the configuration register. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@kernel.org

[PATCH v18 06/10] virtio_ring: add a new API, virtqueue_add_one_desc

address and len. Also, factor out the common code with virtqueue_add in vring_set_avail. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> --- drivers/virtio/virtio_ring.c | 94 +++- include/linux/virtio.h

[PATCH v18 04/10] xbitmap: potential improvement

x_tree_preload() after kmalloc() and returns a boolean to indicate the success or failure. Also, add the __must_check annotation to xb_preload for prudence purpose. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@

[PATCH v18 05/10] xbitmap: add more operations

to help searching. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker.

[PATCH v18 02/10] radix tree test suite: remove ARRAY_SIZE to avoid redefinition

ARRAY_SIZE() has been defined in include/linux/kernel.h, and "make" complains a warning of redefinition of ARRAY_SIZE() in testing/radix/linux/kernel.h. So, remove ARRAY_SIZE() from there. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com

[PATCH v18 03/10] xbitmap: Introduce xbitmap

st_bit(). Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- include/linux/radix-tree.h | 2 + include/linux/xbitmap.h | 52 + lib/Makefile | 2 +- lib/radix-tree.c | 26 - l

[PATCH v18 01/10] idr: add #include

to solve the issue. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Jan Kara <j...@suse.cz> Cc: Eric Biggers <ebigg...@google.com> Cc: Tejun Heo <t...@kernel.org&g

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: You should Cc Nitesh who is working on a related feature. OK, I'll do. We have two more issues which haven't been discussed yet, please have a check below. On Mon, Nov 13, 2017 at 06:34:48PM +0800, Wei Wang wrote: Ping for comments

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/17/2017 07:35 PM, Wei Wang wrote: On 11/16/2017 09:27 PM, Wei Wang wrote: On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/16/2017 09:27 PM, Wei Wang wrote: On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio-balloon. The host requests the guest to report

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/15/2017 09:26 PM, Michael S. Tsirkin wrote: On Wed, Nov 15, 2017 at 11:47:58AM +0800, Wei Wang wrote: On 11/15/2017 05:21 AM, Michael S. Tsirkin wrote: On Tue, Nov 14, 2017 at 08:02:03PM +0800, Wei Wang wrote: On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/15/2017 05:21 AM, Michael S. Tsirkin wrote: On Tue, Nov 14, 2017 at 08:02:03PM +0800, Wei Wang wrote: On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written by the guest to ACK to the host about the commands that have been received. The host will clear

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written by the guest to ACK to the host about the commands that have been received. The host will clear the corresponding bits on the host2guest_cmd register. The guest also uses this register to send commands to the host (e.g.

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

Ping for comments, thanks. On 11/03/2017 04:13 PM, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio-balloon. The host requests the guest to report the free pages by sending commands via

Re: [PULL] virtio: last minute bugfix

On 11/08/2017 03:23 AM, Michael S. Tsirkin wrote: On Tue, Nov 07, 2017 at 08:13:10PM +0200, Michael S. Tsirkin wrote: On Tue, Nov 07, 2017 at 09:29:59AM -0800, Linus Torvalds wrote: On Tue, Nov 7, 2017 at 9:23 AM, Linus Torvalds wrote: I guess I'll take it, but

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

On 11/04/2017 07:28 PM, Tetsuo Handa wrote: Wei Wang wrote: On 11/03/2017 07:25 PM, Tetsuo Handa wrote: If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked on vb->balloon_lock mut

Re: [PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

On 11/03/2017 06:55 PM, Tetsuo Handa wrote: I'm commenting without understanding the logic. Wei Wang wrote: + +bool xb_preload(gfp_t gfp); + Want __must_check annotation, for __radix_tree_preload() is marked with __must_check annotation. By error failing to check result of xb_preload

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

On 11/03/2017 07:25 PM, Tetsuo Handa wrote: Wei Wang wrote: @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) break; } + if (use_sg && xb_set_page(vb, page, _min, _max) < 0) Isn't this leaking "

Re: [PATCH v1 0/3] Virtio-balloon Improvement

On 10/22/2017 11:19 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:23PM +0800, Wei Wang wrote: This patch series intends to summarize the recent contributions made by Michael S. Tsirkin, Tetsuo Handa, Michal Hocko etc. via reporting and discussing the related deadlock issues

[PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

finish free page reporting). - free_page_cmd_id: the sequence id of the free page report command given by the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@ker

[PATCH v17 5/6] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v17 2/6] radix tree test suite: add tests for xbitmap

From: Matthew Wilcox <mawil...@microsoft.com> Add the following tests for xbitmap: 1) single bit test: single bit set/clear/find; 2) bit range test: set/clear a range of bits and find a 0 or 1 bit in the range. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew W

[PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

() in chunks rather than page by page. With this new feature, the above ballooning process takes ~492ms resulting in an improvement of ~88%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Sign

[PATCH v17 3/6] mm/balloon_compaction.c: split balloon page allocation and enqueue

Michal Hocko <mho...@suse.com> Cc: Wei Wang <wei.w.w...@intel.com> Reviewed-by: Wei Wang <wei.w.w...@intel.com> --- drivers/virtio/virtio_balloon.c| 23 ++- include/linux/balloon_compaction.h | 34 +- mm/balloon_compacti

[PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

ed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> v

[PATCH v17 0/6] Virtio-balloon Enhancement

dix tree test suite: add tests for xbitmap Michael S. Tsirkin (1): mm/balloon_compaction.c: split balloon page allocation and enqueue Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ drivers/virtio/

Re: [PATCH] virtio_balloon: fix deadlock on OOM

Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> Cc: Michal Hocko <mho...@suse.com> Cc: Wei Wang <wei.w.w...@intel.com> --- The "virtio-balloon enhancement" series has a dependency on this patch. Could you send out a new version soon? Or I can include it

Re: [PATCH v1 3/3] virtio-balloon: stop inflating when OOM occurs

On 10/23/2017 01:13 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:26PM +0800, Wei Wang wrote: This patch forces the cease of the inflating work when OOM occurs. The fundamental idea of memory ballooning is to take out some guest pages when the guest has low memory utilization, so

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

On 10/22/2017 07:50 PM, Tetsuo Handa wrote: Wei Wang wrote: @@ -162,20 +160,20 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) msleep(200); break; } - set_page_pfns(vb, vb->pfns + vb->nu

Re: [PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

On 10/22/2017 12:11 PM, Tetsuo Handa wrote: Michael S. Tsirkin wrote: - num_freed_pages = leak_balloon(vb, oom_pages); + + /* Don't deflate more than the number of inflated pages */ + while (npages && atomic64_read(>num_pages)) + npages -= leak_balloon(vb,

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

On 10/22/2017 01:20 PM, Tetsuo Handa wrote: Wei Wang wrote: The balloon_lock was used to synchronize the access demand to elements of struct virtio_balloon and its queue operations (please see commit e22504296d). This prevents the concurrent run of the leak_balloon and fill_balloon functions

Re: [PATCH v1 0/3] Virtio-balloon Improvement

On 10/22/2017 11:19 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:23PM +0800, Wei Wang wrote: This patch series intends to summarize the recent contributions made by Michael S. Tsirkin, Tetsuo Handa, Michal Hocko etc. via reporting and discussing the related deadlock issues

[PATCH v1 3/3] virtio-balloon: stop inflating when OOM occurs

requests is sent to guest when the guest is still under memory pressure, still no pages will be inflated. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> Cc: Michal Hocko <mho...@ker

[PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

The current implementation only deflates 256 pages even when a user specifies more than that via the oom_pages module param. This patch enables the deflating of up to oom_pages pages if there are enough inflated pages. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsir

[PATCH v1 0/3] Virtio-balloon Improvement

ble processes and fall into kernel panic finally. Wei Wang (3): virtio-balloon: replace the coarse-grained balloon_lock virtio-balloon: deflate up to oom_pages on OOM virtio-balloon: stop inflating when OOM occurs drivers/virtio/virtio_balloon.c | 149

[PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

of the shared data (i.e. vb->pnfs, vb->num_pfns). This enables leak_balloon and fill_balloon to run concurrently and solves the deadlock issue. Reported-by: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsirkin &l

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/13/2017 09:38 PM, Michael S. Tsirkin wrote: On Thu, Oct 12, 2017 at 11:54:56AM +0800, Wei Wang wrote: But I think flushing is very fragile. You will easily run into races if one of the actors gets out of sync and keeps adding data. I think adding an ID in the free vq stream is a more

Re: [PATCH] virtio_balloon: fix deadlock on OOM

even in presence of lock contention. But Wei Wang is proposing VIRTIO_BALLOON_F_SG which will try to allocate memory, isn't he? Hopefully that can be fixed by allocating outside the lock. I think that would still have an issue even without the lock, because we can't do any memory allocation

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/11/2017 09:49 PM, Michael S. Tsirkin wrote: On Wed, Oct 11, 2017 at 02:03:20PM +0800, Wei Wang wrote: On 10/10/2017 11:15 PM, Michael S. Tsirkin wrote: On Mon, Oct 02, 2017 at 04:38:01PM +, Wang, Wei W wrote: On Sunday, October 1, 2017 11:19 AM, Michael S. Tsirkin wrote: On Sat

Re: [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify()

_BALLOON_F_SG) & wait; Reviewed-by: Wei Wang <wei.w.w...@intel.com> --- drivers/virtio/virtio_balloon.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c index f0b3a0b..03e6

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/10/2017 07:08 PM, Tetsuo Handa wrote: Wei Wang wrote: On 10/09/2017 11:20 PM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:52PM +0800, Wei Wang wrote: +static inline void xb_set_page(struct virtio_balloon *vb, + struct page *page

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/09/2017 11:20 PM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:52PM +0800, Wei Wang wrote: +static inline void xb_set_page(struct virtio_balloon *vb, + struct page *page, + unsigned long *pfn_min

Re: [PATCH v16 0/5] Virtio-balloon Enhancement

On 10/01/2017 09:25 PM, Damian Tometzki wrote: Hello, where i can found the patch in git.kernel.org ? We don't have patches there. If you want to try this feature, you can get the qemu side draft code here: https://github.com/wei-w-wang/qemu-lm Best, Wei

[PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@kernel.org> --- drivers/virtio/virtio_balloon.c | 249

[PATCH v16 2/5] radix tree test suite: add tests for xbitmap

From: Matthew Wilcox <mawil...@microsoft.com> Add the following tests for xbitmap: 1) single bit test: single bit set/clear/find; 2) bit range test: set/clear a range of bits and find a 0 or 1 bit in the range. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew W

[PATCH v16 0/5] Virtio-balloon Enhancement

on patch. 3) commit changes 4) patch re-org Matthew Wilcox (2): lib/xbitmap: Introduce xbitmap radix tree test suite: add tests for xbitmap Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ driv

Re: [virtio-dev] Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 09/29/2017 12:01 PM, Michael S. Tsirkin wrote: On Fri, Sep 08, 2017 at 07:09:24PM +0800, Wei Wang wrote: On 09/08/2017 11:36 AM, Michael S. Tsirkin wrote: On Tue, Aug 29, 2017 at 11:09:18AM +0800, Wei Wang wrote: On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

On 09/27/2017 12:41 AM, Andi Kleen wrote: 1) vCPU context switching and guest side task switching are not identical. That is, when the vCPU is scheduled out, the guest task on the vCPU may not guest task lifetime has nothing to do with this. It's completely independent of what you do here on

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

On 09/25/2017 10:57 PM, Andi Kleen wrote: +static void auto_switch_lbr_msrs(struct vcpu_vmx *vmx) +{ + int i; + struct perf_lbr_stack lbr_stack; + + perf_get_lbr_stack(_stack); + + add_atomic_switch_msr(vmx, MSR_LBR_SELECT, 0, 0); + add_atomic_switch_msr(vmx,

Re: [PATCH v1 0/4] Enable LBR for the guest

On 09/25/2017 10:59 PM, Andi Kleen wrote: On Mon, Sep 25, 2017 at 12:44:52PM +0800, Wei Wang wrote: This patch series enables the Last Branch Recording feature for the guest. Instead of trapping each LBR stack MSR access, the MSRs are passthroughed to the guest. Those MSRs are switched (i.e

Re: [PATCH v1 1/4] KVM/vmx: re-write the msr auto switch feature

On 09/25/2017 07:54 PM, Paolo Bonzini wrote: On 25/09/2017 06:44, Wei Wang wrote: +static void update_msr_autoload_count_max(void) +{ + u64 vmx_msr; + int n; + + /* +* According to the Intel SDM, if Bits 27:25 of MSR_IA32_VMX_MISC is +* n, then (n + 1) * 512

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

On 09/25/2017 05:16 PM, Paolo Bonzini wrote: On 25/09/2017 06:44, Wei Wang wrote: Passthrough the LBR stack to the guest, and auto switch the stack MSRs upon VMEntry and VMExit. Signed-off-by: Wei Wang <wei.w.w...@intel.com> This has to be enabled separately for each guest, because

[PATCH v1 3/4] perf/x86: add a function to get the lbr stack

for the guest. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/events/intel/lbr.c | 23 +++ arch/x86/include/asm/perf_event.h | 14 ++ 2 files changed, 37 insertions(+) diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c index 8

[PATCH v1 4/4] KVM/vmx: enable lbr for the guest

Passthrough the LBR stack to the guest, and auto switch the stack MSRs upon VMEntry and VMExit. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/kvm/vmx.c | 50 ++ 1 file changed, 50 insertions(+) diff --git a/arch/x86/kvm/vmx.c

[PATCH v1 0/4] Enable LBR for the guest

This patch series enables the Last Branch Recording feature for the guest. Instead of trapping each LBR stack MSR access, the MSRs are passthroughed to the guest. Those MSRs are switched (i.e. load and saved) on VMExit and VMEntry. Test: Try "perf record -b ./test_program" on guest. W

[PATCH v1 1/4] KVM/vmx: re-write the msr auto switch feature

. This can be calculated by (MSR_IA32_VMX_MISC[27:25] + 1) * 512. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/kvm/vmx.c | 72 +++--- 1 file changed, 63 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

On 09/12/2017 10:29 PM, Tomáš Golembiovský wrote: On Tue, 29 Aug 2017 20:01:53 +0800 Wei Wang <wei.w.w...@intel.com> wrote: On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote: On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote: Hi, I have CCed the relevant mailing

Re: [PATCH v15 1/5] lib/xbitmap: Introduce xbitmap

On 09/11/2017 08:54 PM, Matthew Wilcox wrote: On Mon, Aug 28, 2017 at 06:08:29PM +0800, Wei Wang wrote: From: Matthew Wilcox <mawil...@microsoft.com> The eXtensible Bitmap is a sparse bitmap representation which is efficient for set bits which tend to cluster. It supports up to 'unsigne

Re: [virtio-dev] Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 09/08/2017 11:36 AM, Michael S. Tsirkin wrote: On Tue, Aug 29, 2017 at 11:09:18AM +0800, Wei Wang wrote: On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06:08:31PM +0800, Wei Wang wrote: Add a new feature, VIRTIO_BALLOON_F_SG, which enables the transfer of balloon

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote: On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote: Hi, I have CCed the relevant mailing lists and people most recently involved in virtio-balloon discussions. Hopefully this will help get the right people to see your questions.

Re: [PATCH v15 4/5] mm: support reporting free page blocks

On 08/28/2017 09:33 PM, Michal Hocko wrote: On Mon 28-08-17 18:08:32, Wei Wang wrote: This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the calle

Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06:08:31PM +0800, Wei Wang wrote: Add a new feature, VIRTIO_BALLOON_F_SG, which enables the transfer of balloon (i.e. inflated/deflated) pages using scatter-gather lists to the host. The implementation of the previous

[PATCH v15 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> --- drivers/virtio/virtio_balloon.c | 247 +--- include/uapi/linux/virtio_balloon.h | 15 +++ 2 files changed, 242 insertions(+), 20 deleti

[PATCH v15 4/5] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

() in chunks rather than page by page. With this new feature, the above ballooning process takes ~597ms resulting in an improvement of ~86%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Sign

[PATCH v15 2/5] lib/xbitmap: add xb_find_next_bit() and xb_zero()

xb_find_next_bit() is used to find the next "1" or "0" bit in the given range. xb_zero() is used to zero the given range of bits. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Matthew Wilcox <mawil...@m

[PATCH v15 1/5] lib/xbitmap: Introduce xbitmap

st_bit(). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> --- include/linux/radix-tree.h | 3

[PATCH v15 0/5] Virtio-balloon Enhancement

tead of two. 2) re-write the virtio-balloon implementation patch. 3) commit changes 4) patch re-org Matthew Wilcox (1): lib/xbitmap: Introduce xbitmap Wei Wang (4): lib/xbitmap: add xb_find_next_bit() and xb_zero() virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page

Re: [PATCH v14 4/5] mm: support reporting free page blocks

On 08/18/2017 09:46 PM, Michal Hocko wrote: On Thu 17-08-17 11:26:55, Wei Wang wrote: This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the calle

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 08/19/2017 02:26 AM, Michael S. Tsirkin wrote: On Fri, Aug 18, 2017 at 04:41:41PM +0800, Wei Wang wrote: On 08/18/2017 10:13 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Please add

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 08/18/2017 10:13 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Please add some text here explaining the report_free_page_signal thing. I also really think we need some kind of ID

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 08/18/2017 10:28 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> --- dr

Re: [PATCH v14 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 08/18/2017 10:22 AM, Michael S. Tsirkin wrote: +static void send_balloon_page_sg(struct virtio_balloon *vb, +struct virtqueue *vq, +void *addr, +uint32_t size) +{ + unsigned int len; +

<    1   2   3   4   5   >