Re: About the performance of hyper-v

2021-06-01 Thread Liang Li
== > > Analyze events for all VMs, all VCPUs: > > VM-EXITSamples Samples% Time%Min TimeMax > > Time Avg time > >MSR_WRITE 92404589.96%81.10% 0.42us > > 68.42us 1.26us ( +- 0.07% ) > >

Re: About the performance of hyper-v

2021-05-23 Thread Liang Li
> >> > Analyze events for all VMs, all VCPUs: > >> > VM-EXITSamples Samples% Time%Min TimeMax > >> > Time Avg time > >> > EXTERNAL_INTERRUPT 47183159.89%68.58% 0.64us > >> > 65.42us 2.34us ( +- 0.11% ) > >> >MSR_WRITE

Re: About the performance of hyper-v

2021-05-21 Thread Liang Li
> > Hi Vitaly, > > > > I found a case that the virtualization overhead was almost doubled > > when turning on Hper-v related features compared to that without any > > no hyper-v feature. It happens when running a 3D game in windows > > guest in qemu kvm environment. > > > > By investigation, I

About the performance of hyper-v

2021-05-19 Thread Liang Li
[resend for missing cc] Hi Vitaly, I found a case that the virtualization overhead was almost doubled when turning on Hper-v related features compared to that without any no hyper-v feature. It happens when running a 3D game in windows guest in qemu kvm environment. By investigation, I found

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-23 Thread Liang Li
> >>> +static int > >>> +hugepage_reporting_cycle(struct page_reporting_dev_info *prdev, > >>> + struct hstate *h, unsigned int nid, > >>> + struct scatterlist *sgl, unsigned int *offset) > >>> +{ > >>> + struct list_head *list =

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-23 Thread Liang Li
> > > > + spin_lock_irq(_lock); > > > > + > > > > + if (huge_page_order(h) > MAX_ORDER) > > > > + budget = HUGEPAGE_REPORTING_CAPACITY; > > > > + else > > > > + budget = HUGEPAGE_REPORTING_CAPACITY * 32; > > > > > > Wouldn't huge_page_order always be

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> On 12/21/20 11:46 PM, Liang Li wrote: > > Free page reporting only supports buddy pages, it can't report the > > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs > > is a good choice for a system with a huge amount of RAM, because it > > ca

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> On 12/22/20 11:59 AM, Alexander Duyck wrote: > > On Mon, Dec 21, 2020 at 11:47 PM Liang Li > > wrote: > >> + > >> + if (huge_page_order(h) > MAX_ORDER) > >> + budget = HUGEPAGE_REPORTING_CAPACITY; > >> + else >

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> > +hugepage_reporting_cycle(struct page_reporting_dev_info *prdev, > > +struct hstate *h, unsigned int nid, > > +struct scatterlist *sgl, unsigned int *offset) > > +{ > > + struct list_head *list = >hugepage_freelists[nid]; > > +

Re: [RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-22 Thread Liang Li
> > Free page reporting in virtio-balloon doesn't give you any guarantees > > regarding zeroing of pages. Take a look at the QEMU implementation - > > e.g., with vfio all reports are simply ignored. > > > > Also, I am not sure if mangling such details ("zeroing of pages") into > > the page

Re: [RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-22 Thread Liang Li
On Tue, Dec 22, 2020 at 4:28 PM David Hildenbrand wrote: > > On 22.12.20 08:48, Liang Li wrote: > > Free page reporting only supports buddy pages, it can't report the > > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs > > The virtio-balloon free

[RFC PATCH 0/3 updated] add support for free hugepage reporting

2020-12-22 Thread Liang Li
for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. This RFC is baed on my previous series: '[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO' Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support

[RFC PATCH 0/3 updated] add support for free hugepage reporting

2020-12-22 Thread Liang Li
for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. This RFC is baed on my previous series: '[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO' Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support

[RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-21 Thread Liang Li
Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- mm/page_prezero.c | 17 + 1 file changed, 17 insertions(+) diff --git a/mm/page_prezero.c b/mm/page_prezero.c index c8ce720bfc54..dff4e0adf402 100644 --- a/mm

[RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-21 Thread Liang Li
: Alex Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 61 + include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 62 insertions(+) diff --git a/drivers/virtio

[RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-21 Thread Liang Li
Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- include/linux/hugetlb.h| 3 + include/linux/page_reporting.h | 5 + mm/hugetlb.c | 29 mm

[RFC PATCH 0/3] add support for free hugepage reporting

2020-12-21 Thread Liang Li
, changes for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support for providing free huge page reports to host mm: support free hugepage pre zero out

Re: [Qemu-devel] [PATCH 0/2] buffer and delay backup COW write operation

2019-05-05 Thread Liang Li
On Tue, Apr 30, 2019 at 10:35:32AM +, Vladimir Sementsov-Ogievskiy wrote: > 28.04.2019 13:01, Liang Li wrote: > > If the backup target is a slow device like ceph rbd, the backup > > process will affect guest BLK write IO performance seriously, > > it's cause by the draw

[Qemu-devel] [PATCH 1/2] backup: buffer COW request and delay the write operation

2019-04-28 Thread Liang Li
Cc: Fam Zheng Signed-off-by: Liang Li --- block/backup.c | 117 ++--- 1 file changed, 104 insertions(+), 13 deletions(-) diff --git a/block/backup.c b/block/backup.c index 9988753249..d436f9e4ee 100644 --- a/block/backup.c +++ b/block/backup.c

[Qemu-devel] [PATCH 2/2] qapi: add interface for setting backup cow buffer size

2019-04-28 Thread Liang Li
Cc: John Snow Cc: Kevin Wolf Cc: Max Reitz Cc: Wen Congyang Cc: Xie Changlong Cc: Markus Armbruster Cc: Eric Blake Cc: Fam Zheng Signed-off-by: Liang Li --- block/backup.c| 3 ++- block/replication.c | 2 +- blockdev.c| 5 + include/block/block_int.h

[Qemu-devel] [PATCH 0/2] buffer and delay backup COW write operation

2019-04-28 Thread Liang Li
Cc: Fam Zheng Liang Li (2): backup: buffer COW request and delay the write operation qapi: add interface for setting backup cow buffer size block/backup.c| 118 +- block/replication.c | 2 +- blockdev.c| 5

Re: [Qemu-devel] [PATCH] vhost-user: fix qemu crash caused by failed backend

2018-10-29 Thread Liang Li
On Tue, Oct 02, 2018 at 01:54:25PM +0400, Marc-André Lureau wrote: > Hi > > On Thu, Sep 27, 2018 at 7:37 PM Liang Li wrote: > > > > During live migration, when stopping vhost-user device, 'vhost_dev_stop' > > will be called, 'vhost_dev_stop' will call

[Qemu-devel] [PATCH] migration: fix concurrent call of multifd_save_cleanup

2018-10-29 Thread Liang Li
Concurrent call of multifd_save_cleanup() is unsafe, it will lead to null pointer dereference. 'multifd_save_cleanup()' should not be called in multifd_new_send_channel_async(), move it to ram_save_cleanup() like other features do. Signed-off-by: Liang Li --- migration/migration.c | 5

[Qemu-devel] [PATCH] vhost-user: fix qemu crash caused by failed backend

2018-09-27 Thread Liang Li
will be triggerd, followed by the call chain chr_closed_bh()->vhost_user_stop()-> vhost_net_cleanup()->vhost_dev_cleanup() vhost_dev_cleanup will clear vhost_dev struct, so the later 'vhost_user_read' or 'vhost_user_read' will reference null pointer and cause qemu crash Signed-off-by:

[Qemu-devel] [PATCH] vhost-user: fix qemu crash caused by failed backend

2018-09-27 Thread Liang Li
will be triggerd, followed by the call chain chr_closed_bh()->vhost_user_stop()-> vhost_net_cleanup()->vhost_dev_cleanup() vhost_dev_cleanup will clear vhost_dev struct, so the later 'vhost_user_read' or 'vhost_user_read' will reference null pointer and cause qemu crash Signed-off-by:

Re: [Qemu-devel] [PATCH V5] migration: add capability to bypass the shared memory

2018-06-27 Thread Liang Li
On Mon, Apr 16, 2018 at 11:00:11PM +0800, Lai Jiangshan wrote: > > migration/migration.c | 22 ++ > migration/migration.h | 1 + > migration/ram.c | 27 ++- > qapi/migration.json | 6 +- > 4 files changed, 46 insertions(+), 10

Re: [Qemu-devel] [PATCH v2 3/3] virtio-balloon: add a timer to limit the free page report waiting time

2018-02-27 Thread Liang Li
On Tue, Feb 27, 2018 at 06:10:47PM +0800, Wei Wang wrote: > On 02/27/2018 08:50 AM, Michael S. Tsirkin wrote: > > On Mon, Feb 26, 2018 at 12:35:31PM +0800, Wei Wang wrote: > > > On 02/09/2018 08:15 PM, Dr. David Alan Gilbert wrote: > > > > * Wei Wang (wei.w.w...@intel.com) wrote: > > > > I think

[Qemu-devel] [PATCH v2 resend] block/mirror: change the semantic of 'force' of block-job-cancel

2018-02-26 Thread Liang Li
c: John Snow <js...@redhat.com> Reported-by: Huaitong Han <huanhuait...@didichuxing.com> Signed-off-by: Huaitong Han <huanhuait...@didichuxing.com> Signed-off-by: Liang Li <liliang...@didichuxing.com> --- block/mirror.c| 10 -- blockdev.c| 4

[Qemu-devel] [PATCH v2] block/mirror: change the semantic of 'force' of block-job-cancel

2018-02-05 Thread Liang Li
c: John Snow <js...@redhat.com> Reported-by: Huaitong Han <huanhuait...@didichuxing.com> Signed-off-by: Huaitong Han <huanhuait...@didichuxing.com> Signed-off-by: Liang Li <liliang...@didichuxing.com> --- block/mirror.c| 10 -- blockdev.c| 4

Re: [Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-02-05 Thread Liang Li
On Mon, Feb 05, 2018 at 02:28:55PM -0500, John Snow wrote: > > > On 01/31/2018 09:19 PM, Liang Li wrote: > > On Tue, Jan 30, 2018 at 03:18:31PM -0500, John Snow wrote: > >> > >> > >> On 01/30/2018 03:38 AM, Liang Li wrote: > >>>

Re: [Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-01-31 Thread Liang Li
On Tue, Jan 30, 2018 at 03:18:31PM -0500, John Snow wrote: > > > On 01/30/2018 03:38 AM, Liang Li wrote: >> When doing drive mirror to a low speed shared storage, if there was heavy >> BLK IO write workload in VM after the 'ready' event, drive mirror block job >> ca

Re: [Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-01-31 Thread Liang Li
On Tue, Jan 30, 2018 at 08:20:03AM -0600, Eric Blake wrote: > On 01/30/2018 02:38 AM, Liang Li wrote: >> When doing drive mirror to a low speed shared storage, if there was heavy >> BLK IO write workload in VM after the 'ready' event, drive mirror block job >> can't be

[Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-01-30 Thread Liang Li
dy <jc...@redhat.com> Cc: Kevin Wolf <kw...@redhat.com> Cc: Max Reitz <mre...@redhat.com> Cc: Eric Blake <ebl...@redhat.com> Cc: John Snow <js...@redhat.com> Reported-by: Huaitong Han <huanhuait...@didichuxing.com> Signed-off-by: Huaitong Han <huanhuait...@didi

Re: [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-28 Thread Liang Li
On Fri, Jan 26, 2018 at 08:04:08AM -0600, Eric Blake wrote: > On 01/26/2018 12:46 AM, Liang Li wrote: > > The current QMP command is: > > > > { 'command': 'block-job-cancel', 'data': { 'device': 'str', '*force': > > 'bool' } } > > > > 'force' has oth

Re: [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-25 Thread Liang Li
On Thu, Jan 25, 2018 at 08:48:22AM -0600, Eric Blake wrote: > On 01/24/2018 10:59 PM, Liang Li wrote: > >> > >> There's ongoing work on adding async mirroring; this may be a better > >> solution to the issue you are seeing. > >> > >> https://lists.

Re: [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-24 Thread Liang Li
On Wed, Jan 24, 2018 at 01:16:39PM -0600, Eric Blake wrote: > On 01/24/2018 12:17 AM, Liang Li wrote: > > We found that when doing drive mirror to a low speed shared storage, > > if there was heavy BLK IO write workload in VM after the 'ready' event, > > drive mirror block

[Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-23 Thread Liang Li
. Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Jeff Cody <jc...@redhat.com> Cc: Kevin Wolf <kw...@redhat.com> Cc: Max Reitz <mre...@redhat.com> Signed-off-by: Huaitong Han <hanhuait...@didichuxing.com> Signed-off-by: Liang Li <liliang...@didichuxing.com> --- bl

[Qemu-devel] [PATCH] hbitmap: fix missing restore count when finish deserialization

2018-01-18 Thread Liang Li
; Signed-off-by: weiping zhang <zhangweip...@didichuxing.com> Signed-off-by: Liang Li <liliang...@didichuxing.com> --- util/hbitmap.c | 1 + 1 file changed, 1 insertion(+) diff --git a/util/hbitmap.c b/util/hbitmap.c index 289778a..58a2c93 100644 --- a/util/hbitmap.c +++ b/uti

[Qemu-devel] [PATCH resend] hbitmap: fix missing restore count when finish deserialization

2018-01-18 Thread Liang Li
t; Reviewed-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Weiping Zhang <zhangweip...@didichuxing.com> Signed-off-by: Liang Li <liliang...@didichuxing.com> --- util/hbitmap.c | 1 + 1 file changed, 1 insertion(+) diff --git a/util/hbitmap.c b/ut

[Qemu-devel] [PATCH v4 qemu 6/6] migration: skip unused pages during live migration

2017-01-11 Thread Liang Li
the unused pages will not be filtered out in this case. The current implementation can not work with post copy, if post copy is enabled, we simply ignore the unused pages. Will make it work later. Signed-off-by: Liang Li <liang.z...@intel.com> --- migration/ram.

[Qemu-devel] [PATCH v4 qemu 5/6] kvm.c: Add two new arch specific functions

2017-01-11 Thread Liang Li
Add a new function to get the vm's max pfn and a new function to filter out the holes in the undressed free page bitmap to get a tight free page bitmap. They are implemented on X86 and should be implemented on other arches for live migration optimization. Signed-off-by: Liang Li <lian

[Qemu-devel] [PATCH v4 qemu 4/6] bitmap: Add a new bitmap_move function

2017-01-11 Thread Liang Li
Sometimes, it is need to move a portion of bitmap to another place in a large bitmap, if overlap happens, the bitmap_copy can't not work correctly, we need a new function to do this work. Signed-off-by: Liang Li <liang.z...@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilb...@r

[Qemu-devel] [PATCH v4 qemu 3/6] balloon: get unused page info from guest

2017-01-11 Thread Liang Li
-by: Liang Li <liang.z...@intel.com> --- balloon.c | 47 +++- hw/virtio/virtio-balloon.c | 149 - include/hw/virtio/virtio-balloon.h | 18 - include/sysemu/balloon.h | 18 - 4 files changed, 227 inse

[Qemu-devel] [PATCH v4 qemu 0/6] Fast (de)inflating & fast live migration

2017-01-11 Thread Liang Li
instead of separate variables. * Use two separate APIs to request free pages and query the status. * Changed the virtio balloon interface. * Addressed some of the comments of v1. Liang Li (6): virtio-balloon: update linux head file virtio-balloon: speed up inflating & deflating pro

[Qemu-devel] [PATCH v4 qemu 2/6] virtio-balloon: speed up inflating & deflating process

2017-01-11 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S.

[Qemu-devel] [PATCH v4 qemu 1/6] virtio-balloon: update linux head file

2017-01-11 Thread Liang Li
Update the linux head file to keep consistent with kernel side. The new definition will be used in the following patches. Signed-off-by: Liang Li <liang.z...@intel.com> --- include/standard-headers/linux/virtio_balloon.h | 34 + 1 file changed, 34 insertions(+)

[Qemu-devel] [PATCH v6 kernel 5/5] virtio-balloon: tell host vm's unused page info

2016-12-20 Thread Liang Li
pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.c

[Qemu-devel] [PATCH v6 kernel 3/5] virtio-balloon: speed up inflate/deflate process

2016-12-20 Thread Liang Li
new feature, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S.

[Qemu-devel] [PATCH v6 kernel 4/5] virtio-balloon: define flags and head for host request vq

2016-12-20 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel

[Qemu-devel] [PATCH v6 kernel 2/5] virtio-balloon: define new feature bit and head struct

2016-12-20 Thread Liang Li
Add a new feature which supports sending the page information with range array. The current implementation uses PFNs array, which is not very efficient. Using ranges can improve the performance of inflating/deflating significantly. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Mic

[Qemu-devel] [PATCH v6 kernel 1/5] virtio-balloon: rework deflate to add page to a list

2016-12-20 Thread Liang Li
pfn|length} down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit

[Qemu-devel] [PATCH v6 kernel 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-20 Thread Liang Li
* Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process

[Qemu-devel] [PATCH kernel v5 4/5] virtio-balloon: define flags and head for host request vq

2016-11-30 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel

[Qemu-devel] [PATCH kernel v5 3/5] virtio-balloon: speed up inflate/deflate process

2016-11-30 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[Qemu-devel] [PATCH kernel v5 2/5] virtio-balloon: define new feature bit and head struct

2016-11-30 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <ami

[Qemu-devel] [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info

2016-11-30 Thread Liang Li
pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.c

[Qemu-devel] [PATCH kernel v5 1/5] virtio-balloon: rework deflate to add page to a list

2016-11-30 Thread Liang Li
ead of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com&

[Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-30 Thread Liang Li
e bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process virtio-balloon: define flags and head for host request vq virtio

[Qemu-devel] [PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-02 Thread Liang Li
process. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <amit.s...@redhat.com> Cc: Dave Hansen <dave.han...@intel.com> --- d

[Qemu-devel] [PATCH kernel v4 6/7] virtio-balloon: define flags and head for host request vq

2016-11-02 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel

[Qemu-devel] [PATCH kernel v4 4/7] virtio-balloon: speed up inflate/deflate process

2016-11-02 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[Qemu-devel] [PATCH kernel v4 5/7] mm: add the related functions to get unused page

2016-11-02 Thread Liang Li
can be corrected by the dirty page logging mechanism. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@re

[Qemu-devel] [PATCH kernel v4 3/7] mm: add a function to get the max pfn

2016-11-02 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li <liang.z...@intel.com> Cc:

[Qemu-devel] [PATCH kernel v4 2/7] virtio-balloon: define new feature bit and head struct

2016-11-02 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <ami

[Qemu-devel] [PATCH kernel v4 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-02 Thread Liang Li
api head file. * Use a new way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new fea

[Qemu-devel] [PATCH kernel v4 1/7] virtio-balloon: rework deflate to add page to a list

2016-11-02 Thread Liang Li
ead of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com&

[Qemu-devel] [PATCH qemu v3 6/6] migration: skip free pages during live migration

2016-10-21 Thread Liang Li
will not be filtered out in this case. The current implementation can not work with post copy, if post copy is enabled, we simply ignore the free pages. Will make it work later. Signed-off-by: Liang Li <liang.z...@intel.com> --- migration/ram.

[Qemu-devel] [PATCH qemu v3 1/6] virtio-balloon: update linux head file

2016-10-21 Thread Liang Li
Update the new feature bit definition for virtio balloon and the page bitmap header, request header struct to keep consistent with kernel side. Signed-off-by: Liang Li <liang.z...@intel.com> --- include/standard-headers/linux/virtio_balloon.h | 41 + 1 file chang

[Qemu-devel] [PATCH qemu v3 5/6] kvm: Add two new arch specific functions

2016-10-21 Thread Liang Li
Add a new function to get the vm's max pfn and a new function to filter out the holes in the undressed free page bitmap to get a tight free page bitmap. They are implemented on X86 and should be implemented on other arches for live migration optimization. Signed-off-by: Liang Li <lian

[Qemu-devel] [PATCH qemu v3 4/6] bitmap: Add a new bitmap_move function

2016-10-21 Thread Liang Li
Sometimes, it is need to move a portion of bitmap to another place in a large bitmap, if overlap happens, the bitmap_copy can't not work correctly, we need a new function to do this work. Signed-off-by: Liang Li <liang.z...@intel.com> --- include/qemu/bitmap.h | 13 + 1 file c

[Qemu-devel] [PATCH qemu v3 0/6] Fast (de)inflating & fast live migration

2016-10-21 Thread Liang Li
interface. * Addressed some of the comments of v1. Liang Li (6): virtio-balloon: update linux head file virtio-balloon: speed up inflating & deflating process balloon: get free page info from guest bitmap: Add a new bitmap_move function kvm: Add two new arch specific functions migra

[Qemu-devel] [PATCH qemu v3 3/6] balloon: get free page info from guest

2016-10-21 Thread Liang Li
Add a new feature to get the free page information from guest, the free page information is saved in a bitmap. Please note that 'free page' means page is free sometime after host set the value of request ID and before it receive response with the same ID. Signed-off-by: Liang Li <lian

[Qemu-devel] [PATCH qemu v3 2/6] virtio-balloon: speed up inflating & deflating process

2016-10-21 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[Qemu-devel] [RESEND PATCH v3 kernel 5/7] mm: add the related functions to get unused page

2016-10-21 Thread Liang Li
be corrected by the dirty page logging mechanism. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com>

[Qemu-devel] [RESEND PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue

2016-10-21 Thread Liang Li
. And the VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bo

[Qemu-devel] [RESEND PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process

2016-10-21 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[Qemu-devel] [RESEND PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info

2016-10-21 Thread Liang Li
Support the request for vm's unused page information, response with a page bitmap. QEMU can make use of this bitmap and the dirty page logging mechanism to skip the transportation of these unused pages, this is very helpful to speed up the live migration process. Signed-off-by: Liang Li <lian

[Qemu-devel] [RESEND PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-10-21 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li <liang.z...@intel.com> Cc:

[Qemu-devel] [RESEND PATCH v3 kernel 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-10-21 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <amit.s.

[Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-10-21 Thread Liang Li
Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and page bitmap head mm: add a function to get the max pfn virtio-balloon: speed up inflate/deflate process mm: add the related function

[Qemu-devel] [RESEND PATCH v3 kernel 1/7] virtio-balloon: rework deflate to add page to a list

2016-10-21 Thread Liang Li
Will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Corneli

[Qemu-devel] [PATCH repost] virtio-balloon: Remove needless precompiled directive

2016-08-08 Thread Liang Li
Since there in wrapper around madvise(), the virtio-balloon code is able to work without the precompiled directive, the directive can be removed. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Thomas Huth <th...@redhat.com> Reviewd-by: Dr. David Alan Gilbert <dgil

[Qemu-devel] [PATCH] migration: fix live migration failure with compression

2016-08-08 Thread Liang Li
Because of commit 11808bb0c422, which remove some condition checks of 'f->ops->writev_buffer', 'qemu_put_qemu_file' should be enhanced to clear the 'f_src->iovcnt', or 'f_src->iovcnt' may exceed the MAX_IOV_SIZE which will break live migration. This should be fixed. Signed-off-

[Qemu-devel] [PATCH v3 kernel 5/7] mm: add the related functions to get unused page

2016-08-08 Thread Liang Li
be corrected by the dirty page logging mechanism. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com>

[Qemu-devel] [PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-08-08 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li <liang.z...@intel.com> Cc:

[Qemu-devel] [PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info

2016-08-08 Thread Liang Li
Support the request for vm's unused page information, response with a page bitmap. QEMU can make use of this bitmap and the dirty page logging mechanism to skip the transportation of these unused pages, this is very helpful to speed up the live migration process. Signed-off-by: Liang Li <lian

[Qemu-devel] [PATCH v3 kernel 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-08-08 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <ami

[Qemu-devel] [PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue

2016-08-08 Thread Liang Li
. And the VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bo

[Qemu-devel] [PATCH v3 kernel 1/7] virtio-balloon: rework deflate to add page to a list

2016-08-08 Thread Liang Li
Will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Corneli

[Qemu-devel] [PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process

2016-08-08 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[Qemu-devel] [PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-08-08 Thread Liang Li
Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and page bitmap head mm: add a function to get the max pfn virtio-balloon: speed up inflate/deflate process mm: add the related function

[Qemu-devel] [PATCH v2 repost 6/7] mm: add the related functions to get free page info

2016-07-26 Thread Liang Li
Save the free page info into a page bitmap, will be used in virtio balloon device driver. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Vlastimil Babka <vba...@suse.cz> Cc: Mel Gorman <mgor...@techsingularity.net> C

[Qemu-devel] [PATCH v2 repost 5/7] virtio-balloon: define feature bit and head for misc virt queue

2016-07-26 Thread Liang Li
. And the VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's free page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bo

[Qemu-devel] [PATCH v2 repost 1/7] virtio-balloon: rework deflate to add page to a list

2016-07-26 Thread Liang Li
will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Corneli

[Qemu-devel] [PATCH v2 repost 7/7] virtio-balloon: tell host vm's free page info

2016-07-26 Thread Liang Li
Support the request for vm's free page information, response with a page bitmap. QEMU can make use of this free page bitmap to speed up live migration process by skipping process the free pages. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com>

[Qemu-devel] [PATCH v2 repost 3/7] mm: add a function to get the max pfn

2016-07-26 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Vlastimil Babka <vba...@suse.cz> Cc: Mel Gorman <mgor...@techsingularity.net> C

[Qemu-devel] [PATCH v2 repost 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-07-26 Thread Liang Li
from v1 to v2: * Abandon the patch for dropping page cache. * Put some structures to uapi head file. * Use a new way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang L

[Qemu-devel] [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-26 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[Qemu-devel] [PATCH v2 repost 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-07-26 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <amit.s.

[Qemu-devel] [QEMU v2 5/9] balloon: get free page info from guest

2016-07-14 Thread Liang Li
Add a new feature to get the free page information from guest, the free page information is saved in a bitmap. Please note that 'free page' means page is free sometime after host set the value of request ID and before it receive response with the same ID. Signed-off-by: Liang Li <lian

  1   2   3   4   >