[PATCH v2] migration/postcopy: ensure preempt channel is ready before loading states

2024-04-04 Thread Wei Wang
dispatched. Originally-by: Lei Wang Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced...@intel.com/T/ Suggested-by: Peter Xu Signed-off-by: Lei Wang Signed-off-by: Wei Wang --- migration/savevm.c | 17 + 1 file changed, 17 insertions(+) diff --git a/migration

[PATCH v1] migration/postcopy: ensure preempt channel is ready before loading states

2024-04-04 Thread Wei Wang
. Originally-by: Lei Wang Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced...@intel.com/T/ Suggested-by: Peter Xu Signed-off-by: Lei Wang Signed-off-by: Wei Wang --- migration/savevm.c | 17 + 1 file changed, 17 insertions(+) diff --git a/migration/savevm.c

[PATCH v2] migration: refactor migration_completion

2023-08-04 Thread Wei Wang
related cleanup on migration completion. It is named to match with open_return_path_on_source. This improves readability and is easier for future updates (e.g. add new subfunctions when completion code related to new features are needed). No functional changes intended. Signed-off-by: Wei Wang

[PATCH v1] migration: refactor migration_completion

2023-07-14 Thread Wei Wang
related cleanup on migration completion. It is named to match with open_return_path_on_source. This improves readability and is easier for future updates (e.g. add new subfunctions when completion code related to new features are needed). No functional changes intended. Signed-off-by: Wei Wang

[PATCH RESEND v2 0/2] Enfore multifd and postcopy preempt setting

2023-06-06 Thread Wei Wang
and postcopy preempt to be set before incoming. If not, disable the feature and return with error messages to remind users to adjust the commands. Fix the qtest/migration-test to do so. v2 RESEND change: - Only patch commit change with more explanations Wei Wang (2): migration: enfocre multifd and postcopy

[PATCH v2 1/2] migration: enfocre multifd and postcopy preempt to be set before incoming

2023-06-06 Thread Wei Wang
: - need to use "-incoming defer" on the destination; and - set_capability and set_parameter need to be done before migrate_incoming Otherwise, disable the use of the features and report error messages to remind users to adjust the commands. Signed-off-by: Wei Wang Reviewed-by

[PATCH v2 2/2] qtest/migration-tests.c: use "-incoming defer" for postcopy tests

2023-06-06 Thread Wei Wang
the number of pending connections allowed is actually "backlog + 1", which is 2 in this case. That said, the implementation of socket_start_incoming_migration_internal expects "migrate defer" to be used, and for safety, change the test to work with the expected usage. Sig

[PATCH v2 1/2] migration: enfocre multifd and postcopy preempt to be set before incoming

2023-05-30 Thread Wei Wang
: - need to use "-incoming defer" on the destination; and - set_capability and set_parameter need to be done before migrate_incoming Otherwise, disable the use of the features and report error messages to remind users to adjust the commands. Signed-off-by: Wei Wang --- migration/opti

[PATCH v2 2/2] qtest/migration-tests.c: use "-incoming defer" for postcopy tests

2023-05-30 Thread Wei Wang
The Postcopy preempt capability requires to be set before incoming starts, so change the postcopy tests to start with deferred incoming and call migrate-incoming after the cap has been set. Signed-off-by: Wei Wang --- tests/qtest/migration-test.c | 10 -- 1 file changed, 8 insertions

[PATCH v2 0/2] Enfore multifd and postcopy preempt setting

2023-05-30 Thread Wei Wang
and postcopy preempt to be set before incoming. If not, disable the feature and return with error messages to remind users to adjust the commands. Fix the qtest/migration-test to do so. Wei Wang (2): migration: enfocre multifd and postcopy preempt to be set before incoming qtest/migration-tests.c: use

[PATCH v1] migration/multifd: always post the sync semaphores when sendesr exit

2023-05-24 Thread Wei Wang
s when the sender threads exits, posting on the semaphores will just be like a noop. Signed-off-by: Wei Wang --- migration/multifd.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 0bf5958a9c..1ff461675e 100644 --- a/migration/mul

[PATCH v1] migration: fix migrate_params_test_apply to set the dest param correctly

2023-05-24 Thread Wei Wang
ame) related fields need to be set, as they will be checked by migrate_params_check. Signed-off-by: Wei Wang --- migration/options.c | 24 ++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/migration/options.c b/migration/options.c index b62ab30cd5..a560483

[PATCH v1] migration: fail the cap check if it requires the use of deferred incoming

2023-05-18 Thread Wei Wang
-off-by: Wei Wang --- migration/options.c | 9 + 1 file changed, 9 insertions(+) diff --git a/migration/options.c b/migration/options.c index c2a278ee2d..25b333b3f4 100644 --- a/migration/options.c +++ b/migration/options.c @@ -537,6 +537,11 @@ bool migrate_caps_check(bool *old_caps, bool

[PATCH v1] migration/multifd: correct multifd_send_thread to trace the flags

2023-03-09 Thread Wei Wang
the local flags after the callback, so that the correct flags can be traced. Fixes: ab7cbb0b9a3b ("multifd: Make no compression operations into its own structure") Signed-off-by: Wei Wang --- migration/multifd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/migration

Re: KVM developers conference call agenda

2023-03-06 Thread Wei Wang
On 2/21/23 21:46, juan.quint...@gmail.com wrote: KVM developers conference call Hi for today call: Trivial stuff (less that 5mins each) - should we record the sessions - should we have the call every week. We have on the backburner: * TDX migration Hi Juan, Can I get 30 min (or more if no

Re: [PATCH v7 11/14] KVM: Register/unregister the guest private memory regions

2022-07-21 Thread Wei Wang
On 7/21/22 00:21, Sean Christopherson wrote: On Wed, Jul 20, 2022, Gupta, Pankaj wrote: +bool __weak kvm_arch_private_mem_supported(struct kvm *kvm) Use kvm_arch_has_private_mem(), both because "has" makes it obvious this is checking a flag of sorts, and to align with other helpers of this

[PATCH v3] migration: clear the memory region dirty bitmap when skipping free pages

2021-07-22 Thread Wei Wang
Cc: Michael S. Tsirkin Reported-by: David Hildenbrand Signed-off-by: Wei Wang --- migration/ram.c | 74 + 1 file changed, 56 insertions(+), 18 deletions(-) v2->v3 changelog: - change migration_clear_memory_region_dirty_bitmap_range to

[PATCH v2] migration: clear the memory region dirty bitmap when skipping free pages

2021-07-15 Thread Wei Wang
Cc: Michael S. Tsirkin Reported-by: David Hildenbrand Signed-off-by: Wei Wang --- migration/ram.c | 72 - 1 file changed, 54 insertions(+), 18 deletions(-) v1->v2 changelog: - move migration_clear_memory_region_dirty_bitmap under bitmap_mu

[PATCH v1] migration: clear the memory region dirty bitmap when skipping free pages

2021-07-14 Thread Wei Wang
Hildenbrand Signed-off-by: Wei Wang --- capstone| 2 +- migration/ram.c | 73 + slirp | 2 +- ui/keycodemapdb | 2 +- 4 files changed, 58 insertions(+), 21 deletions(-) diff --git a/capstone b/capstone index f8b1b83301..22ead3e0bf

Re: [raw] Guest stuck during live live-migration

2020-12-14 Thread Wei Wang
On 11/23/2020 05:36 PM, Quentin Grolleau wrote: Hello, In our company, we are hosting a large number of Vm, hosted behind Openstack (so libvirt/qemu). A large majority of our Vms are runnign with local data only, stored on NVME, and most of them are RAW disks. With Qemu 4.0(can be even with

[PATCH] migration: fix xbzrle encoding rate calculation

2020-06-07 Thread Wei Wang
It's reported an error of implicit conversion from "unsigned long" to "double" when compiling with Clang 10. Simply make the encoding rate 0 when the encoded_size is 0. Fixes: e460a4b1a4 Reported-by: Richard Henderson Signed-off-by: Wei Wang --- migration/ram.c | 4 +--

Re: [PATCH v3] migration/xbzrle: add encoding rate

2020-06-04 Thread Wei Wang
On 06/05/2020 12:57 AM, Richard Henderson wrote: On 6/4/20 3:27 AM, Wei Wang wrote: On 06/04/2020 05:38 PM, Dr. David Alan Gilbert wrote: Hmm OK; I'll admit to not liking NaN/Inf in output. Dave OK. To deal with the reported issue, how about using FLT_MAX (as opposed to UINT64_MAX or inf

Re: [PATCH v3] migration/xbzrle: add encoding rate

2020-06-04 Thread Wei Wang
On 06/04/2020 05:38 PM, Dr. David Alan Gilbert wrote: * Richard Henderson (richard.hender...@linaro.org) wrote: On 6/3/20 7:58 PM, Wei Wang wrote: It is possible that encoded_size==0, but unencoded_size !=0. For example, a page is written with the same data that it already has. That really

Re: [PATCH v3] migration/xbzrle: add encoding rate

2020-06-04 Thread Wei Wang
On 06/04/2020 11:22 AM, Richard Henderson wrote: On 6/3/20 7:58 PM, Wei Wang wrote: It is possible that encoded_size==0, but unencoded_size !=0. For example, a page is written with the same data that it already has. That really contains 0 bytes? Not even the ones that say "same data&qu

Re: [PATCH v3] migration/xbzrle: add encoding rate

2020-06-03 Thread Wei Wang
On 06/04/2020 03:28 AM, Richard Henderson wrote: On Wed, 29 Apr 2020 at 18:54, Wei Wang wrote: +if (xbzrle_counters.pages == rs->xbzrle_pages_prev) { +xbzrle_counters.encoding_rate = 0; +} else if (!encoded_size) { +xbzrle_counters.encoding_r

[PATCH v3] migration/xbzrle: add encoding rate

2020-04-29 Thread Wei Wang
Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun Signed-off-by: Wei Wang --- migration/migration.c | 1 + migration/ram.c | 39

Re: [PATCH v2] migration/xbzrle: add encoding rate

2020-04-28 Thread Wei Wang
On 04/28/2020 10:51 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun

[PATCH v2] migration/xbzrle: add encoding rate

2020-04-27 Thread Wei Wang
Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun Signed-off-by: Wei Wang --- migration/migration.c | 1 + migration/ram.c | 38

Re: [PATCH v1 1/2] migration/xbzrle: replace transferred xbzrle bytes with encoded bytes

2020-04-27 Thread Wei Wang
On 04/24/2020 06:47 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: On 04/22/2020 03:21 AM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: Like compressed_size which indicates how many bytes are compressed, we need encoded_size to understand

Re: [PATCH v1 1/2] migration/xbzrle: replace transferred xbzrle bytes with encoded bytes

2020-04-21 Thread Wei Wang
On 04/22/2020 03:21 AM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: Like compressed_size which indicates how many bytes are compressed, we need encoded_size to understand how many bytes are encoded with xbzrle during migration. Replace the old xbzrle_counter.bytes

Re: [PATCH v1 2/2] migration/xbzrle: add encoding rate

2020-04-20 Thread Wei Wang
On 04/20/2020 10:53 PM, Eric Blake wrote: On 4/19/20 10:06 PM, Wei Wang wrote: Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun Signed-off-by: Wei

Re: [PATCH v1 1/2] migration/xbzrle: replace transferred xbzrle bytes with encoded bytes

2020-04-20 Thread Wei Wang
On 04/20/2020 05:29 PM, Daniel P. Berrangé wrote: On Mon, Apr 20, 2020 at 11:06:42AM +0800, Wei Wang wrote: Like compressed_size which indicates how many bytes are compressed, we need encoded_size to understand how many bytes are encoded with xbzrle during migration. Replace the old

[PATCH v1 1/2] migration/xbzrle: replace transferred xbzrle bytes with encoded bytes

2020-04-19 Thread Wei Wang
, which includes 3 more bytes of the migration transfer protocol header (in addition to the encoding header). The encoded_size will further be used to calculate the encoding rate. Signed-off-by: Yi Sun Signed-off-by: Wei Wang --- migration/migration.c | 2 +- migration/ram.c | 18

[PATCH v1 2/2] migration/xbzrle: add encoding rate

2020-04-19 Thread Wei Wang
Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun Signed-off-by: Wei Wang --- migration/migration.c | 1 + migration/ram.c | 31

[PATCH v1 0/2] Migration xbzrle changes

2020-04-19 Thread Wei Wang
This patches change/add some xbzrle statistics to give users some useful feedbacks about the delta operations. Wei Wang (2): migration/xbzrle: replace transferred xbzrle bytes with encoded bytes migration/xbzrle: add encoding rate migration/migration.c | 3 ++- migration/ram.c | 49

Re: [Qemu-devel] [PATCH] migration/ram.c: Fix codes conflict about bitmap_mutex

2019-04-01 Thread Wei Wang
On 03/30/2019 06:29 AM, Zhang Chen wrote: From: Zhang Chen I found upstream codes conflict with COLO and lead to crash, and I located to this patch: commit 386a907b37a9321bc5d699bc37104d6ffba1b34d Author: Wei Wang Date: Tue Dec 11 16:24:49 2018 +0800 migration: use bitmap_mutex

[Qemu-devel] [PATCH] virtio-balloon: fix a use-after-free case

2019-03-12 Thread Wei Wang
The elem could theorically contain both outbuf and inbufs. We move the free operation to the end of this function to avoid using elem->in_sg while elem has been freed. Fixes: c13c4153f7 ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT") Reported-by: Peter Maydell Signed-off-by:

Re: [Qemu-devel] [PULL 19/22] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-03-09 Thread Wei Wang
On 03/09/2019 02:30 AM, Dr. David Alan Gilbert wrote: * Peter Maydell (peter.mayd...@linaro.org) wrote: On Wed, 6 Mar 2019 at 11:55, Dr. David Alan Gilbert (git) wrote: From: Wei Wang Wei: Can you look at this Sure, I'll send a followup patch to fix this. The new feature enables

Re: [Qemu-devel] [PATCH v11 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-03-05 Thread Wei Wang
On 03/05/2019 10:50 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: The new feature enables the virtio-balloon device to receive hints of guest free pages from the free page vq. A notifier is registered to the migration precopy notifier chain. The notifier calls

Re: [Qemu-devel] [PATCH v11 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-02-21 Thread Wei Wang
On 02/21/2019 06:18 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: On 02/20/2019 09:12 PM, Dr. David Alan Gilbert wrote: * Wang, Wei W (wei.w.w...@intel.com) wrote: On Friday, December 14, 2018 7:17 PM, Dr. David Alan Gilbert wrote: On 12/14/2018 05:56 PM, Dr

Re: [Qemu-devel] [PATCH v11 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-02-20 Thread Wei Wang
On 02/20/2019 09:12 PM, Dr. David Alan Gilbert wrote: * Wang, Wei W (wei.w.w...@intel.com) wrote: On Friday, December 14, 2018 7:17 PM, Dr. David Alan Gilbert wrote: On 12/14/2018 05:56 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: On 12/13/2018 11:45 PM, Dr

Re: [Qemu-devel] [PATCH v11 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-12-14 Thread Wei Wang
On 12/14/2018 05:56 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: On 12/13/2018 11:45 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: The new feature enables the virtio-balloon device to receive hints of guest free pages from the free

Re: [Qemu-devel] [PATCH v11 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-12-13 Thread Wei Wang
On 12/13/2018 11:45 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: The new feature enables the virtio-balloon device to receive hints of guest free pages from the free page vq. A notifier is registered to the migration precopy notifier chain. The notifier calls

Re: [Qemu-devel] [PATCH v11 1/7] bitmap: fix bitmap_count_one

2018-12-13 Thread Wei Wang
On 12/13/2018 10:28 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: BITMAP_LAST_WORD_MASK(nbits) returns 0x when "nbits=0", which makes bitmap_count_one fail to handle the "nbits=0" case. It appears to be preferred to remain BITMAP_LAST

[Qemu-devel] [PATCH v11 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-12-11 Thread Wei Wang
-by: Wei Wang CC: Michael S. Tsirkin CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Peter Xu --- hw/virtio/virtio-balloon.c | 263 include/hw/virtio/virtio-balloon.h | 28 ++- include/standard-headers/linux/virtio_balloon.h | 5 + 3

[Qemu-devel] [PATCH v11 4/7] migration: API to clear bits of guest free pages from the dirty bitmap

2018-12-11 Thread Wei Wang
This patch adds an API to clear bits corresponding to guest free pages from the dirty bitmap. Spilt the free page block if it crosses the QEMU RAMBlock boundary. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-by: Peter Xu

[Qemu-devel] [PATCH v11 1/7] bitmap: fix bitmap_count_one

2018-12-11 Thread Wei Wang
_count_one to handle the nbits=0 case. Inital Discussion Link: https://www.mail-archive.com/qemu-devel@nongnu.org/msg554316.html Signed-off-by: Wei Wang CC: Juan Quintela CC: Dr. David Alan Gilbert CC: Peter Xu --- include/qemu/bitmap.h | 4 1 file changed, 4 insertions(+) diff --git a/i

[Qemu-devel] [PATCH v11 6/7] migration/ram.c: add the free page optimization enable flag

2018-12-11 Thread Wei Wang
. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration/misc.h | 1 + migration/ram.c | 18 +- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/include/migration/misc.h b/include

[Qemu-devel] [PATCH v11 2/7] bitmap: bitmap_count_one_with_offset

2018-12-11 Thread Wei Wang
Count the number of 1s in a bitmap starting from an offset. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin Reviewed-by: Dr. David Alan Gilbert --- include/qemu/bitmap.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include

[Qemu-devel] [PATCH v11 3/7] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-12-11 Thread Wei Wang
under the mutex. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-by: Peter Xu --- migration/ram.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 7e7deec..01d267f

[Qemu-devel] [PATCH v11 0/7] virtio-balloon: free page hint support

2018-12-11 Thread Wei Wang
AMState: enable the "fpo_enabled" flag, when the free page optimization feature is used, instead of disabling ram_bulk_stage. Please see patch 6 commit log for details. Previous changelog: http://lists.nongnu.org/archive/html/qemu-devel/2018-12/msg00055.html Wei Wang (7): bitm

[Qemu-devel] [PATCH v11 5/7] migration/ram.c: add a notifier chain for precopy

2018-12-11 Thread Wei Wang
This patch adds a notifier chain for the memory precopy. This enables various precopy optimizations to be invoked at specific places. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-by: Peter Xu --- include/migration/misc.h

Re: [Qemu-devel] [PATCH v10 6/7] migration/ram.c: add a function to disable the bulk stage

2018-12-03 Thread Wei Wang
On 12/03/2018 01:31 PM, Peter Xu wrote: On Mon, Dec 03, 2018 at 10:18:30AM +0800, Wei Wang wrote: This patch adds a function to enable a precopy notifier callback outside the migration subsystem to disable the bulk stage flag. This is needed by the free page optimization offered by virtio

Re: [Qemu-devel] [PATCH v10 5/7] migration/ram.c: add a notifier chain for precopy

2018-12-03 Thread Wei Wang
On 12/03/2018 01:20 PM, Peter Xu wrote: On Mon, Dec 03, 2018 at 10:18:29AM +0800, Wei Wang wrote: This patch adds a notifier chain for the memory precopy. This enables various precopy optimizations to be invoked at specific places. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan

Re: [Qemu-devel] [virtio-dev] Re: [PATCH v10 6/7] migration/ram.c: add a function to disable the bulk stage

2018-12-03 Thread Wei Wang
On 12/03/2018 04:20 PM, Wei Wang wrote: On 12/03/2018 01:31 PM, Peter Xu wrote: On Mon, Dec 03, 2018 at 10:18:30AM +0800, Wei Wang wrote: This patch adds a function to enable a precopy notifier callback outside the migration subsystem to disable the bulk stage flag. This is needed

Re: [Qemu-devel] [PATCH v10 4/7] migration: API to clear bits of guest free pages from the dirty bitmap

2018-12-03 Thread Wei Wang
On 12/03/2018 01:10 PM, Peter Xu wrote: On Mon, Dec 03, 2018 at 10:18:28AM +0800, Wei Wang wrote: This patch adds an API to clear bits corresponding to guest free pages from the dirty bitmap. Spilt the free page block if it crosses the QEMU RAMBlock boundary. Signed-off-by: Wei Wang CC: Dr

[Qemu-devel] [PATCH v10 7/7] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-12-02 Thread Wei Wang
-by: Wei Wang CC: Michael S. Tsirkin CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Peter Xu --- hw/virtio/virtio-balloon.c | 263 include/hw/virtio/virtio-balloon.h | 28 ++- include/standard-headers/linux/virtio_balloon.h | 5 + 3

[Qemu-devel] [PATCH v10 6/7] migration/ram.c: add a function to disable the bulk stage

2018-12-02 Thread Wei Wang
This patch adds a function to enable a precopy notifier callback outside the migration subsystem to disable the bulk stage flag. This is needed by the free page optimization offered by virtio-balloon. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin

[Qemu-devel] [PATCH v10 5/7] migration/ram.c: add a notifier chain for precopy

2018-12-02 Thread Wei Wang
This patch adds a notifier chain for the memory precopy. This enables various precopy optimizations to be invoked at specific places. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration/misc.h | 19

[Qemu-devel] [PATCH v10 4/7] migration: API to clear bits of guest free pages from the dirty bitmap

2018-12-02 Thread Wei Wang
This patch adds an API to clear bits corresponding to guest free pages from the dirty bitmap. Spilt the free page block if it crosses the QEMU RAMBlock boundary. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration

[Qemu-devel] [PATCH v10 0/7] virtio-balloon: free page hint support

2018-12-02 Thread Wei Wang
te_postcopy check becasue the migration code has guaranteed that the notifier callback is invoked when in the precopy mode. Previous changelog: https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg02854.html Wei Wang (7): bitmap: fix bitmap_count_one bitmap: bitmap_count_o

[Qemu-devel] [PATCH v10 1/7] bitmap: fix bitmap_count_one

2018-12-02 Thread Wei Wang
_count_one to handle the nbits=0 case. Inital Discussion Link: https://www.mail-archive.com/qemu-devel@nongnu.org/msg554316.html Signed-off-by: Wei Wang CC: Juan Quintela CC: Dr. David Alan Gilbert CC: Peter Xu --- include/qemu/bitmap.h | 4 1 file changed, 4 insertions(+) diff --git a/i

[Qemu-devel] [PATCH v10 2/7] bitmap: bitmap_count_one_with_offset

2018-12-02 Thread Wei Wang
Count the number of 1s in a bitmap starting from an offset. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin Reviewed-by: Dr. David Alan Gilbert --- include/qemu/bitmap.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include

[Qemu-devel] [PATCH v10 3/7] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-12-02 Thread Wei Wang
under the mutex. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- migration/ram.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 7e7deec..01d267f 100644 --- a/migration

Re: [Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-29 Thread Wei Wang
On 11/30/2018 01:57 PM, Peter Xu wrote: Or you can introduce migration_bitmap_sync_precopy() and let precopy code call that one instead. Agree, thanks. PS. I'm a bit unsure on why we need to sync bitmap in ram_init_bitmaps. I feel like it can be removed... Seems it was added in early

Re: [Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-29 Thread Wei Wang
On 11/29/2018 01:10 PM, Peter Xu wrote: On Thu, Nov 29, 2018 at 11:40:57AM +0800, Wei Wang wrote: I think this precopy notifier callchain is expected to be used only for the precopy mode. Postcopy has its dedicated notifier callchain that users could use. How about changing the migrate_postcopy

Re: [Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-28 Thread Wei Wang
On 11/29/2018 01:10 PM, Peter Xu wrote: On Thu, Nov 29, 2018 at 11:40:57AM +0800, Wei Wang wrote: On 11/28/2018 05:32 PM, Peter Xu wrote: I'm not sure we can use start_postcopy. It's a variable being set in the QMP handler but it does not mean postcopy has started. I'm afraid there can

Re: [Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-28 Thread Wei Wang
On 11/28/2018 05:32 PM, Peter Xu wrote: So what I am worrying here are corner cases where we might forget to stop the hinting. I'm fabricating one example sequence of events: (start migration) START_MIGRATION BEFORE_SYNC AFTER_SYNC ... BEFORE_SYNC AFTER_SYNC (some

Re: [Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-28 Thread Wei Wang
On 11/28/2018 01:26 PM, Peter Xu wrote: Ok thanks. Please just make sure you will capture all the error cases, e.g., I also see path like this (a few lines below): if (pages < 0) { qemu_file_set_error(f, pages); break; } It seems that you missed

Re: [Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-27 Thread Wei Wang
On 11/27/2018 03:38 PM, Peter Xu wrote: On Thu, Nov 15, 2018 at 06:08:01PM +0800, Wei Wang wrote: +typedef enum PrecopyNotifyReason { +PRECOPY_NOTIFY_ERR = 0, +PRECOPY_NOTIFY_START_ITERATION = 1, +PRECOPY_NOTIFY_BEFORE_SYNC_BITMAP = 2, +PRECOPY_NOTIFY_AFTER_SYNC_BITMAP = 3

Re: [Qemu-devel] [virtio-dev] Re: [PATCH v9 3/8] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-11-27 Thread Wei Wang
On 11/27/2018 03:41 PM, Peter Xu wrote: Ok then I'm fine with it. Though you could update the comments too if you like: /* protects modification of the bitmap and migration_dirty_pages */ QemuMutex bitmap_mutex; And it's tricky that sometimes we don't take the lock when reading

Re: [Qemu-devel] [PATCH v9 4/8] migration: API to clear bits of guest free pages from the dirty bitmap

2018-11-26 Thread Wei Wang
On 11/27/2018 02:06 PM, Peter Xu wrote: On Thu, Nov 15, 2018 at 06:08:00PM +0800, Wei Wang wrote: Again, is it possible to resize during migration? So I think the check is fine, but uncertain about the comment. Yes, resize would not happen with the current implementation. But heard it could

Re: [Qemu-devel] [virtio-dev] Re: [PATCH v9 3/8] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-11-26 Thread Wei Wang
On 11/27/2018 02:02 PM, Wei Wang wrote: On 11/27/2018 01:40 PM, Peter Xu wrote: On Thu, Nov 15, 2018 at 06:07:59PM +0800, Wei Wang wrote: The bitmap mutex is used to synchronize threads to update the dirty bitmap and the migration_dirty_pages counter. For example, the free page optimization

Re: [Qemu-devel] [PATCH v9 3/8] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-11-26 Thread Wei Wang
On 11/27/2018 01:40 PM, Peter Xu wrote: On Thu, Nov 15, 2018 at 06:07:59PM +0800, Wei Wang wrote: The bitmap mutex is used to synchronize threads to update the dirty bitmap and the migration_dirty_pages counter. For example, the free page optimization clears bits of free pages from the bitmap

Re: [Qemu-devel] [PATCH v9 0/8] virtio-balloon: free page hint support

2018-11-26 Thread Wei Wang
On 11/15/2018 06:07 PM, Wei Wang wrote: This is the deivce part implementation to add a new feature, VIRTIO_BALLOON_F_FREE_PAGE_HINT to the virtio-balloon device. The device receives the guest free page hints from the driver and clears the corresponding bits in the dirty bitmap, so that those

Re: [Qemu-devel] [PATCH v9 0/8] virtio-balloon: free page hint support

2018-11-15 Thread Wei Wang
On 11/16/2018 02:50 AM, no-re...@patchew.org wrote: Hi, This series failed docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. CC net/filter.o CC net/filter-buffer.o

[Qemu-devel] [PATCH v9 8/8] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-11-15 Thread Wei Wang
-by: Wei Wang CC: Michael S. Tsirkin CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Peter Xu --- hw/virtio/virtio-balloon.c | 255 include/hw/virtio/virtio-balloon.h | 28 ++- include/standard-headers/linux/virtio_balloon.h | 5 + 3

[Qemu-devel] [PATCH v9 2/8] bitmap: bitmap_count_one_with_offset

2018-11-15 Thread Wei Wang
Count the number of 1s in a bitmap starting from an offset. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin Reviewed-by: Dr. David Alan Gilbert --- include/qemu/bitmap.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include

[Qemu-devel] [PATCH v9 6/8] migration/ram.c: add a function to disable the bulk stage

2018-11-15 Thread Wei Wang
This patch adds a function to enable a precopy notifier callback outside the migration subsystem to disable the bulk stage flag. This is needed by the free page optimization offered by virtio-balloon. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin

[Qemu-devel] [PATCH v9 3/8] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-11-15 Thread Wei Wang
under the mutex. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- migration/ram.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 7e7deec..ef69dbe 100644 --- a/migration/ram.c +++ b

[Qemu-devel] [PATCH v9 7/8] migration: move migrate_postcopy() to include/migration/misc.h

2018-11-15 Thread Wei Wang
The ram save state notifier callback, for example the free page optimization offerred by virtio-balloon, may need to check if postcopy is in use, so move migrate_postcopy() to the outside header. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC

[Qemu-devel] [PATCH v9 1/8] bitmap: fix bitmap_count_one

2018-11-15 Thread Wei Wang
_count_one to handle the nbits=0 case. Inital Discussion Link: https://www.mail-archive.com/qemu-devel@nongnu.org/msg554316.html Signed-off-by: Wei Wang CC: Juan Quintela CC: Dr. David Alan Gilbert CC: Peter Xu --- include/qemu/bitmap.h | 4 1 file changed, 4 insertions(+) diff --git a/i

[Qemu-devel] [PATCH v9 0/8] virtio-balloon: free page hint support

2018-11-15 Thread Wei Wang
is not a limit to the free page optimization now. Plan to add the device support for VIRTIO_BALLOON_F_PAGE_POISON in another patch later. Previous changelog: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg01908.html Wei Wang (8): bitmap: fix bitmap_count_one bitmap: bitmap_

[Qemu-devel] [PATCH v9 5/8] migration/ram.c: add a notifier chain for precopy

2018-11-15 Thread Wei Wang
This patch adds a notifier chain for the memory precopy. This enables various precopy optimizations to be invoked at specific places. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration/misc.h | 12

[Qemu-devel] [PATCH v9 4/8] migration: API to clear bits of guest free pages from the dirty bitmap

2018-11-15 Thread Wei Wang
This patch adds an API to clear bits corresponding to guest free pages from the dirty bitmap. Spilt the free page block if it crosses the QEMU RAMBlock boundary. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration

Re: [Qemu-devel] [PATCH v2] bitmap: fix BITMAP_LAST_WORD_MASK

2018-08-07 Thread Wei Wang
On 08/07/2018 05:53 PM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: On 08/07/2018 03:39 PM, Peter Xu wrote: On Tue, Jul 31, 2018 at 06:01:18PM +0800, Wei Wang wrote: When "nbits = 0", which means no bits to mask, this macro is expected to return

Re: [Qemu-devel] [PATCH v2] bitmap: fix BITMAP_LAST_WORD_MASK

2018-08-07 Thread Wei Wang
On 08/07/2018 08:17 PM, Peter Xu wrote: On Tue, Aug 07, 2018 at 04:21:15PM +0800, Wei Wang wrote: On 08/07/2018 03:39 PM, Peter Xu wrote: On Tue, Jul 31, 2018 at 06:01:18PM +0800, Wei Wang wrote: When "nbits = 0", which means no bits to mask, this macro is expected to return

Re: [Qemu-devel] [PATCH v2] bitmap: fix BITMAP_LAST_WORD_MASK

2018-08-07 Thread Wei Wang
On 08/07/2018 03:39 PM, Peter Xu wrote: On Tue, Jul 31, 2018 at 06:01:18PM +0800, Wei Wang wrote: When "nbits = 0", which means no bits to mask, this macro is expected to return 0, instead of 0x. This patch changes the macro to return 0 when there is no bit needs to be maske

[Qemu-devel] [PATCH v2] bitmap: fix BITMAP_LAST_WORD_MASK

2018-07-31 Thread Wei Wang
When "nbits = 0", which means no bits to mask, this macro is expected to return 0, instead of 0x. This patch changes the macro to return 0 when there is no bit needs to be masked. Signed-off-by: Wei Wang CC: Juan Quintela CC: Dr. David Alan Gilbert CC: Peter Xu --- in

Re: [Qemu-devel] [PATCH 1/2] bitmap.h: add comments to BITMAP_LAST_WORD_MASK

2018-07-31 Thread Wei Wang
On 07/30/2018 09:19 PM, Juan Quintela wrote: Wei Wang wrote: On 07/30/2018 03:13 PM, Wei Wang wrote: This macro was ported from Linux and we've reached an aggreement there that the corner case "nbits = 0" is not applicable to this macro, because when "nbits = 0", which m

Re: [Qemu-devel] [PATCH 1/2] bitmap.h: add comments to BITMAP_LAST_WORD_MASK

2018-07-30 Thread Wei Wang
On 07/30/2018 03:13 PM, Wei Wang wrote: This macro was ported from Linux and we've reached an aggreement there that the corner case "nbits = 0" is not applicable to this macro, because when "nbits = 0", which means no bits to mask, this macro is expected to return 0, i

[Qemu-devel] [PATCH 1/2] bitmap.h: add comments to BITMAP_LAST_WORD_MASK

2018-07-30 Thread Wei Wang
adds a comment above the macro as a note to users about the corner case. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Peter Xu --- include/qemu/bitmap.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h index 509eedd..f53c640

[Qemu-devel] [PATCH 0/2] bitmap: some fixes

2018-07-30 Thread Wei Wang
"nbits = 0" is not applicable to BITMAP_LAST_WORD_MASK, callers should avoid that. Fix bitmap_count_one to avoid passing 0 to the macro. Wei Wang (2): bitmap.h: add comments to BITMAP_LAST_WORD_MASK bitmap: fix bitmap_count_one include/qemu/bitmap.h | 3 ++- 1 file changed, 2

[Qemu-devel] [PATCH 2/2] bitmap: fix bitmap_count_one

2018-07-30 Thread Wei Wang
Since "nbits = 0" is not applicable to the BITMAP_LAST_WORD_MASK macro, callers need to avoid passing "nbits = 0" to this macro, which generates incorrect results. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Peter Xu --- include/qemu/bit

Re: [Qemu-devel] [PATCH 09/12] ring: introduce lockless ring buffer

2018-06-28 Thread Wei Wang
On 06/28/2018 06:02 PM, Xiao Guangrong wrote: CC: Paul, Peter Zijlstra, Stefani, Lai who are all good at memory barrier. On 06/20/2018 12:52 PM, Peter Xu wrote: On Mon, Jun 04, 2018 at 05:55:17PM +0800, guangrong.x...@gmail.com wrote: From: Xiao Guangrong It's the simple lockless ring

Re: [Qemu-devel] [PATCH v8 4/6] migration/ram.c: add ram save state notifiers

2018-06-20 Thread Wei Wang
On 06/20/2018 01:31 AM, Dr. David Alan Gilbert wrote: * Wei Wang (wei.w.w...@intel.com) wrote: On 06/12/2018 03:50 PM, Peter Xu wrote: On Fri, Jun 08, 2018 at 04:10:41PM +0800, Wei Wang wrote: This patch adds a ram save state notifier list, and expose RAMState for the notifer callbacks to use

Re: [Qemu-devel] [PATCH v8 4/6] migration/ram.c: add ram save state notifiers

2018-06-12 Thread Wei Wang
On 06/12/2018 03:50 PM, Peter Xu wrote: On Fri, Jun 08, 2018 at 04:10:41PM +0800, Wei Wang wrote: This patch adds a ram save state notifier list, and expose RAMState for the notifer callbacks to use. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin

[Qemu-devel] [PATCH v8 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-06-08 Thread Wei Wang
: - If free pages are poisoned by guest, the hints are dropped currently. We will support clearing bits of poisoned pages from the bitmap in the future. Signed-off-by: Wei Wang CC: Michael S. Tsirkin CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Peter Xu --- hw/virtio/virtio-balloon.c

[Qemu-devel] [PATCH v8 4/6] migration/ram.c: add ram save state notifiers

2018-06-08 Thread Wei Wang
This patch adds a ram save state notifier list, and expose RAMState for the notifer callbacks to use. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration/misc.h | 52 +++ migration

[Qemu-devel] [PATCH v8 2/6] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2018-06-08 Thread Wei Wang
under the mutex. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- migration/ram.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index c53e836..2eabbe9 100644 --- a/migration/ram.c +++ b

  1   2   3   4   5   6   7   >