[PATCH 2/2] migration/postcopy: map large zero page in postcopy_ram_incoming_setup()

2019-10-05 Thread Wei Yang
postcopy_ram_incoming_setup() and postcopy_ram_incoming_cleanup() are counterpart. It is reasonable to map/unmap large zero page in these two functions respectively. Signed-off-by: Wei Yang --- migration/postcopy-ram.c | 34 +- 1 file changed, 17 insertions

[PATCH 1/2] migration/postcopy: allocate tmp_page in setup stage

2019-10-05 Thread Wei Yang
it. This patch moves the allocation to setup stage and if failed error message would be printed and caller would notice it. Signed-off-by: Wei Yang --- migration/postcopy-ram.c | 40 ++-- migration/postcopy-ram.h | 7 --- migration/ram.c | 2

[PATCH 0/2] migration/postcopy: map tmp and large zero page in setup stage

2019-10-05 Thread Wei Yang
Currently we map these page when we want to use it, while this may be a little late. To make the code consistency, these two patches move the map into postcopy_ram_incoming_setup. Wei Yang (2): migration/postcopy: allocate tmp_page in setup stage migration/postcopy: map large zero page

[PATCH 3/3] migration/postcopy: handle POSTCOPY_INCOMING_RUNNING corner case properly

2019-10-01 Thread Wei Yang
ve yet. This patch makes sure state is set to RUNNING only previous state is LISTENING by introducing an old_state parameter in postcopy_state_set(). New state only would be set when current state equals to old_state. Signed-off-by: Wei Yang --- migration/migration.c| 2 +- migration/postcopy-ram.

[PATCH 2/3] migration/postcopy: not necessary to do postcopy_ram_incoming_cleanup when state is ADVISE

2019-10-01 Thread Wei Yang
postcopy_ram_incoming_cleanup() does cleanup for postcopy_ram_incoming_setup(), while the setup happens only after migration enters LISTEN state. This means there is nothing to cleanup when migration is still ADVISE state. Signed-off-by: Wei Yang --- migration/migration.c | 1 - 1 file changed

[PATCH 0/3] migration/postcopy: cleanup related to postcopy

2019-10-01 Thread Wei Yang
The first one just tries to make function name more easy for reading and understanding. The next two patches are related to PostcopyState. Wei Yang (3): migration/postcopy: rename postcopy_ram_enable_notify to postcopy_ram_incoming_setup migration/postcopy: not necessary to do

[PATCH 1/3] migration/postcopy: rename postcopy_ram_enable_notify to postcopy_ram_incoming_setup

2019-10-01 Thread Wei Yang
Function postcopy_ram_incoming_setup and postcopy_ram_incoming_cleanup is a pair. Rename to make it clear for audience. Signed-off-by: Wei Yang --- migration/postcopy-ram.c | 4 ++-- migration/postcopy-ram.h | 2 +- migration/savevm.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions

Re: [Qemu-devel] [PATCH] migration: use migration_is_active to represent active state

2019-10-01 Thread Wei Yang
On Fri, Sep 13, 2019 at 11:54:33PM +, Wei Yang wrote: >On Wed, Jul 17, 2019 at 08:53:41AM +0800, Wei Yang wrote: >>Wrap the check into a function to make it easy to read. >> > >Hi, Dave & Juan > >Do you like this one :-) ? > Sorry for ping again. >>S

Re: [Qemu-devel] [PATCH] migration: check length directly to make sure the range is aligned

2019-10-01 Thread Wei Yang
discard_range: Unaligned start address: %p", >> host_startaddr); >> goto err; >> } >> >> just before this context. >> >> Paolo >-- >Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK -- Wei Yang Help you, Help me

Re: [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap

2019-09-24 Thread Wei Yang
On Tue, Sep 24, 2019 at 11:02:08AM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> All pages, either partially sent or partially dirty, will be discarded in >> postcopy_send_discard_bm_ram(), since we update the unsentmap to be >>

Re: [PATCH 0/3] migration/postcopy: unsentmap is not necessary

2019-09-22 Thread Wei Yang
On Fri, Sep 20, 2019 at 07:53:51PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> Hi, Dave and Juan >> >> Would you mind taking a look at this one? > >Yes, I'll have a look, I'm just fighting through a bunch of other stu

Re: [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary

2019-09-16 Thread Wei Yang
Hi, Dave and Juan Would you mind taking a look at this one? On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote: >Three patches to cleanup postcopy: > >[1]: split canonicalize bitmap and discard page >[2]: remove unsentmap since it is not necessary &

Re: [Qemu-devel] [PATCH 2/6] exec.c: remove an unnecessary assert on PHYS_MAP_NODE_NIL in phys_map_node_alloc()

2019-09-15 Thread Wei Yang
On Fri, Sep 13, 2019 at 11:12:05AM +0200, Paolo Bonzini wrote: >On 13/09/19 01:02, Wei Yang wrote: >> It shows PHYS_MAP_NODE_NIL may represents more node the tree could hold. > >Which is good, it means the assert can trigger. > Per my understanding, it means the as

Re: [Qemu-devel] [PATCH] migration: check length directly to make sure the range is aligned

2019-09-13 Thread Wei Yang
On Fri, Jul 19, 2019 at 06:54:00PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> Since the start addr is already checked, to make sure the range is >> aligned, checking the length is enough. >> >> Signed-off-by: Wei

Re: [Qemu-devel] [PATCH] migration: use migration_is_active to represent active state

2019-09-13 Thread Wei Yang
On Wed, Jul 17, 2019 at 08:53:41AM +0800, Wei Yang wrote: >Wrap the check into a function to make it easy to read. > Hi, Dave & Juan Do you like this one :-) ? >Signed-off-by: Wei Yang >--- > include/migration/misc.h | 1 + > migration/migration.c| 12 ---

Re: [Qemu-devel] [PATCH v2 0/2] refine memory_device_get_free_addr

2019-09-13 Thread Wei Yang
On Tue, Jul 30, 2019 at 08:37:38AM +0800, Wei Yang wrote: >When we iterate the memory-device list to get the available range, it is not >necessary to iterate the whole list. > >1) no more overlap for hinted range if tmp exceed it > >v2: > * remove #2 as suggested by Igor and

Re: [Qemu-devel] [PATCH 2/6] exec.c: remove an unnecessary assert on PHYS_MAP_NODE_NIL in phys_map_node_alloc()

2019-09-12 Thread Wei Yang
On Thu, Sep 12, 2019 at 02:42:26PM +0200, Paolo Bonzini wrote: >On 12/09/19 04:51, Wei Yang wrote: >> On Fri, Aug 23, 2019 at 09:07:50AM +0800, Wei Yang wrote: >>> On Thu, Aug 22, 2019 at 12:24:32PM +0200, Paolo Bonzini wrote: >>>> On 21/03/19 09:25, Wei Yang

Re: [Qemu-devel] [PATCH v2] docs/nvdimm: add example on persistent backend setup

2019-09-12 Thread Wei Yang
On Thu, Sep 12, 2019 at 02:16:00PM +0200, Stefan Hajnoczi wrote: >On Thu, Aug 01, 2019 at 08:40:53AM +0800, Wei Yang wrote: >> Persistent backend setup requires some knowledge about nvdimm and ndctl >> tool. Some users report they may struggle to gather these knowledge and >

Re: [Qemu-devel] [PATCH] migration: fix one typo in comment of function migration_total_bytes()

2019-09-12 Thread Wei Yang
On Thu, Sep 12, 2019 at 09:44:41AM +0200, Juan Quintela wrote: >Wei Yang wrote: >> Signed-off-by: Wei Yang > >Reviewed-by: Juan Quintela > >for(i = 0; i < 0; i++) > printf("Beginning is with double n, not double g"); :-) -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH 2/6] exec.c: remove an unnecessary assert on PHYS_MAP_NODE_NIL in phys_map_node_alloc()

2019-09-11 Thread Wei Yang
On Fri, Aug 23, 2019 at 09:07:50AM +0800, Wei Yang wrote: >On Thu, Aug 22, 2019 at 12:24:32PM +0200, Paolo Bonzini wrote: >>On 21/03/19 09:25, Wei Yang wrote: >>> PHYS_MAP_NODE_NIL is assigned to PhysPageEntry.ptr in case this is not a >>> leaf entry, whil

[Qemu-devel] [PATCH] migration: fix one typo in comment of function migration_total_bytes()

2019-09-11 Thread Wei Yang
Signed-off-by: Wei Yang --- migration/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index fbdabd34d9..cae3e894ad 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3023,7 +3023,7 @@ static MigThrError

[Qemu-devel] [PATCH V2 1/2] migration/qemu-file: remove check on writev_buffer in qemu_put_compression_data

2019-09-11 Thread Wei Yang
From: Wei Yang The check of writev_buffer is in qemu_fflush, which means it is not harmful if it is NULL. And removing it will make the code consistent since all other add_to_iovec() is called without the check. Signed-off-by: Wei Yang Reviewed-by: Dr. David Alan Gilbert --- migration/qemu

[Qemu-devel] [PATCH V2 2/2] migration/qemu-file: fix potential buf waste for extra buf_index adjustment

2019-09-11 Thread Wei Yang
From: Wei Yang In add_to_iovec(), qemu_fflush() will be called if iovec is full. If this happens, buf_index is reset. Currently, this is not checked and buf_index would always been adjust with buf size. This is not harmful, but will waste some space in file buffer. This patch make add_to_iovec

[Qemu-devel] [PATCH V2 0/2] migration/qemu-file: cleanup and refine qemu-file

2019-09-11 Thread Wei Yang
Two cleanup: Patch #1 make code consistent on calling add_to_iovec Patch #2 refine the code to handle the case when buf already flushed v2: * wrap common steps into add_buf_to_iovec() Wei Yang (2): migration/qemu-file: remove check on writev_buffer in qemu_put_compression_data

Re: [Qemu-devel] [PATCH 2/2] migration/qemu-file: fix potential buf waste for extra buf_index adjustment

2019-09-11 Thread Wei Yang
On Tue, Sep 03, 2019 at 07:43:24PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richard.weiy...@gmail.com) wrote: >> On Fri, Aug 23, 2019 at 12:06:09PM +0100, Dr. David Alan Gilbert wrote: >> >(Copying Dan in) >> > >> >* Wei Yang (richardw.y...@linux.int

Re: [Qemu-devel] [PATCH v2] docs/nvdimm: add example on persistent backend setup

2019-09-11 Thread Wei Yang
On Thu, Aug 01, 2019 at 08:40:53AM +0800, Wei Yang wrote: >Persistent backend setup requires some knowledge about nvdimm and ndctl >tool. Some users report they may struggle to gather these knowledge and >have difficulty to setup it properly. > >Here we provide two examples for per

Re: [Qemu-devel] [RFC PATCH] migration/postcopy: enable compress with postcopy

2019-08-26 Thread Wei Yang
On Fri, Aug 23, 2019 at 04:59:19PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> This patch enable compress with postcopy. >> >> This is a RFC and based on some unmerged patch >> >> "migration: extract ram_

Re: [Qemu-devel] [RFC PATCH] migration/postcopy: skip compression when postcopy is active

2019-08-26 Thread Wei Yang
On Fri, Aug 23, 2019 at 05:27:01PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> Now postcopy is not compatible with compression. And we disable setting >> these two capability at the same time. While we can still leverage >> com

Re: [Qemu-devel] [PATCH 2/2] migration/qemu-file: fix potential buf waste for extra buf_index adjustment

2019-08-24 Thread Wei Yang
On Fri, Aug 23, 2019 at 02:05:02PM +0100, Dr. David Alan Gilbert wrote: >* Daniel P. Berrang? (berra...@redhat.com) wrote: >> On Fri, Aug 23, 2019 at 12:06:09PM +0100, Dr. David Alan Gilbert wrote: >> > (Copying Dan in) >> > >> > * Wei Yang (

Re: [Qemu-devel] [PATCH 2/2] migration/qemu-file: fix potential buf waste for extra buf_index adjustment

2019-08-24 Thread Wei Yang
On Fri, Aug 23, 2019 at 12:06:09PM +0100, Dr. David Alan Gilbert wrote: >(Copying Dan in) > >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> In add_to_iovec(), qemu_fflush() will be called if iovec is full. If >> this happens, buf_index is reset. Currently

Re: [Qemu-devel] [PATCH] migrtion: define MigrationState/MigrationIncomingState.state as MigrationStatus

2019-08-23 Thread Wei Yang
On Fri, Aug 23, 2019 at 11:21:50AM -0500, Eric Blake wrote: >On 8/19/19 9:08 AM, Wei Yang wrote: >> On Mon, Aug 19, 2019 at 12:26:32PM +0100, Dr. David Alan Gilbert wrote: >>> * Wei Yang (richardw.y...@linux.intel.com) wrote: > >Typo in the subject line: migr

Re: [Qemu-devel] [PATCH 2/6] exec.c: remove an unnecessary assert on PHYS_MAP_NODE_NIL in phys_map_node_alloc()

2019-08-22 Thread Wei Yang
On Thu, Aug 22, 2019 at 12:24:32PM +0200, Paolo Bonzini wrote: >On 21/03/19 09:25, Wei Yang wrote: >> PHYS_MAP_NODE_NIL is assigned to PhysPageEntry.ptr in case this is not a >> leaf entry, while map->nodes_nb range in [0, nodes_nb_alloc). >> >> Seems we are ass

Re: [Qemu-devel] [PATCH 0/6] Refine exec

2019-08-22 Thread Wei Yang
On Thu, Aug 22, 2019 at 12:25:44PM +0200, Paolo Bonzini wrote: >On 19/08/19 05:06, Wei Yang wrote: >> On Thu, Mar 21, 2019 at 04:25:49PM +0800, Wei Yang wrote: >>> This serial refine exec a little. >>> >> >> Ping again. > >Queued all except 2,

Re: [Qemu-devel] [PATCH] tests: make filemonitor test more robust to event ordering

2019-08-21 Thread Wei Yang
pe with this, when we see an unexpected event, we >push it back onto the queue and look and the subsequent >event to see if that matches instead. > >Signed-off-by: Daniel P. Berrangé Tested-by: Wei Yang >--- > tests/test-util-filemonitor.c | 43 +++ >

Re: [Qemu-devel] [PATCH] migrtion: define MigrationState/MigrationIncomingState.state as MigrationStatus

2019-08-21 Thread Wei Yang
On Mon, Aug 19, 2019 at 12:26:55PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> On Fri, Jun 21, 2019 at 10:27:39PM +0800, Wei Yang wrote: >> >No functional change. Add default case to fix warning. >> > >> >&

Re: [Qemu-devel] [PATCH v2] migration: cleanup check on ops in savevm.handlers iterations

2019-08-19 Thread Wei Yang
On Mon, Aug 19, 2019 at 03:50:44PM +0200, Juan Quintela wrote: >Wei Yang wrote: >> During migration, there are several places to iterate on >> savevm.handlers. And on each iteration, we need to check its ops and >> related callbacks before invoke it. >> >> Ge

Re: [Qemu-devel] [PATCH] migrtion: define MigrationState/MigrationIncomingState.state as MigrationStatus

2019-08-19 Thread Wei Yang
On Mon, Aug 19, 2019 at 12:26:32PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> No functional change. Add default case to fix warning. > >I think the problem with this is that migrate_set_state uses an >atomic_cmpxchg and so we

Re: [Qemu-devel] [PATCH] migrtion: define MigrationState/MigrationIncomingState.state as MigrationStatus

2019-08-19 Thread Wei Yang
On Mon, Aug 19, 2019 at 12:26:55PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> On Fri, Jun 21, 2019 at 10:27:39PM +0800, Wei Yang wrote: >> >No functional change. Add default case to fix warning. >> > >> >&

Re: [Qemu-devel] [PATCH v2 1/2] memory-device: not necessary to use goto for the last check

2019-08-19 Thread Wei Yang
On Mon, Aug 19, 2019 at 05:32:14AM +, Zeng, Star wrote: > > >> -Original Message----- >> From: Wei Yang [mailto:richardw.y...@linux.intel.com] >> Sent: Monday, August 19, 2019 10:39 AM >> To: David Hildenbrand >> Cc: Wei Yang ; Zeng, Star >> ;

[Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap

2019-08-19 Thread Wei Yang
separate the page discard into two individual steps: * canonicalize bitmap * discard page Signed-off-by: Wei Yang --- migration/ram.c | 14 +- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 35552c090b..075ddc468c 100644

[Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty

2019-08-19 Thread Wei Yang
This is a cleanup for previous removal of unsentmap. The sent parameter is not necessary now. Signed-off-by: Wei Yang --- migration/ram.c| 2 +- migration/trace-events | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index

[Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary

2019-08-19 Thread Wei Yang
Three patches to cleanup postcopy: [1]: split canonicalize bitmap and discard page [2]: remove unsentmap since it is not necessary [3]: cleanup the get_queued_page_not_dirty Wei Yang (3): migration/postcopy: not necessary to do discard when canonicalizing bitmap migration/postcopy

[Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy

2019-08-19 Thread Wei Yang
removes it. Signed-off-by: Wei Yang --- include/exec/ram_addr.h | 6 migration/ram.c | 80 - 2 files changed, 14 insertions(+), 72 deletions(-) diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index a327a80cfe..e96e621de5 100644

Re: [Qemu-devel] [PATCH] migrtion: define MigrationState/MigrationIncomingState.state as MigrationStatus

2019-08-18 Thread Wei Yang
On Fri, Jun 21, 2019 at 10:27:39PM +0800, Wei Yang wrote: >No functional change. Add default case to fix warning. > Hi, David & Juan Do you like this? >Signed-off-by: Wei Yang >--- > migration/migration.c | 8 +++- > migration/migration.h | 6 +++--- > 2 files cha

[Qemu-devel] [PATCH v2] migration: cleanup check on ops in savevm.handlers iterations

2019-08-18 Thread Wei Yang
in savevm.c to check ops only once in those iterations. Signed-off-by: Wei Yang --- v2: rebase on latest upstream --- migration/savevm.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 4a86128ac4..35426d1db8 100644

Re: [Qemu-devel] [PATCH 0/6] Refine exec

2019-08-18 Thread Wei Yang
On Thu, Mar 21, 2019 at 04:25:49PM +0800, Wei Yang wrote: >This serial refine exec a little. > Ping again. >Wei Yang (6): > exec.c: replace hwaddr with uint64_t for better understanding > exec.c: remove an unnecessary assert on PHYS_MAP_NODE_NIL in >phys_map_node_allo

Re: [Qemu-devel] [PATCH v2 1/2] memory-device: not necessary to use goto for the last check

2019-08-18 Thread Wei Yang
On Thu, Aug 08, 2019 at 09:06:21AM +0200, David Hildenbrand wrote: >On 08.08.19 04:38, Wei Yang wrote: >> On Thu, Aug 08, 2019 at 02:30:02AM +, Zeng, Star wrote: >>>> -Original Message- >>>> From: Wei Yang [mailto:richardw.y...@linux.intel.com] >>

Re: [Qemu-devel] [PATCH 0/2] migration/qemu-file: cleanup and refine qemu-file

2019-08-18 Thread Wei Yang
On Wed, Jul 31, 2019 at 10:42:23PM +0800, Wei Yang wrote: >Two cleanup: > >Patch #1 make code consistent on calling add_to_iovec >Patch #2 refine the code to handle the case when buf already flushed > Ping~ >Wei Yang (2): > migration/qemu-file: remove c

Re: [Qemu-devel] [PATCH] migration: check length directly to make sure the range is aligned

2019-08-18 Thread Wei Yang
On Fri, Jul 19, 2019 at 06:54:00PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> Since the start addr is already checked, to make sure the range is >> aligned, checking the length is enough. >> >> Signed-off-by: Wei

Re: [Qemu-devel] [PATCH] test-bitmap: test set 1 bit case for bitmap_set

2019-08-14 Thread Wei Yang
On Wed, Aug 14, 2019 at 02:38:24PM +0200, Paolo Bonzini wrote: >On 14/08/19 02:27, Wei Yang wrote: >> All current bitmap_set test cases set range across word, while the >> handle of a range within one word is different from that. >> >> Add case to set 1 bit as a represe

[Qemu-devel] [PATCH] test-bitmap: test set 1 bit case for bitmap_set

2019-08-13 Thread Wei Yang
All current bitmap_set test cases set range across word, while the handle of a range within one word is different from that. Add case to set 1 bit as a represent for set range within one word. Signed-off-by: Wei Yang --- Thanks for Paolo's finding. --- tests/test-bitmap.c | 12

Re: [Qemu-devel] [PATCH v3 1/2] bitmap: get last word mask from nr directly

2019-08-13 Thread Wei Yang
On Mon, Aug 12, 2019 at 06:18:43PM +0200, Paolo Bonzini wrote: >On 18/07/19 03:04, Wei Yang wrote: >> The value left in nr is the number of bits for the last word, which >> could be calculate the last word mask directly. >> >> Remove the unnecessary size. > >Hi,

Re: [Qemu-devel] [Fail] tests/test-util-filemonitor fails

2019-08-11 Thread Wei Yang
On Fri, Aug 09, 2019 at 10:12:14AM +0100, Daniel P. Berrangé wrote: >On Fri, Aug 09, 2019 at 08:06:09AM +0800, Wei Yang wrote: >> On Thu, Aug 08, 2019 at 10:22:13AM +0100, Daniel P. Berrangé wrote: >> >On Thu, Aug 08, 2019 at 04:46:53PM +0800, Wei Yang wrote: >> >>

Re: [Qemu-devel] [Fail] tests/test-util-filemonitor fails

2019-08-08 Thread Wei Yang
On Thu, Aug 08, 2019 at 10:22:13AM +0100, Daniel P. Berrangé wrote: >On Thu, Aug 08, 2019 at 04:46:53PM +0800, Wei Yang wrote: >> On Thu, Aug 08, 2019 at 09:02:29AM +0100, Daniel P. Berrangé wrote: >> >On Thu, Aug 08, 2019 at 10:07:23AM +0800, Wei Yang wrote: >> >>

Re: [Qemu-devel] [Fail] tests/test-util-filemonitor fails

2019-08-08 Thread Wei Yang
On Thu, Aug 08, 2019 at 09:02:29AM +0100, Daniel P. Berrangé wrote: >On Thu, Aug 08, 2019 at 10:07:23AM +0800, Wei Yang wrote: >> Current qemu fails tests/test-util-filemonitor. > >You'll need to provide more info. The test works for me and passes in all >the QEMU CI environm

[Qemu-devel] [PATCH v2] migration: rename migration_bitmap_sync_range to ramblock_sync_dirty_bitmap

2019-08-07 Thread Wei Yang
Rename for better understanding of the code. Suggested-by: Paolo Bonzini Signed-off-by: Wei Yang --- v2: * rebase on top of "just pass RAMBlock is enough" --- migration/ram.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/r

Re: [Qemu-devel] [PATCH] migration: rename migration_bitmap_sync_range to ramblock_sync_dirty_bitmap

2019-08-07 Thread Wei Yang
On Wed, Aug 07, 2019 at 06:49:48PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> Rename for better understanding of the code. >> >> Suggested-by: Paolo Bonzini >> Signed-off-by: Wei Yang > >this needs fixing after 'ju

Re: [Qemu-devel] [PATCH v2 1/2] memory-device: not necessary to use goto for the last check

2019-08-07 Thread Wei Yang
On Thu, Aug 08, 2019 at 02:30:02AM +, Zeng, Star wrote: >> -Original Message- >> From: Wei Yang [mailto:richardw.y...@linux.intel.com] >> Sent: Thursday, August 8, 2019 10:13 AM >> To: Zeng, Star >> Cc: Wei Yang ; qemu-devel@nongnu.org; >> imam

Re: [Qemu-devel] [PATCH v2 1/2] memory-device: not necessary to use goto for the last check

2019-08-07 Thread Wei Yang
On Thu, Aug 08, 2019 at 01:42:14AM +, Zeng, Star wrote: >> -Original Message- >> From: Qemu-devel [mailto:qemu-devel- >> bounces+star.zeng=intel@nongnu.org] On Behalf Of Wei Yang >> Sent: Tuesday, July 30, 2019 8:38 AM >> To: qemu-devel@nongnu.org

[Qemu-devel] [Fail] tests/test-util-filemonitor fails

2019-08-07 Thread Wei Yang
h IDs are unique to QFileMonitor scope" "filemon: fix watch IDs to avoid potential wraparound issues" The test pass. -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH v3] migration/postcopy: use mis->bh instead of allocating a QEMUBH

2019-08-07 Thread Wei Yang
On Wed, Aug 07, 2019 at 07:35:34PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> For migration incoming side, it either quit in precopy or postcopy. It >> is safe to use the mis->bh for both instead of allocating a dedicated >

Re: [Qemu-devel] [PATCH v3] migration/postcopy: use mis->bh instead of allocating a QEMUBH

2019-08-07 Thread Wei Yang
On Wed, Aug 07, 2019 at 07:35:34PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> For migration incoming side, it either quit in precopy or postcopy. It >> is safe to use the mis->bh for both instead of allocating a dedicated >

Re: [Qemu-devel] [PATCH] migration: rename migration_bitmap_sync_range to ramblock_sync_dirty_bitmap

2019-08-07 Thread Wei Yang
On Wed, Aug 07, 2019 at 06:49:48PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> Rename for better understanding of the code. >> >> Suggested-by: Paolo Bonzini >> Signed-off-by: Wei Yang > >this needs fixing af

Re: [Qemu-devel] [PATCH 1/2] migration/postcopy: simplify calculation of run_start and fixup_start_addr

2019-08-06 Thread Wei Yang
On Tue, Aug 06, 2019 at 03:51:17PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> The purpose of the calculation is to find a HostPage which is partially >> dirty. >> >> * fixup_start_addr points to the start

[Qemu-devel] [PATCH 2/2] migration/postcopy: use QEMU_IS_ALIGNED to replace host_offset

2019-08-05 Thread Wei Yang
Use QEMU_IS_ALIGNED for the check, it would be more consistent with other align calculations. Signed-off-by: Wei Yang --- migration/ram.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index d86661a015..dfbf71c580 100644

[Qemu-devel] [PATCH 0/2] migration/postcopy: simplify postcopy_chunk_hostpages_pass

2019-08-05 Thread Wei Yang
When looking into function postcopy_chunk_hostpages_pass(), we could use alignment calculation to simplify it. Wei Yang (2): migration/postcopy: simplify calculation of run_start and fixup_start_addr migration/postcopy: use QEMU_IS_ALIGNED to replace host_offset migration/ram.c | 37

[Qemu-devel] [PATCH 1/2] migration/postcopy: simplify calculation of run_start and fixup_start_addr

2019-08-05 Thread Wei Yang
with: * fixup_start_addr = QEMU_ALIGN_DOWN(run_start, host_ratio) * run_start = QEMU_ALIGN_UP(run_start, host_ratio) By doing so, run_start always points to the next HostPage to check. fixup_start_addr always points to the HostPage to discard. Signed-off-by: Wei Yang --- migration/ram.c | 34

[Qemu-devel] [PATCH v3] hmp: Remove migration capabilities from "info migrate"

2019-08-05 Thread Wei Yang
With the growth of migration capabilities, it is not proper to display them in "info migrate". Users are recommended to use "info migrate_capabiltiies" to list them. Signed-off-by: Wei Yang Suggested-by: Dr. David Alan Gilbert --- v3: * remove un-used variable

Re: [Qemu-devel] [PATCH v2] hmp: Remove migration capabilities from "info migrate"

2019-08-05 Thread Wei Yang
On Mon, Aug 05, 2019 at 12:12:25PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> With the growth of migration capabilities, it is not proper to display >> them in "info migrate". Users are recommended to use "info >>

[Qemu-devel] [PATCH v3] migration/postcopy: use mis->bh instead of allocating a QEMUBH

2019-08-04 Thread Wei Yang
For migration incoming side, it either quit in precopy or postcopy. It is safe to use the mis->bh for both instead of allocating a dedicated QEMUBH for postcopy. Signed-off-by: Wei Yang Reviewed-by: Dr. David Alan Gilbert --- v3: rebase on latest upstream v2: fix a typo in change

Re: [Qemu-devel] [PATCH] migration/postcopy: PostcopyState is already set in loadvm_postcopy_handle_advise()

2019-08-04 Thread Wei Yang
On Thu, Jul 11, 2019 at 04:08:16PM +0800, Wei Yang wrote: >PostcopyState is already set to ADVISE at the beginning of >loadvm_postcopy_handle_advise(). > >Remove the redundant set. Ping~ > >Signed-off-by: Wei Yang >--- > migration/savevm.c | 2 -- > 1 file changed, 2

Re: [Qemu-devel] [PATCH v3] migration: always initial ram_counters for a new migration

2019-08-04 Thread Wei Yang
On Mon, Aug 05, 2019 at 09:16:24AM +0800, Ivan Ren wrote: >On Mon, Aug 5, 2019 at 8:34 AM Wei Yang wrote: >> >> On Fri, Aug 02, 2019 at 06:18:41PM +0800, Ivan Ren wrote: >> >From: Ivan Ren >> > >> >This patch fix a multifd migration bug in migrati

Re: [Qemu-devel] [PATCH v3] migration: always initial ram_counters for a new migration

2019-08-04 Thread Wei Yang
in migration_iteration_run >after the first migration_update_counters. > >Signed-off-by: Ivan Ren >Reviewed-by: Juan Quintela >Suggested-by: Wei Yang >--- >v2->v3: >- fix the bug of update_iteration_initial_status function prototype > Code looks good. Have you verified on this version? BTW, you didn't address the multifd count in this patch, right? -- Wei Yang Help you, Help me

[Qemu-devel] [PATCH v2] hmp: Remove migration capabilities from "info migrate"

2019-08-04 Thread Wei Yang
With the growth of migration capabilities, it is not proper to display them in "info migrate". Users are recommended to use "info migrate_capabiltiies" to list them. Signed-off-by: Wei Yang Suggested-by: Dr. David Alan Gilbert --- v2: * remove capabilities from "i

Re: [Qemu-devel] [PATCH] hmp/info_migration: formatting migration capability output

2019-08-02 Thread Wei Yang
On Fri, Aug 02, 2019 at 11:02:20AM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.y...@linux.intel.com) wrote: >> On Fri, Aug 02, 2019 at 07:24:34AM +0200, Markus Armbruster wrote: >> >Subject doesn't quite conform to conventions. Suggest >> > >> &g

[Qemu-devel] [RFC PATCH] migration/postcopy: enable compress with postcopy

2019-08-02 Thread Wei Yang
This patch enable compress with postcopy. This is a RFC and based on some unmerged patch "migration: extract ram_load_precopy" "migration/postcopy: skip compression when postcopy is active" Signed-off-by: Wei Yang --- migration/postcopy-ram.c | 3 +-- migration

Re: [Qemu-devel] [PATCH] migration: always initial ram_counters for a new migration

2019-08-02 Thread Wei Yang
es transferred by multifd threads >is >counted by ram_counters.normal. >You mean other multifd bytes like multifd packet or multifd sync info? > Ok, it is counted in normal. While if my understanding is correct, normal is used to count pages sent by save_normal_page(). Sounds this is misused? >Thanks. > >On Fri, Aug 2, 2019 at 8:49 AM Wei Yang >wrote: > -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH] hmp/info_migration: formatting migration capability output

2019-08-01 Thread Wei Yang
On Fri, Aug 02, 2019 at 07:24:34AM +0200, Markus Armbruster wrote: >Subject doesn't quite conform to conventions. Suggest > >hmp: Improve how "info migrate" formats capabilities Thanks > >Wei Yang writes: > >> Current we put all migration capabili

[Qemu-devel] [PATCH] hmp/info_migration: formatting migration capability output

2019-08-01 Thread Wei Yang
postcopy-blocktime: off late-block-activate: off x-ignore-shared: off Signed-off-by: Wei Yang --- monitor/hmp-cmds.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 5ca3ebe942..29ce5b73e4 100644 --- a/monitor/hmp

Re: [Qemu-devel] [PATCH] migration: always initial RAMBlock.bmap to 1 for new migration

2019-08-01 Thread Wei Yang
>> >>It is after failure or cancel? > >Here we can treat cancel as a special fail. > Well, I am not sure this is officially documented. Would this lead to confusion? >Thanks. > -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH] migration: always initial ram_counters for a new migration

2019-08-01 Thread Wei Yang
function. So each time all of them. 2. In function ram_get_total_transferred_pages, do we missed multifd_bytes? -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH] migration: always initial RAMBlock.bmap to 1 for new migration

2019-07-31 Thread Wei Yang
+bitmap_set(block->bmap, 0, pages); > if (migrate_postcopy_ram()) { > block->unsentmap = bitmap_new(pages); > bitmap_set(block->unsentmap, 0, pages); >-- >2.17.2 (Apple Git-113) > Reviewed-by: Wei Yang -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH] migration: always initial ram_counters for a new migration

2019-07-31 Thread Wei Yang
tes(s); > } > > current_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); >diff --git a/migration/savevm.c b/migration/savevm.c >index 79ed44d475..480c511b19 100644 >--- a/migration/savevm.c >+++ b/migration/savevm.c >@@ -1424,6 +1424,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) > } > > migrate_init(ms); >+memset(_counters, 0, sizeof(ram_counters)); > ms->to_dst_file = f; > > qemu_mutex_unlock_iothread(); >-- >2.17.2 (Apple Git-113) > -- Wei Yang Help you, Help me

[Qemu-devel] [PATCH v2] docs/nvdimm: add example on persistent backend setup

2019-07-31 Thread Wei Yang
it directly and do more investigation on persistent backend setup with ndctl. Signed-off-by: Wei Yang Reviewed-by: Pankaj Gupta --- v2: rephrase the doc based on Stefan Hajnoczi's suggestion --- docs/nvdimm.txt | 31 +++ 1 file changed, 31 insertions(+) diff --git a/docs

Re: [Qemu-devel] [PATCH] docs/nvdimm: add example on persistent backend setup

2019-07-31 Thread Wei Yang
On Tue, Jul 30, 2019 at 07:03:45AM +0100, Stefan Hajnoczi wrote: >On Tue, Jul 30, 2019 at 1:52 AM Wei Yang wrote: >> >> Hi, Stefan >> >> Thanks for your comments :-) >> >> On Mon, Jul 29, 2019 at 02:58:59PM +0100, Stefan Hajnoczi wrote: >> >On Wed,

[Qemu-devel] [PATCH 2/2] migration/qemu-file: fix potential buf waste for extra buf_index adjustment

2019-07-31 Thread Wei Yang
it has flushed the file. Then the caller could check the return value to see whether it is necessary to adjust the buf_index any more. Signed-off-by: Wei Yang --- migration/qemu-file.c | 42 -- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git

[Qemu-devel] [PATCH 1/2] migration/qemu-file: remove check on writev_buffer in qemu_put_compression_data

2019-07-31 Thread Wei Yang
The check of writev_buffer is in qemu_fflush, which means it is not harmful if it is NULL. And removing it will make the code consistent since all other add_to_iovec() is called without the check. Signed-off-by: Wei Yang --- migration/qemu-file.c | 4 +--- 1 file changed, 1 insertion(+), 3

[Qemu-devel] [PATCH 0/2] migration/qemu-file: cleanup and refine qemu-file

2019-07-31 Thread Wei Yang
Two cleanup: Patch #1 make code consistent on calling add_to_iovec Patch #2 refine the code to handle the case when buf already flushed Wei Yang (2): migration/qemu-file: remove check on writev_buffer in qemu_put_compression_data migration/qemu-file: fix potential buf waste for extra

Re: [Qemu-devel] [PATCH v3 2/3] migration: add speed limit for multifd migration

2019-07-30 Thread Wei Yang
On Tue, Jul 30, 2019 at 01:33:35PM +0800, Ivan Ren wrote: >From: Ivan Ren > >Limit the speed of multifd migration through common speed limitation >qemu file. > >Signed-off-by: Ivan Ren Reviewed-by: Wei Yang >--- > migration/ram.c | 22 -- > 1

Re: [Qemu-devel] [PATCH] docs/nvdimm: add example on persistent backend setup

2019-07-29 Thread Wei Yang
Hi, Stefan Thanks for your comments :-) On Mon, Jul 29, 2019 at 02:58:59PM +0100, Stefan Hajnoczi wrote: >On Wed, Jul 24, 2019 at 03:03:07PM +0800, Wei Yang wrote: >> Persistent backend setup requires some knowledge about nvdimm and ndctl >> tool. Some users report they may str

Re: [Qemu-devel] [PATCH v2 0/3] migration: add speed limit for multifd migration

2019-07-29 Thread Wei Yang
ltifd migration > migration: update ram_counters for multifd sync packet > > migration/qemu-file.c | 5 + > migration/qemu-file.h | 1 + > migration/ram.c | 24 ++-- > 3 files changed, 20 insertions(+), 10 deletions(-) > >-- >2.17.2 (Apple Git-113) > -- Wei Yang Help you, Help me

Re: [Qemu-devel] [PATCH v2 3/3] migration: update ram_counters for multifd sync packet

2019-07-29 Thread Wei Yang
p->flags |= MULTIFD_FLAG_SYNC; > p->pending_job++; > qemu_file_update_transfer(rs->f, p->packet_len); >+ram_counters.multifd_bytes += p->packet_len; >+ram_counters.transferred += p->packet_len; > qemu_mutex_unlock(>mutex); >

Re: [Qemu-devel] [PATCH v2 1/3] migration: add qemu_file_update_transfer interface

2019-07-29 Thread Wei Yang
t; void qemu_file_skip(QEMUFile *f, int size); > void qemu_update_position(QEMUFile *f, size_t size); > void qemu_file_reset_rate_limit(QEMUFile *f); >+void qemu_file_update_transfer(QEMUFile *f, int64_t len); > void qemu_file_set_rate_limit(QEMUFile *f, int64_t new_rate); > int64_t qemu_file_

[Qemu-devel] [PATCH v2 0/2] refine memory_device_get_free_addr

2019-07-29 Thread Wei Yang
this change Wei Yang (2): memory-device: not necessary to use goto for the last check memory-device: break the loop if tmp exceed the hinted range hw/mem/memory-device.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.17.1

[Qemu-devel] [PATCH v2 1/2] memory-device: not necessary to use goto for the last check

2019-07-29 Thread Wei Yang
We are already at the last condition check. Signed-off-by: Wei Yang Reviewed-by: Igor Mammedov Reviewed-by: David Hildenbrand --- hw/mem/memory-device.c | 1 - 1 file changed, 1 deletion(-) diff --git a/hw/mem/memory-device.c b/hw/mem/memory-device.c index 5f2c408036..df3261b32a 100644

[Qemu-devel] [PATCH v2 2/2] memory-device: break the loop if tmp exceed the hinted range

2019-07-29 Thread Wei Yang
. Signed-off-by: Wei Yang --- hw/mem/memory-device.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/mem/memory-device.c b/hw/mem/memory-device.c index df3261b32a..df4e338b83 100644 --- a/hw/mem/memory-device.c +++ b/hw/mem/memory-device.c @@ -180,6 +180,8 @@ static uint64_t

Re: [Qemu-devel] [PATCH 3/3] memory-device: break the loop if tmp exceed the hinted range

2019-07-29 Thread Wei Yang
On Mon, Jul 29, 2019 at 10:30:56AM +0200, Igor Mammedov wrote: >On Mon, 29 Jul 2019 09:49:37 +0200 >David Hildenbrand wrote: > >> On 28.07.19 15:13, Wei Yang wrote: >> > The memory-device list built by memory_device_build_list is ordered by >> > its address, t

Re: [Qemu-devel] [PATCH 3/3] memory-device: break the loop if tmp exceed the hinted range

2019-07-29 Thread Wei Yang
On Mon, Jul 29, 2019 at 09:49:37AM +0200, David Hildenbrand wrote: >On 28.07.19 15:13, Wei Yang wrote: >> The memory-device list built by memory_device_build_list is ordered by >> its address, this means if the tmp range exceed the hinted range, all >> the following r

Re: [Qemu-devel] [PATCH 2/3] memory-device: break the loop if no hint is provided

2019-07-29 Thread Wei Yang
On Mon, Jul 29, 2019 at 09:45:24AM +0200, David Hildenbrand wrote: >On 28.07.19 15:13, Wei Yang wrote: >> When there is no hint, the first un-overlapped range is the proper one. >> Just break the loop instead of iterate the whole list. >> >> Signed-off-by: Wei Yan

Re: [Qemu-devel] [PATCH 2/3] memory-device: break the loop if no hint is provided

2019-07-29 Thread Wei Yang
On Mon, Jul 29, 2019 at 09:04:01AM +0200, Igor Mammedov wrote: >On Sun, 28 Jul 2019 21:13:03 +0800 >Wei Yang wrote: > >> When there is no hint, the first un-overlapped range is the proper one. >> Just break the loop instead of iterate the whole list. >could it change

<    1   2   3   4   5   6   7   8   >