[Qemu-devel] [PATCH 1/9 V5] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease futur

[Qemu-devel] [PATCH 7/9 V5] migration/ram: ensure write persistence on loading compressed pages to PMEM

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> When loading a compressed page to persistent memory, flush CPU cache after the data is decompressed. Combined with a call to pmem_drain() at the end of memory loading, we can guarantee those compressed pages are persistently loaded to PMEM. Sign

[Qemu-devel] [PATCH 6/9 V5] migration/ram: ensure write persistence on loading normal pages to PMEM

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> When loading a normal page to persistent memory, load its data by libpmem function pmem_memcpy_nodrain() instead of memcpy(). Combined with a call to pmem_drain() at the end of memory loading, we can guarantee all those normal pages are persistenly

[Qemu-devel] [PATCH 2/9 V5] hostmem-file: add the 'pmem' option

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean

[Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failu

[Qemu-devel] [PATCH 4/9 V5] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power f

[Qemu-devel] [PATCH 5/9 V5] migration/ram: ensure write persistence on loading zero pages to PMEM

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> When loading a zero page, check whether it will be loaded to persistent memory If yes, load it by libpmem function pmem_memset_nodrain(). Combined with a call to pmem_drain() at the end of RAM loading, we can guarantee all those zero pages are persis

[Qemu-devel] [PATCH 8/9 V5] migration/ram: ensure write persistence on loading xbzrle pages to PMEM

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> When loading a xbzrle encoded page to persistent memory, load the data via libpmem function pmem_memcpy_nodrain() instead of memcpy(). Combined with a call to pmem_drain() at the end of memory loading, we can guarantee those xbzrle encoded

[Qemu-devel] [PATCH 3/9 V5] configure: add libpmem support

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper oper

[Qemu-devel] [PATCH 9/9 V5] migration/ram: Add check and info message to nvdimm post copy.

2018-05-09 Thread junyan . he
From: Junyan He <junyan...@intel.com> The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He <junyan...@intel.com> --- migration/ram.c | 9 + 1 file changed, 9 inserti

Re: [Qemu-devel] [PATCH 1/9 V5] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-05-11 Thread Junyan He
@gmail.com, m...@redhat.com, qemu-devel@nongnu.org, >dgilb...@redhat.com, quint...@redhat.com, "Junyan He" <junyan...@intel.com>, >stefa...@redhat.com, imamm...@redhat.com, pbonz...@redhat.com, >r...@twiddle.net, ehabk...@redhat.com >Subject: Re: [Qemu-devel] [PATCH 1/9 V5] mem

Re: [Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-05-20 Thread Junyan He
edhat.com, ehabk...@redhat.com, quint...@redhat.com, "Junyan He" <junyan...@intel.com>, stefa...@redhat.com, pbonz...@redhat.com, imamm...@redhat.com, r...@twiddle.net Subject: [Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee persistence of QEMU writes to persistent memory From

[Qemu-devel] [PATCH 0/7 V7] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-06-12 Thread junyan . he
From: Junyan He QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 3/7 V7] hostmem-file: add the 'pmem' option

2018-06-12 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH 2/7 V7] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-06-12 Thread junyan . he
From: Junyan He As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease future flag additions

[Qemu-devel] [PATCH 4/7 V7] configure: add libpmem support

2018-06-12 Thread junyan . he
From: Junyan He Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper operations to ensure

[Qemu-devel] [PATCH 1/7 V7] memory, exec: Expose all memory block related flags.

2018-06-12 Thread junyan . he
From: Junyan He We need to use these flags in other files rather than just in exec.c, For example, RAM_SHARED should be used when create a ram block from file. We expose them the exec/memory.h Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi --- exec.c| 20

[Qemu-devel] [PATCH 6/7 V7] migration/ram: Add check and info message to nvdimm post copy.

2018-06-12 Thread junyan . he
From: Junyan He The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi --- migration/ram.c | 9 + 1 file changed, 9 insertions(+) diff --git

[Qemu-devel] [PATCH 7/7 V7] migration/ram: ensure write persistence on loading all data to PMEM.

2018-06-12 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi

[Qemu-devel] [PATCH 5/7 V7] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-06-12 Thread junyan . he
From: Junyan He Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power failure may result in the loss

Re: [Qemu-devel] [PATCH V6 0/7] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-06-12 Thread Junyan He
tate pmem=force for testing purposes. I think we can print a warning message to give user a hint, it can really help user when they mis-configure. From: Qemu-devel on behalf of Igor Mammedov Sent: Tuesday, June 12, 2018 2:55:46 PM To: Junyan He Cc: Haozh

Re: [Qemu-devel] [PATCH V6 0/7] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-06-12 Thread Junyan He
______ From: Qemu-devel on behalf of Junyan He Sent: Tuesday, June 12, 2018 3:27:38 PM To: Igor Mammedov Cc: Haozhong Zhang; xiaoguangrong.e...@gmail.com; crosthwaite.pe...@gmail.com; m...@redhat.com; qemu-devel@nongnu.org; dgilb...@redhat.com; quint...@redhat.com; Junyan He; s

Re: [Qemu-devel] [Qemu-block] Some question about savem/qcow2 incremental snapshot

2018-06-08 Thread Junyan He
AM To: Junyan He Cc: Kevin Wolf; qemu block; qemu-devel@nongnu.org; Stefan Hajnoczi; Max Reitz Subject: Re: [Qemu-devel] [Qemu-block] Some question about savem/qcow2 incremental snapshot Hi Junyan, AFAICU you are trying to utilize qcow2 capabilities to do incremental snapshot. As I understand

[Qemu-devel] [PATCH V6 RESEND 7/7] migration/ram: ensure write persistence on loading all data to PMEM.

2018-06-08 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He --- include/qemu/pmem.h | 1

Re: [Qemu-devel] [PATCH V6 RESEND 7/7] migration/ram: ensure write persistence on loading all data to PMEM.

2018-06-08 Thread Junyan He
...@gmail.com; m...@redhat.com; dgilb...@redhat.com; ehabk...@redhat.com; quint...@redhat.com; Junyan He; stefa...@redhat.com; pbonz...@redhat.com; imamm...@redhat.com; r...@twiddle.net Subject: [Qemu-devel] [PATCH V6 RESEND 7/7] migration/ram: ensure write persistence on loading all data to PMEM. From

[Qemu-devel] [PATCH 3/7 V7 RESEND] hostmem-file: add the 'pmem' option

2018-06-18 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

Re: [Qemu-devel] [PATCH V6 0/7] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-06-12 Thread Junyan He
...@gmail.com; crosthwaite.pe...@gmail.com; m...@redhat.com; qemu-devel@nongnu.org; dgilb...@redhat.com; quint...@redhat.com; Junyan He; stefa...@redhat.com; pbonz...@redhat.com; r...@twiddle.net; ehabk...@redhat.com Subject: Re: [Qemu-devel] [PATCH V6 0/7] nvdimm: guarantee persistence of QEMU

[Qemu-devel] [PATCH V6 RESEND 7/7] migration/ram: ensure write persistence on loading all data to PMEM.

2018-06-07 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He --- include/qemu/pmem.h | 1

Re: [Qemu-devel] [PATCH 2/9 V5] hostmem-file: add the 'pmem' option

2018-05-31 Thread Junyan He
To: junyan...@gmx.com Cc: qemu-devel@nongnu.org; Haozhong Zhang; xiaoguangrong.e...@gmail.com; crosthwaite.pe...@gmail.com; m...@redhat.com; dgilb...@redhat.com; ehabk...@redhat.com; quint...@redhat.com; Junyan He; stefa...@redhat.com; pbonz...@redhat.com; imamm...@redhat.com; r...@twiddle.net

[Qemu-devel] [PATCH V6 5/7] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-06-01 Thread junyan . he
From: Junyan He Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH V6 2/7] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-06-01 Thread junyan . he
From: Junyan He As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease future flag additions

[Qemu-devel] [PATCH V6 4/7] configure: add libpmem support

2018-06-01 Thread junyan . he
From: Junyan He Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper operations to ensure

[Qemu-devel] [PATCH V6 3/7] hostmem-file: add the 'pmem' option

2018-06-01 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH V6 6/7] migration/ram: Add check and info message to nvdimm post copy.

2018-06-01 Thread junyan . he
From: Junyan He The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He --- migration/ram.c | 9 + 1 file changed, 9 insertions(+) diff --git a/migration/ram.c b/migration/ram.c

[Qemu-devel] [PATCH V6 1/7] memory, exec: Expose all memory block related flags.

2018-06-01 Thread junyan . he
From: Junyan He We need to use these flags in other files rather than just in exec.c, For example, RAM_SHARED should be used when create a ram block from file. We expose them the exec/memory.h Signed-off-by: Junyan He --- exec.c| 17 - include/exec/memory.h

[Qemu-devel] [PATCH V6 0/7] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-06-01 Thread junyan . he
From: Junyan He QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH V6 7/7] migration/ram: ensure write persistence on loading all data to PMEM.

2018-06-01 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He --- include/qemu/pmem.h | 1

Re: [Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-05-27 Thread Junyan He
that this is not caused by my patch set? And I can not duplicate in local. Pings, thanks From: Qemu-devel <qemu-devel-bounces+junyan.he=gmx@nongnu.org> on behalf of Junyan He <junyan...@gmx.com> Sent: Monday, May 21, 2018 3:19:48 AM To: junyan..

Re: [Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-05-31 Thread Junyan He
may be faster and I think the benchmark shows that they are close. The worst case is that all the pmem-like memory will be flushed after completing migration, I am not sure whether there are some unused pmem for new guest will also be flushed. > Dave From:

Re: [Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-05-31 Thread Junyan He
Zhang; xiaoguangrong.e...@gmail.com; crosthwaite.pe...@gmail.com; m...@redhat.com; dgilb...@redhat.com; ehabk...@redhat.com; quint...@redhat.com; Junyan He; stefa...@redhat.com; pbonz...@redhat.com; imamm...@redhat.com; r...@twiddle.net Subject: Re: [Qemu-devel] [PATCH V5 0/9] nvdimm: guarantee pe

[Qemu-devel] [PATCH 3/7 V7] hostmem-file: add the 'pmem' option

2018-07-03 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> We want to support incremental snapshot saving, this needs the file system support dependency saving. Later snapshots may ref the dependent snapshot's content, and most time should be cluster aligned. Add a query function to check whether the file

[Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> The nvdimm size is huge, sometimes it is more than 256G or even more. This is a huge burden for snapshot saving. One snapshot point with nvdimm may occupy more than 50G disk space even with compression enabled. We need to introduce dependent snapshot

[Qemu-devel] [PATCH 06/10] RFC: Add save dependency functions to qemu_file

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> When we save snapshot, we need qemu_file to support save dependency operations. It should call brv_driver's save dependency functions to implement these operations. Signed-off-by: Junyan He <junyan...@intel.com> --- migration/qemu

[Qemu-devel] [PATCH 09/10] RFC: Add nvdimm snapshot saving to migration.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> The nvdimm size is huge, sometimes is more than 256G or even more. This is a huge burden for snapshot saving. One snapshot point with nvdimm may occupy more than 50G disk space even with compression enabled. We need to introduce dependent snapshot

[Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> For qcow2 format, we can increase the cluster's reference count of dependent snapshot content and link the offset to the L2 table of the new snapshot point. This way can avoid obvious snapshot's dependent relationship, so when we delete some snapshot

[Qemu-devel] [PATCH 04/10] RFC: Set memory_region_set_log available for more client.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to collect dirty log for nvdimm kind memory, need to enable memory_region_set_log for more clients rather than just VGA. Signed-off-by: Junyan He <junyan...@intel.com> --- memory.c | 11 ++- 1 file changed, 6 insertions(+),

[Qemu-devel] [PATCH 07/10] RFC: Add get_current_snapshot_info to get the snapshot state.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to know the snapshot saving information when we do dependent snapshot saving, e.g the name of previous snapshot. Add this global function to query the snapshot status is usable. Signed-off-by: Junyan He <junyan...@intel.com> --- inclu

[Qemu-devel] [PATCH 10/10] RFC: Enable nvdimm snapshot functions.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> In snapshot saving, all nvdimm kind memory will be saved in different way and we exclude all nvdimm kind memory region in ram.c Signed-off-by: Junyan He <junyan...@intel.com> --- migration/ram.c | 17 + vl.c|

[Qemu-devel] [PATCH 03/10] RFC: Implement save and support snapshot dependency in block driver layer.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> Signed-off-by: Junyan He <junyan...@intel.com> --- block/snapshot.c | 45 + include/block/snapshot.h | 7 +++ 2 files changed, 52 insertions(+) diff --git a/block/snapshot.c b/block/snap

[Qemu-devel] [PATCH 05/10] RFC: Add memory region snapshot bitmap get function.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to get the bitmap content of the snapshot when enable dirty log trace for nvdimm. Signed-off-by: Junyan He <junyan...@intel.com> --- exec.c | 7 +++ include/exec/memory.h | 9 + include/exec/ram_addr.h | 2

[Qemu-devel] [PATCH 08/10] RFC: Add a section_id parameter to save_live_iterate call.

2018-03-15 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to know the section_id when we do snapshot saving. Add a parameter to save_live_iterate function call. Signed-off-by: Junyan He <junyan...@intel.com> --- hw/ppc/spapr.c | 2 +- hw/s390x/s390-stattrib.c | 2 +- inclu

[Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> The nvdimm size is huge, sometimes it is more than 256G or even more. This is a huge burden for snapshot saving. One snapshot point with nvdimm may occupy more than 50G disk space even with compression enabled. We need to introduce dependent snapshot

[Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> We want to support incremental snapshot saving, this needs the file system support dependency saving. Later snapshots may ref the dependent snapshot's content, and most time should be cluster aligned. Add a query function to check whether the file

[Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> For qcow2 format, we can increase the cluster's reference count of dependent snapshot content and link the offset to the L2 table of the new snapshot point. This way can avoid obvious snapshot's dependent relationship, so when we delete some snapshot

[Qemu-devel] [PATCH 06/10] RFC: Add save dependency functions to qemu_file

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> When we save snapshot, we need qemu_file to support save dependency operations. It should call brv_driver's save dependency functions to implement these operations. Signed-off-by: Junyan He <junyan...@intel.com> --- migration/qemu

[Qemu-devel] [PATCH 03/10] RFC: Implement save and support snapshot dependency in block driver layer.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> Signed-off-by: Junyan He <junyan...@intel.com> --- block/snapshot.c | 45 + include/block/snapshot.h | 7 +++ 2 files changed, 52 insertions(+) diff --git a/block/snapshot.c b/block/snap

[Qemu-devel] [PATCH 10/10] RFC: Enable nvdimm snapshot functions.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> In snapshot saving, all nvdimm kind memory will be saved in different way and we exclude all nvdimm kind memory region in ram.c Signed-off-by: Junyan He <junyan...@intel.com> --- migration/ram.c | 17 + vl.c|

[Qemu-devel] [PATCH 04/10] RFC: Set memory_region_set_log available for more client.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to collect dirty log for nvdimm kind memory, need to enable memory_region_set_log for more clients rather than just VGA. Signed-off-by: Junyan He <junyan...@intel.com> --- memory.c | 11 ++- 1 file changed, 6 insertions(+),

[Qemu-devel] [PATCH 05/10] RFC: Add memory region snapshot bitmap get function.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to get the bitmap content of the snapshot when enable dirty log trace for nvdimm. Signed-off-by: Junyan He <junyan...@intel.com> --- exec.c | 7 +++ include/exec/memory.h | 9 + include/exec/ram_addr.h | 2

[Qemu-devel] [PATCH 08/10] RFC: Add a section_id parameter to save_live_iterate call.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to know the section_id when we do snapshot saving. Add a parameter to save_live_iterate function call. Signed-off-by: Junyan He <junyan...@intel.com> --- hw/ppc/spapr.c | 2 +- hw/s390x/s390-stattrib.c | 2 +- inclu

[Qemu-devel] [PATCH 09/10] RFC: Add nvdimm snapshot saving to migration.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> The nvdimm size is huge, sometimes is more than 256G or even more. This is a huge burden for snapshot saving. One snapshot point with nvdimm may occupy more than 50G disk space even with compression enabled. We need to introduce dependent snapshot

[Qemu-devel] [PATCH 07/10] RFC: Add get_current_snapshot_info to get the snapshot state.

2018-03-13 Thread junyan . he
From: Junyan He <junyan...@intel.com> We need to know the snapshot saving information when we do dependent snapshot saving, e.g the name of previous snapshot. Add this global function to query the snapshot status is usable. Signed-off-by: Junyan He <junyan...@intel.com> --- inclu

[Qemu-devel] [PATCH 3/7 V9] configure: add libpmem support

2018-07-10 Thread junyan . he
From: Junyan He Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper operations to ensure

[Qemu-devel] [PATCH 0/7 V9] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-07-10 Thread junyan . he
From: Junyan He QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 2/7 V9] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-07-10 Thread junyan . he
From: Junyan He As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease future flag additions. The existing

[Qemu-devel] [PATCH 1/7 V9] memory, exec: Expose all memory block related flags.

2018-07-10 Thread junyan . he
From: Junyan He We need to use these flags in other files rather than just in exec.c, For example, RAM_SHARED should be used when create a ram block from file. We expose them the exec/memory.h Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- exec.c

[Qemu-devel] [PATCH 6/7 V9] migration/ram: Add check and info message to nvdimm post copy.

2018-07-10 Thread junyan . he
From: Junyan He The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- migration/ram.c | 9 + 1 file changed, 9

[Qemu-devel] [PATCH 5/7 V9] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-07-10 Thread junyan . he
From: Junyan He Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 7/7 V9] migration/ram: ensure write persistence on loading all data to PMEM.

2018-07-10 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi

[Qemu-devel] [PATCH 4/7 V9] hostmem-file: add the 'pmem' option

2018-07-10 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH 0/7 V8] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-07-10 Thread junyan . he
From: Junyan He QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 2/7 V8] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-07-10 Thread junyan . he
From: Junyan He As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease future flag additions. The existing

[Qemu-devel] [PATCH 6/7 V8] migration/ram: Add check and info message to nvdimm post copy.

2018-07-10 Thread junyan . he
From: Junyan He The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- migration/ram.c | 9 + 1 file changed, 9

[Qemu-devel] [PATCH 7/7 V8] migration/ram: ensure write persistence on loading all data to PMEM.

2018-07-10 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi

[Qemu-devel] [PATCH 5/7 V8] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-07-10 Thread junyan . he
From: Junyan He Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 1/7 V8] memory, exec: Expose all memory block related flags.

2018-07-10 Thread junyan . he
From: Junyan He We need to use these flags in other files rather than just in exec.c, For example, RAM_SHARED should be used when create a ram block from file. We expose them the exec/memory.h Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- exec.c

[Qemu-devel] [PATCH 3/7 V8] hostmem-file: add the 'pmem' option

2018-07-10 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH 4/7 V8] configure: add libpmem support

2018-07-10 Thread junyan . he
From: Junyan He Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper operations to ensure

[Qemu-devel] [PATCH 3/7 V11] configure: add libpmem support

2018-07-18 Thread junyan . he
From: Junyan He Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper operations to ensure

[Qemu-devel] [PATCH 0/7 V11] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-07-18 Thread junyan . he
From: Junyan He QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 2/7 V11] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-07-18 Thread junyan . he
From: Junyan He As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease future flag additions. The existing

[Qemu-devel] [PATCH 5/7 V11] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-07-18 Thread junyan . he
From: Junyan He Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 7/7 V11] migration/ram: ensure write persistence on loading all data to PMEM.

2018-07-18 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi

[Qemu-devel] [PATCH 6/7 V11] migration/ram: Add check and info message to nvdimm post copy.

2018-07-18 Thread junyan . he
From: Junyan He The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- migration/ram.c | 9 + 1 file changed, 9

[Qemu-devel] [PATCH 1/7 V11] memory, exec: Expose all memory block related flags.

2018-07-18 Thread junyan . he
From: Junyan He We need to use these flags in other files rather than just in exec.c, For example, RAM_SHARED should be used when create a ram block from file. We expose them the exec/memory.h Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov Reviewed

[Qemu-devel] [PATCH 4/7 V11] hostmem-file: add the 'pmem' option

2018-07-18 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH 4/7 V10] hostmem-file: add the 'pmem' option

2018-07-17 Thread junyan . he
From: Junyan He When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it needs to know whether the backend storage is a real persistent memory, in order to decide whether special operations should be performed to ensure the data persistence. This boolean option 'pmem' allows users

[Qemu-devel] [PATCH 2/7 V10] memory, exec: switch file ram allocation functions to 'flags' parameters

2018-07-17 Thread junyan . he
From: Junyan He As more flag parameters besides the existing 'share' are going to be added to following functions memory_region_init_ram_from_file qemu_ram_alloc_from_fd qemu_ram_alloc_from_file let's switch them to use the 'flags' parameters so as to ease future flag additions. The existing

[Qemu-devel] [PATCH 1/7 V10] memory, exec: Expose all memory block related flags.

2018-07-17 Thread junyan . he
From: Junyan He We need to use these flags in other files rather than just in exec.c, For example, RAM_SHARED should be used when create a ram block from file. We expose them the exec/memory.h Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- exec.c

[Qemu-devel] [PATCH 7/7 V10] migration/ram: ensure write persistence on loading all data to PMEM.

2018-07-17 Thread junyan . he
From: Junyan He Because we need to make sure the pmem kind memory data is synced after migration, we choose to call pmem_persist() when the migration finish. This will make sure the data of pmem is safe and will not lose if power is off. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi

[Qemu-devel] [PATCH 6/7 V10] migration/ram: Add check and info message to nvdimm post copy.

2018-07-17 Thread junyan . he
From: Junyan He The nvdimm kind memory does not support post copy now. We disable post copy if we have nvdimm memory and print some log hint to user. Signed-off-by: Junyan He Reviewed-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov --- migration/ram.c | 9 + 1 file changed, 9

[Qemu-devel] [PATCH 3/7 V10] configure: add libpmem support

2018-07-17 Thread junyan . he
From: Junyan He Add a pair of configure options --{enable,disable}-libpmem to control whether QEMU is compiled with PMDK libpmem [1]. QEMU may write to the host persistent memory (e.g. in vNVDIMM label emulation and live migration), so it must take the proper operations to ensure

[Qemu-devel] [PATCH 0/7 V10] nvdimm: guarantee persistence of QEMU writes to persistent memory

2018-07-17 Thread junyan . he
From: Junyan He QEMU writes to vNVDIMM backends in the vNVDIMM label emulation and live migration. If the backend is on the persistent memory, QEMU needs to take proper operations to ensure its writes persistent on the persistent memory. Otherwise, a host power failure may result in the loss

[Qemu-devel] [PATCH 5/7 V10] mem/nvdimm: ensure write persistence to PMEM in label emulation

2018-07-17 Thread junyan . he
From: Junyan He Guest writes to vNVDIMM labels are intercepted and performed on the backend by QEMU. When the backend is a real persistent memort, QEMU needs to take proper operations to ensure its write persistence on the persistent memory. Otherwise, a host power failure may result in the loss