[RFC PATCH v2 3/7] migration/snapshot: Move RAM_SAVE_FLAG_xxx defines to migration/ram.h

2021-05-12 Thread Andrey Gruzdev
Move RAM_SAVE_FLAG_xxx defines from migration/ram.c to migration/ram.h Signed-off-by: Andrey Gruzdev --- migration/ram.c | 16 migration/ram.h | 16 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index

[RFC PATCH v2 5/7] migration/snapshot: Implementation of qemu-snapshot save path

2021-05-12 Thread Andrey Gruzdev
Includes code to parse incoming migration stream, dispatch data to section handlers and deal with complications of open-coded migration format without introducing strong dependencies on QEMU migration code. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 34 +- qemu-snapshot-vm.c

[RFC PATCH v2 7/7] migration/snapshot: Implementation of qemu-snapshot load path in postcopy mode

2021-05-12 Thread Andrey Gruzdev
The commit enables asynchronous snapshot loading using standard postcopy migration mechanism on destination VM. The point of switchover to postcopy is trivially selected based on percentage of non-zero pages loaded in precopy. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 12

[RFC PATCH v2 1/7] migration/snapshot: Introduce qemu-snapshot tool

2021-05-12 Thread Andrey Gruzdev
Execution environment, command-line argument parsing, usage/version info etc. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 59 ++ meson.build | 2 + qemu-snapshot-vm.c | 57 ++ qemu-snapshot.c | 439 4

[RFC PATCH v2 6/7] migration/snapshot: Implementation of qemu-snapshot load path

2021-05-12 Thread Andrey Gruzdev
This part implements snapshot loading in precopy mode. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 24 +- qemu-snapshot-vm.c | 588 +++- qemu-snapshot.c | 47 +++- 3 files changed, 654 insertions(+), 5 deletions(-) diff --git

[RFC PATCH v2 4/7] migration/snapshot: Block layer AIO support in qemu-snapshot

2021-05-12 Thread Andrey Gruzdev
This commit enables asynchronous block layer I/O for qemu-snapshot tool. Implementation provides in-order request completion delivery to simplify migration code. Several file utility routines are introduced as well. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 30

[RFC PATCH v2 0/7] migration/snapshot: External snapshot utility

2021-05-12 Thread Andrey Gruzdev
irect=on,file.aio=native" And yes, asynchronous revert works well only with SSD, not with rotational disk.. Some performance stats: * SATA SSD drive with ~500/450 MB/s sequantial read/write and ~60K IOPS max. * 220 MB/s average save rate (depends on workload). * 440 MB/s average load rate i

[RFC PATCH v2 2/7] migration/snapshot: Introduce qemu_ftell2() routine

2021-05-12 Thread Andrey Gruzdev
In qemu-snapshot it is needed to retrieve current QEMUFile offset as a number of bytes read by qemu_get_byte()/qemu_get_buffer(). The existing qemu_ftell() routine would give read position as a number of bytes fetched from underlying IOChannel which is not the same. Signed-off-by: Andrey Gruzdev

[RFC PATCH v1 5/7] migration/snapshot: Implementation of qemu-snapshot save path

2021-05-12 Thread Andrey Gruzdev
Includes code to parse incoming migration stream, dispatch data to section handlers and deal with complications of open-coded migration format without introducing strong dependencies on QEMU migration code. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 34 +- qemu-snapshot-vm.c

[RFC PATCH v1 6/7] migration/snapshot: Implementation of qemu-snapshot load path

2021-05-12 Thread Andrey Gruzdev
This part implements snapshot loading in precopy mode. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 24 +- qemu-snapshot-vm.c | 588 +++- qemu-snapshot.c | 47 +++- 3 files changed, 654 insertions(+), 5 deletions(-) diff --git

[RFC PATCH v1 1/7] migration/snapshot: Introduce qemu-snapshot tool

2021-05-12 Thread Andrey Gruzdev
Execution environment, command-line argument parsing, usage/version info etc. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 59 ++ meson.build | 2 + qemu-snapshot-vm.c | 57 ++ qemu-snapshot.c | 439 4

[RFC PATCH v1 7/7] migration/snapshot: Implementation of qemu-snapshot load path in postcopy mode

2021-05-12 Thread Andrey Gruzdev
The commit enables asynchronous snapshot loading using standard postcopy migration mechanism on destination VM. The point of switchover to postcopy is trivially selected based on percentage of non-zero pages loaded in precopy. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 12

[RFC PATCH v1 0/7] migration/snapshot: External snapshot utility

2021-05-12 Thread Andrey Gruzdev
t; And yes, asynchronous revert works well only with SSD, not with rotational disk.. Some performance stats: * SATA SSD drive with ~500/450 MB/s sequantial read/write and ~60K IOPS max. * 220 MB/s average save rate (depends on workload). * 440 MB/s average load rate in precopy. * 260 MB/s average load rat

[RFC PATCH v1 2/7] migration/snapshot: Introduce qemu_ftell2() routine

2021-05-12 Thread Andrey Gruzdev
In qemu-snapshot it is needed to retrieve current QEMUFile offset as a number of bytes read by qemu_get_byte()/qemu_get_buffer(). The existing qemu_ftell() routine would give read position as a number of bytes fetched from underlying IOChannel which is not the same. Signed-off-by: Andrey Gruzdev

[RFC PATCH v1 4/7] migration/snapshot: Block layer AIO support in qemu-snapshot

2021-05-12 Thread Andrey Gruzdev
This commit enables asynchronous block layer I/O for qemu-snapshot tool. Implementation provides in-order request completion delivery to simplify migration code. Several file utility routines are introduced as well. Signed-off-by: Andrey Gruzdev --- include/qemu-snapshot.h | 30

[RFC PATCH v1 3/7] migration/snapshot: Move RAM_SAVE_FLAG_xxx defines to migration/ram.h

2021-05-12 Thread Andrey Gruzdev
Move RAM_SAVE_FLAG_xxx defines from migration/ram.c to migration/ram.h Signed-off-by: Andrey Gruzdev --- migration/ram.c | 16 migration/ram.h | 16 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index

Re: [RFC PATCH 0/9] migration/snap-tool: External snapshot utility

2021-04-16 Thread Andrey Gruzdev
On 16.04.2021 02:50, Peter Xu wrote: On Wed, Mar 17, 2021 at 07:32:13PM +0300, Andrey Gruzdev wrote: This series is a kind of PoC for asynchronous snapshot reverting. This is about external snapshots only and doesn't involve block devices. Thus, it's mainly intended to be used with the new

Re: [PULL 0/6] migration + virtiofsd queue

2021-04-08 Thread Andrey Gruzdev
On 08.04.2021 13:50, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: On 07.04.2021 19:50, Dr. David Alan Gilbert wrote: * Peter Maydell (peter.mayd...@linaro.org) wrote: On Wed, 7 Apr 2021 at 11:22, Dr. David Alan Gilbert (git) wrote: From: "Dr.

Re: [PULL 0/6] migration + virtiofsd queue

2021-04-08 Thread Andrey Gruzdev
6_64 The definition of ram_write_tracking_prepare() is inside an #if defined(__linux__), but the callsite is not, I think. OK, reproduced here. Let me see. Dave Seems that non-linux stub is missing, I'll respin. thanks -- PMM -- Andrey Gruzdev, Principal Engineer Virtuozzo GmbH +7-903-247-6397 virtuzzo.com

Re: [PATCH for-6.0 v1 0/4] migration: Fixes to the 'background-snapshot' code

2021-04-06 Thread Andrey Gruzdev
On 06.04.2021 19:53, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: Changes v0->v1: * Fixes to coding style and commit messages * Renamed 'bs' to 'block' in migration/ram.c background snapshot code This patch series contains: * Fix to the is

Re: [PATCH for-6.0 v1 1/4] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-04-06 Thread Andrey Gruzdev
On 06.04.2021 15:29, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require

[PATCH for-6.0 v1 0/4] migration: Fixes to the 'background-snapshot' code

2021-04-01 Thread Andrey Gruzdev
ice * Fix to the issue when discarded or never populated pages miss UFFD write protection and get into migration stream in dirty state * Renaming of 'bs' to commonly used 'block' in migration/ram.c background snapshot code Andrey Gruzdev (4): migration: Fix missing qemu_fflush() on buf

[PATCH for-6.0 v1 3/4] migration: Pre-fault memory before starting background snasphot

2021-04-01 Thread Andrey Gruzdev
block to be protected before making a userfault_fd wr-protect ioctl(). Fixes: 278e2f551a095b234de74dca9c214d5502a1f72c (migration: support UFFD write fault processing in ram_save_iterate()) Signed-off-by: Andrey Gruzdev Reported-by: David Hildenbrand Reviewed-by: David Hildenbrand --- migration

[PATCH for-6.0 v1 4/4] migration: Rename 'bs' to 'block' in background snapshot code

2021-04-01 Thread Andrey Gruzdev
Rename 'bs' to commonly used 'block' in migration/ram.c background snapshot code. Signed-off-by: Andrey Gruzdev Reported-by: David Hildenbrand --- migration/ram.c | 86 + 1 file changed, 44 insertions(+), 42 deletions(-) diff --git a/migration

[PATCH for-6.0 v1 1/4] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-04-01 Thread Andrey Gruzdev
ion of background snapshot thread) Signed-off-by: Andrey Gruzdev --- migration/migration.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index ca8b97baa5..00e13f9d58 100644 --- a/migration/migration.c +++ b/migrat

[PATCH for-6.0 v1 2/4] migration: Inhibit virtio-balloon for the duration of background snapshot

2021-04-01 Thread Andrey Gruzdev
The same thing as for incoming postcopy - we cannot deal with concurrent RAM discards when using background snapshot feature in outgoing migration. Fixes: 8518278a6af589ccc401f06e35f171b1e6fae800 (migration: implementation of background snapshot thread) Signed-off-by: Andrey Gruzdev Reported

Re: [PATCH for-6.0 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-31 Thread Andrey Gruzdev
On 31.03.2021 20:37, David Hildenbrand wrote: On 31.03.21 19:33, David Hildenbrand wrote: On 31.03.21 19:28, Andrey Gruzdev wrote: This commit solves the issue with userfault_fd WP feature that background snapshot is based on. For any never poluated or discarded memory page

Re: [PATCH for-6.0 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-31 Thread Andrey Gruzdev
On 31.03.2021 20:33, David Hildenbrand wrote: On 31.03.21 19:28, Andrey Gruzdev wrote: This commit solves the issue with userfault_fd WP feature that background snapshot is based on. For any never poluated or discarded memory page, the UFFDIO_WRITEPROTECT ioctl() would skip updating PTE

[PATCH for-6.0 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-31 Thread Andrey Gruzdev
Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require >200KB for device state and VMDESC. Signed-off-by: Andrey Gruzdev --- migration/migration.c | 8 +++-

[PATCH for-6.0 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-31 Thread Andrey Gruzdev
state Andrey Gruzdev (3): migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread migration: Inhibit virtio-balloon for the duration of background snapshot migration: Pre-fault memory before starting background snasphot hw/virtio/virtio-balloon.c | 8 +-- include

[PATCH for-6.0 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-31 Thread Andrey Gruzdev
block to be protected before making a userfault_fd wr-protect ioctl(). Signed-off-by: Andrey Gruzdev --- migration/migration.c | 6 ++ migration/ram.c | 48 +++ migration/ram.h | 1 + 3 files changed, 55 insertions(+) diff --git a/migration

[PATCH for-6.0 2/3] migration: Inhibit virtio-balloon for the duration of background snapshot

2021-03-31 Thread Andrey Gruzdev
The same thing as for incoming postcopy - we cannot deal with concurrent RAM discards when using background snapshot feature in outgoing migration. Signed-off-by: Andrey Gruzdev Reviewed-by: David Hildenbrand --- hw/virtio/virtio-balloon.c | 8 ++-- include/migration/misc.h | 2

Re: [PATCH v2 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-31 Thread Andrey Gruzdev
On 31.03.2021 19:02, Peter Xu wrote: On Wed, Mar 31, 2021 at 06:48:06PM +0300, Andrey Gruzdev wrote: Changes v1->v2: * Added comment over the overlooked qemu_flush() in bg_migration_thread Changes v0->v1: * Using qemu_real_host_page_size instead of TARGET_PAGE_SIZE for host pag

[PATCH v2 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-31 Thread Andrey Gruzdev
block to be protected before making a userfault_fd wr-protect ioctl(). Signed-off-by: Andrey Gruzdev --- migration/migration.c | 6 ++ migration/ram.c | 48 +++ migration/ram.h | 1 + 3 files changed, 55 insertions(+) diff --git a/migration

[PATCH v2 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-31 Thread Andrey Gruzdev
patch series contains: * Fix to the issue with occasionally truncated non-iterable device state * Solution to compatibility issues with virtio-balloon device * Fix to the issue when discarded or never populated pages miss UFFD write protection and get into migration stream in dirty state Andrey G

[PATCH v2 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-31 Thread Andrey Gruzdev
Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require >200KB for device state and VMDESC. Signed-off-by: Andrey Gruzdev --- migration/migration.c | 8 +++-

[PATCH v2 2/3] migration: Inhibit virtio-balloon for the duration of background snapshot

2021-03-31 Thread Andrey Gruzdev
The same thing as for incoming postcopy - we cannot deal with concurrent RAM discards when using background snapshot feature in outgoing migration. Signed-off-by: Andrey Gruzdev Reviewed-by: David Hildenbrand --- hw/virtio/virtio-balloon.c | 8 ++-- include/migration/misc.h | 2

Re: [RFC PATCH 0/9] migration/snap-tool: External snapshot utility

2021-03-29 Thread Andrey Gruzdev
Ping On 17.03.2021 19:32, Andrey Gruzdev wrote: This series is a kind of PoC for asynchronous snapshot reverting. This is about external snapshots only and doesn't involve block devices. Thus, it's mainly intended to be used with the new 'background-snapshot' migration capability

Re: [PATCH v1 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-25 Thread Andrey Gruzdev
On 24.03.2021 18:41, Peter Xu wrote: On Wed, Mar 24, 2021 at 11:09:27AM +0300, Andrey Gruzdev wrote: I'm also looking into introducing UFFD_FEATURE_WP_UNALLOCATED so as to wr-protect page holes too for a uffd-wp region when the feature bit is set. With that feature we should be able to avoid

Re: [PATCH v1 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-24 Thread Andrey Gruzdev
On 24.03.2021 01:21, Peter Xu wrote: On Fri, Mar 19, 2021 at 05:52:46PM +0300, Andrey Gruzdev wrote: Changes v0->v1: * Using qemu_real_host_page_size instead of TARGET_PAGE_SIZE for host page size in ram_block_populate_pages() * More elegant implementation of ram_block_populate_pa

Re: [PATCH v1 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-24 Thread Andrey Gruzdev
On 23.03.2021 21:35, Peter Xu wrote: On Tue, Mar 23, 2021 at 08:21:43PM +0300, Andrey Gruzdev wrote: For the long term I think we'd better have a helper: qemu_put_qio_channel_buffer(QEMUFile *file, QIOChannelBuffer *bioc) So as to hide this flush operation, which is tricky. We'll

Re: [PATCH v1 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-23 Thread Andrey Gruzdev
On 23.03.2021 17:54, Peter Xu wrote: On Tue, Mar 23, 2021 at 10:51:57AM +0300, Andrey Gruzdev wrote: On 22.03.2021 23:17, Peter Xu wrote: On Fri, Mar 19, 2021 at 05:52:47PM +0300, Andrey Gruzdev wrote: Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial

Re: [PATCH v1 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-23 Thread Andrey Gruzdev
On 22.03.2021 23:17, Peter Xu wrote: On Fri, Mar 19, 2021 at 05:52:47PM +0300, Andrey Gruzdev wrote: Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require >20

[PATCH v1 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-19 Thread Andrey Gruzdev
ate * Solution to compatibility issues with virtio-balloon device * Fix to the issue when discarded or never populated pages miss UFFD write protection and get into migration stream in dirty state Andrey Gruzdev (3): migration: Fix missing qemu_fflush() on buffer file in bg_migration_thr

[PATCH v1 2/3] migration: Inhibit virtio-balloon for the duration of background snapshot

2021-03-19 Thread Andrey Gruzdev
The same thing as for incoming postcopy - we cannot deal with concurrent RAM discards when using background snapshot feature in outgoing migration. Signed-off-by: Andrey Gruzdev Reviewed-by: David Hildenbrand --- hw/virtio/virtio-balloon.c | 8 ++-- include/migration/misc.h | 2

[PATCH v1 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-19 Thread Andrey Gruzdev
block to be protected before making a userfault_fd wr-protect ioctl(). Signed-off-by: Andrey Gruzdev --- migration/migration.c | 6 ++ migration/ram.c | 48 +++ migration/ram.h | 1 + 3 files changed, 55 insertions(+) diff --git a/migration

[PATCH v1 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-19 Thread Andrey Gruzdev
Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require >200KB for device state and VMDESC. Signed-off-by: Andrey Gruzdev --- migration/migration.c | 4 +++- 1 f

Re: [PATCH 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-19 Thread Andrey Gruzdev
On 19.03.2021 15:39, David Hildenbrand wrote: On 18.03.21 18:46, Andrey Gruzdev wrote: Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require >200KB for dev

Re: [PATCH 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-19 Thread Andrey Gruzdev
On 19.03.2021 14:27, David Hildenbrand wrote: On 19.03.21 12:05, Andrey Gruzdev wrote: On 19.03.2021 12:28, David Hildenbrand wrote: +/* + * ram_block_populate_pages: populate memory in the RAM block by reading + *   an integer from the beginning of each page. + * + * Since it's solely used

Re: [PATCH 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-19 Thread Andrey Gruzdev
latile char *)(ptr + offset) I wanted to do a "= *(ptr + offset)" here. Yep /* Don't optimize the read out. */ asm volatile ("" : "+r" (tmp)); So this is the only volatile thing that the compiler must guarantee to not optimize away. -- Andrey Gr

Re: [PATCH 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-19 Thread Andrey Gruzdev
are still stuck waiting for acks ... and now we're in soft-freeze. RamDiscardManager patches - do they also modify migration code? I mean which part is responsible of not migrating discarded ranges. -- Andrey Gruzdev, Principal Engineer Virtuozzo GmbH +7-903-247-6397 virtuzzo.com

Re: [PATCH 2/3] migration: Inhibit virtio-balloon for the duration of background snapshot

2021-03-19 Thread Andrey Gruzdev
On 18.03.2021 21:16, David Hildenbrand wrote: On 18.03.21 18:46, Andrey Gruzdev wrote: The same thing as for incoming postcopy - we cannot deal with concurrent RAM discards when using background snapshot feature in outgoing migration. Signed-off-by: Andrey Gruzdev ---   hw/virtio/virtio

[PATCH 2/3] migration: Inhibit virtio-balloon for the duration of background snapshot

2021-03-18 Thread Andrey Gruzdev
The same thing as for incoming postcopy - we cannot deal with concurrent RAM discards when using background snapshot feature in outgoing migration. Signed-off-by: Andrey Gruzdev --- hw/virtio/virtio-balloon.c | 8 ++-- include/migration/misc.h | 2 ++ migration/migration.c | 8

[PATCH 3/3] migration: Pre-fault memory before starting background snasphot

2021-03-18 Thread Andrey Gruzdev
block to be protected before making a userfault_fd wr-protect ioctl(). Signed-off-by: Andrey Gruzdev --- migration/migration.c | 6 + migration/ram.c | 51 +++ migration/ram.h | 1 + 3 files changed, 58 insertions(+) diff --git a/migration

[PATCH 0/3] migration: Fixes to the 'background-snapshot' code

2021-03-18 Thread Andrey Gruzdev
state Andrey Gruzdev (3): migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread migration: Inhibit virtio-balloon for the duration of background snapshot migration: Pre-fault memory before starting background snasphot hw/virtio/virtio-balloon.c | 8 -- include

[PATCH 1/3] migration: Fix missing qemu_fflush() on buffer file in bg_migration_thread

2021-03-18 Thread Andrey Gruzdev
Added missing qemu_fflush() on buffer file holding precopy device state. Increased initial QIOChannelBuffer allocation to 512KB to avoid reallocs. Typical configurations often require >200KB for device state and VMDESC. Signed-off-by: Andrey Gruzdev --- migration/migration.c | 4 +++- 1 f

[RFC PATCH 8/9] migration/snap-tool: Implementation of snapshot loading in precopy

2021-03-17 Thread Andrey Gruzdev
This part implements snapshot loading in precopy mode. Signed-off-by: Andrey Gruzdev --- include/qemu-snap.h | 24 ++ qemu-snap-handlers.c | 586 ++- qemu-snap.c | 44 +++- 3 files changed, 649 insertions(+), 5 deletions(-) diff --git

[RFC PATCH 7/9] migration/snap-tool: Complete implementation of snapshot saving

2021-03-17 Thread Andrey Gruzdev
Includes code to parse incoming migration stream, dispatch data to section handlers and deal with complications of open-coded migration format without introducing strong dependencies on QEMU migration code. Signed-off-by: Andrey Gruzdev --- include/qemu-snap.h | 42 +++ qemu-snap-handlers.c

[RFC PATCH 2/9] migration/snap-tool: Snapshot image create/open routines for qemu-snap tool

2021-03-17 Thread Andrey Gruzdev
-by: Andrey Gruzdev --- qemu-snap.c | 94 ++--- 1 file changed, 90 insertions(+), 4 deletions(-) diff --git a/qemu-snap.c b/qemu-snap.c index c7118927f7..c9f8d7166a 100644 --- a/qemu-snap.c +++ b/qemu-snap.c @@ -31,6 +31,16 @@ #include "migration

[RFC PATCH 9/9] migration/snap-tool: Implementation of snapshot loading in postcopy

2021-03-17 Thread Andrey Gruzdev
Implementation of asynchronous snapshot loading using standard postcopy migration mechanism on destination VM. The point of switchover to postcopy is trivially selected based on percentage of non-zero pages loaded in precopy. Signed-off-by: Andrey Gruzdev --- include/qemu-snap.h | 11 + qemu

[RFC PATCH 5/9] migration/snap-tool: Block layer AIO support and file utility routines

2021-03-17 Thread Andrey Gruzdev
ght Virtuozzo GmbH, 2021 + * + * Authors: + * Andrey Gruzdev + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/coroutine.h" +#include "qe

[RFC PATCH 3/9] migration/snap-tool: Preparations to run code in main loop context

2021-03-17 Thread Andrey Gruzdev
Major part of code is using QEMUFile and block layer routines, thus to take advantage from concurrent I/O operations we need to use coroutines and run in the the main loop context. Signed-off-by: Andrey Gruzdev --- include/qemu-snap.h | 3 +++ meson.build | 2 +- qemu-snap

[RFC PATCH 4/9] migration/snap-tool: Introduce qemu_ftell2() routine to qemu-file.c

2021-03-17 Thread Andrey Gruzdev
-by: Andrey Gruzdev --- migration/qemu-file.c | 6 ++ migration/qemu-file.h | 1 + 2 files changed, 7 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index d6e03dbc0e..66be5e6460 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -657,6 +657,12 @@ int64_t

[RFC PATCH 0/9] migration/snap-tool: External snapshot utility

2021-03-17 Thread Andrey Gruzdev
MB/s sequantial read/write and ~60K IOPS max. * 220 MB/s average save rate (depends on workload) * 440 MB/s average load rate in precopy * 260 MB/s average load rate in postcopy Andrey Gruzdev (9): migration/snap-tool: Introduce qemu-snap tool migration/snap-tool: Snapshot image create/open rou

[RFC PATCH 6/9] migration/snap-tool: Move RAM_SAVE_FLAG_xxx defines to migration/ram.h

2021-03-17 Thread Andrey Gruzdev
Move RAM_SAVE_FLAG_xxx defines from migration/ram.c to migration/ram.h Signed-off-by: Andrey Gruzdev --- migration/ram.c | 16 migration/ram.h | 16 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index

[RFC PATCH 1/9] migration/snap-tool: Introduce qemu-snap tool

2021-03-17 Thread Andrey Gruzdev
Initial commit with code to set up execution environment, parse command-line arguments, show usage/version info and so on. Signed-off-by: Andrey Gruzdev --- include/qemu-snap.h | 35 meson.build | 2 + qemu-snap.c | 414 3

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-24 Thread Andrey Gruzdev
On 24.02.2021 20:01, David Hildenbrand wrote: On 24.02.21 17:56, Andrey Gruzdev wrote: On 22.02.2021 21:11, David Hildenbrand wrote: On 22.02.21 18:54, Peter Xu wrote: On Mon, Feb 22, 2021 at 06:33:27PM +0100, David Hildenbrand wrote: On 22.02.21 18:29, Peter Xu wrote: On Sat, Feb 20, 2021

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-24 Thread Andrey Gruzdev
On 19.02.2021 23:50, Peter Xu wrote: Andrey, On Fri, Feb 19, 2021 at 09:57:37AM +0300, Andrey Gruzdev wrote: For the discards that happen before snapshot is started, I need to dig into Linux and QEMU virtio-baloon code more to get clear with it. Yes it's very tricky on how the error could

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-24 Thread Andrey Gruzdev
is not the only route. Thanks, Just to add: one of the good options is too keep track of virtio-baloon discarded pages and pre-fault them before migration starts. What do you think? Just pre-fault everything and inhibit the balloon. That should work. Yep. -- Andrey Gruzdev, Principal

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-24 Thread Andrey Gruzdev
iscarded" parts and later having to find bits to migrate. At least find_next_bit() can skip whole longs (8 bytes) and is fairly efficient. There is certainly room for improvement (the current guest free page hinting API is certainly a hack). -- Andrey Gruzdev, Principal Engineer Virtuozzo G

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-18 Thread Andrey Gruzdev
On 17.02.2021 02:35, Peter Xu wrote: Hi, Andrey, On Sat, Feb 13, 2021 at 12:34:07PM +0300, Andrey Gruzdev wrote: On 12.02.2021 19:11, Peter Xu wrote: On Fri, Feb 12, 2021 at 09:52:52AM +0100, David Hildenbrand wrote: On 12.02.21 04:06, Peter Xu wrote: On Thu, Feb 11, 2021 at 10:09:58PM

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-13 Thread Andrey Gruzdev
hat the saved state remains consistent and secure, off course if you agree with my arguments. -- Andrey Gruzdev, Principal Engineer Virtuozzo GmbH +7-903-247-6397 virtuzzo.com

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-11 Thread Andrey Gruzdev
On 11.02.2021 20:32, Peter Xu wrote: On Thu, Feb 11, 2021 at 07:19:47PM +0300, Andrey Gruzdev wrote: On 09.02.2021 22:06, David Hildenbrand wrote: Hi, just stumbled over this, quick question: I recently played with UFFD_WP and notices that write protection is only effective on pages/ranges

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-11 Thread Andrey Gruzdev
On 11.02.2021 20:18, Peter Xu wrote: On Thu, Feb 11, 2021 at 12:21:51PM +0300, Andrey Gruzdev wrote: On 09.02.2021 23:31, Peter Xu wrote: On Tue, Feb 09, 2021 at 03:09:28PM -0500, Peter Xu wrote: Hi, David, Andrey, On Tue, Feb 09, 2021 at 08:06:58PM +0100, David Hildenbrand wrote: Hi, just

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-11 Thread Andrey Gruzdev
ional UFFD events, much more MISSING events then WP-faults. And the main problem is that adding MISSING handler is impossible in current single-threaded snapshot code. We'll get an immediate deadlock on iterative page read. -- Andrey Gruzdev, Principal Engineer Virtuozzo GmbH +7-90

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-11 Thread Andrey Gruzdev
time we enter "check" Thanks, Hi David, Peter, A little unexpected behavior, from my point of view, for UFFD write-protection. So, that means that UFFD_WP protection/events works only for locked memory? I'm now looking at kernel implementation, to understan

Re: [PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-02-09 Thread Andrey Gruzdev
npopulated page, we should get first page fault on non-present page and populate it with protection bits from respective vma. For UFFD_WP vma's  page will be populated non-writable. So we'll get another page fault on present but read-only page and go to handle_userfault. -- Andrey Gruzdev, Principa

Re: [PATCH v14 0/5] UFFD write-tracking migration/snapshots

2021-02-08 Thread Andrey Gruzdev
On 04.02.2021 19:53, Dr. David Alan Gilbert wrote: * Dr. David Alan Gilbert (dgilb...@redhat.com) wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add

Re: [PATCH v14 0/5] UFFD write-tracking migration/snapshots

2021-02-04 Thread Andrey Gruzdev
On 04.02.2021 19:53, Dr. David Alan Gilbert wrote: * Dr. David Alan Gilbert (dgilb...@redhat.com) wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add

Re: [PATCH v14 0/5] UFFD write-tracking migration/snapshots

2021-02-04 Thread Andrey Gruzdev
On 04.02.2021 18:01, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make

[PATCH v14 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-29 Thread andrey . gruzdev--- via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 122 +++ 1 file changed, 122 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v14 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-29 Thread andrey . gruzdev--- via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 + include

[PATCH v14 4/5] migration: implementation of background snapshot thread

2021-01-29 Thread andrey . gruzdev--- via
and is Linux-specific. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 255 +- migration/migration.h | 3 + migration/savevm.c| 1 - migration/savevm.h| 2 + 4 files changed, 258

[PATCH v14 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-29 Thread andrey . gruzdev--- via
memory chunk that has hit the migration stream. That's valid for pages from linear page scan along with write fault pages. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- include/exec/memory.h | 7 + migration/ram.c| 324

[PATCH v14 1/5] migration: introduce 'background-snapshot' migration capability

2021-01-29 Thread andrey . gruzdev--- via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked

[PATCH v14 0/5] UFFD write-tracking migration/snapshots

2021-01-29 Thread andrey . gruzdev--- via
y fault detection (i.e. timestamp of *UFFD event fetch) is not representative enough for this task. *At the moment it looks somehow like premature optimization effort. * 8. Dropped some unnecessary/unused code. Andrey Gruzdev (5): migration: introduce 'background-snapshot' migration capability

Re: [PATCH v13 4/5] migration: implementation of background snapshot thread

2021-01-29 Thread Andrey Gruzdev
On 28.01.2021 21:29, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze

Re: [PATCH v11 4/5] migration: implementation of background snapshot thread

2021-01-21 Thread Andrey Gruzdev
On 21.01.2021 20:48, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: On 21.01.2021 19:11, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: On 21.01.2021 12:56, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz

Re: [PATCH v11 4/5] migration: implementation of background snapshot thread

2021-01-21 Thread Andrey Gruzdev
On 21.01.2021 19:11, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: On 21.01.2021 12:56, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz...@virtuozzo.com) wrote: On 19.01.2021 21:49, Dr. David Alan Gilbert wrote: * Andrey Gruzdev (andrey.gruz

Re: [PATCH v11 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-21 Thread Andrey Gruzdev
On 21.01.2021 18:37, Peter Xu wrote: On Thu, Jan 21, 2021 at 04:12:23PM +0300, Andrey Gruzdev wrote: +/* KRETPROBE for handle_userfault(). */ +int retprobe_handle_userfault(struct pt_regs *ctx) +{ +u64 pid = (u32) bpf_get_current_pid_tgid(); +u64 *addr_p; + +/* + * Here we just

[PATCH v13 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-21 Thread andrey . gruzdev--- via
memory chunk that has hit the migration stream. That's valid for pages from linear page scan along with write fault pages. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- include/exec/memory.h | 7 + migration/ram.c| 324

[PATCH v13 1/5] migration: introduce 'background-snapshot' migration capability

2021-01-21 Thread andrey . gruzdev--- via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked

[PATCH v13 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-21 Thread andrey . gruzdev--- via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 122 +++ 1 file changed, 122 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v13 4/5] migration: implementation of background snapshot thread

2021-01-21 Thread andrey . gruzdev--- via
and is Linux-specific. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 263 -- migration/migration.h | 3 + migration/savevm.c| 1 - migration/savevm.h| 2 + 4 files changed, 258

[PATCH v13 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-21 Thread andrey . gruzdev--- via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 + include

[PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-01-21 Thread andrey . gruzdev--- via
te-limiting' patch. The reason is that *that choosen criteria for high-latency fault detection (i.e. timestamp of *UFFD event fetch) is not representative enough for this task. *At the moment it looks somehow like premature optimization effort. * 8. Dropped some unnecessary/unused code. Andrey Gruzd

[PATCH v12 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-21 Thread andrey . gruzdev--- via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 122 +++ 1 file changed, 122 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v12 4/5] migration: implementation of background snapshot thread

2021-01-21 Thread andrey . gruzdev--- via
and is Linux-specific. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 263 -- migration/migration.h | 3 + migration/savevm.c| 1 - migration/savevm.h| 2 + 4 files changed, 258

[PATCH v12 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-21 Thread andrey . gruzdev--- via
memory chunk that has hit the migration stream. That's valid for pages from linear page scan along with write fault pages. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- include/exec/memory.h | 7 + migration/ram.c| 324

[PATCH v12 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-21 Thread andrey . gruzdev--- via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 + include

  1   2   3   >