[PATCH v14 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-29 Thread andrey . gruzdev--- via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 122 +++ 1 file changed, 122 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v14 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-29 Thread andrey . gruzdev--- via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v14 4/5] migration: implementation of background snapshot thread

2021-01-29 Thread andrey . gruzdev--- via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v14 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-29 Thread andrey . gruzdev--- via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v14 1/5] migration: introduce 'background-snapshot' migration capability

2021-01-29 Thread andrey . gruzdev--- via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked-by:

[PATCH v14 0/5] UFFD write-tracking migration/snapshots

2021-01-29 Thread andrey . gruzdev--- via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v13 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-21 Thread andrey . gruzdev--- via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v13 1/5] migration: introduce 'background-snapshot' migration capability

2021-01-21 Thread andrey . gruzdev--- via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked-by:

[PATCH v13 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-21 Thread andrey . gruzdev--- via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 122 +++ 1 file changed, 122 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v13 4/5] migration: implementation of background snapshot thread

2021-01-21 Thread andrey . gruzdev--- via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v13 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-21 Thread andrey . gruzdev--- via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v13 0/5] UFFD write-tracking migration/snapshots

2021-01-21 Thread andrey . gruzdev--- via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v12 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-21 Thread andrey . gruzdev--- via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 122 +++ 1 file changed, 122 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v12 4/5] migration: implementation of background snapshot thread

2021-01-21 Thread andrey . gruzdev--- via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v12 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-21 Thread andrey . gruzdev--- via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v12 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-21 Thread andrey . gruzdev--- via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v12 1/5] migration: introduce 'background-snapshot' migration capability

2021-01-21 Thread andrey . gruzdev--- via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked-by:

[PATCH v12 0/5] UFFD write-tracking migration/snapshots

2021-01-21 Thread andrey . gruzdev--- via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v11 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2021-01-06 Thread Andrey Gruzdev via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev Acked-by: Peter Xu --- scripts/userfaultfd-wrlat.py | 148 +++ 1 file changed, 148 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff

[PATCH v11 4/5] migration: implementation of background snapshot thread

2021-01-06 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v11 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2021-01-06 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v11 1/5] migration: introduce 'background-snapshot' migration capability

2021-01-06 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked-by:

[PATCH v11 2/5] migration: introduce UFFD-WP low-level interface helpers

2021-01-06 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v11 0/5] UFFD write-tracking migration/snapshots

2021-01-06 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v10 1/5] migration: introduce 'background-snapshot' migration capability

2020-12-17 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu Acked-by:

[PATCH v10 0/5] UFFD write-tracking migration/snapshots

2020-12-17 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v10 4/5] migration: implementation of background snapshot thread

2020-12-17 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v10 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2020-12-17 Thread Andrey Gruzdev via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev --- scripts/userfaultfd-wrlat.py | 148 +++ 1 file changed, 148 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff --git

[PATCH v10 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-17 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v10 2/5] migration: introduce UFFD-WP low-level interface helpers

2020-12-17 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v9 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-17 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v9 4/5] migration: implementation of background snapshot thread

2020-12-17 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v9 2/5] migration: introduce UFFD-WP low-level interface helpers

2020-12-17 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v9 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2020-12-17 Thread Andrey Gruzdev via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev --- scripts/userfaultfd-wrlat.py | 148 +++ 1 file changed, 148 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff --git

[PATCH v9 1/5] migration: introduce 'background-snapshot' migration capability

2020-12-17 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu ---

[PATCH v9 0/5] UFFD write-tracking migration/snapshots

2020-12-17 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v8 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2020-12-17 Thread Andrey Gruzdev via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev --- scripts/userfaultfd-wrlat.py | 148 +++ 1 file changed, 148 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff --git

[PATCH v8 4/5] migration: implementation of background snapshot thread

2020-12-17 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v8 1/5] migration: introduce 'background-snapshot' migration capability

2020-12-17 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu ---

[PATCH v8 2/5] migration: introduce UFFD-WP low-level interface helpers

2020-12-17 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v8 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-17 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v7 0/5] migration: UFFD write-tracking migration/snapshots

2020-12-17 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v7 4/5] migration: implementation of background snapshot thread

2020-12-17 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v7 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2020-12-17 Thread Andrey Gruzdev via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev --- scripts/userfaultfd-wrlat.py | 148 +++ 1 file changed, 148 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff --git

[PATCH v7 2/5] migration: introduce UFFD-WP low-level interface helpers

2020-12-17 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v7 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-17 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v7 0/5] migration: UFFD write-tracking migration/snapshots

2020-12-17 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v7 1/5] migration: introduce 'background-snapshot' migration capability

2020-12-17 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu ---

[PATCH v7 5/5] migration: introduce 'userfaultfd-wrlat.py' script

2020-12-17 Thread Andrey Gruzdev via
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution. Signed-off-by: Andrey Gruzdev --- scripts/userfaultfd-wrlat.py | 148 +++ 1 file changed, 148 insertions(+) create mode 100755 scripts/userfaultfd-wrlat.py diff --git

[PATCH v7 4/5] migration: implementation of background snapshot thread

2020-12-17 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v7 1/5] migration: introduce 'background-snapshot' migration capability

2020-12-17 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu ---

[PATCH v7 3/5] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-17 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v7 2/5] migration: introduce UFFD-WP low-level interface helpers

2020-12-17 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v7 0/5] migration: UFFD write-tracking migration/snapshots

2020-12-17 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v6 1/4] migration: introduce 'background-snapshot' migration capability

2020-12-09 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu ---

[PATCH v6 2/4] migration: introduce UFFD-WP low-level interface helpers

2020-12-09 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev Reviewed-by: Peter Xu --- include/exec/memory.h | 1 +

[PATCH v6 4/4] migration: implementation of background snapshot thread

2020-12-09 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v6 3/4] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-09 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v6 0/4] migration: UFFD write-tracking migration/snapshots

2020-12-09 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v5 2/4] migration: introduce UFFD-WP low-level interface helpers

2020-12-04 Thread Andrey Gruzdev via
Glue code to the userfaultfd kernel implementation. Querying feature support, createing file descriptor, feature control, memory region registration, IOCTLs on registered registered regions. Signed-off-by: Andrey Gruzdev --- include/exec/memory.h | 1 + include/qemu/userfaultfd.h | 35

[PATCH v5 0/4] migration: UFFD write-tracking migration/snapshots

2020-12-04 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v5 1/4] migration: introduce 'background-snapshot' migration capability

2020-12-04 Thread Andrey Gruzdev via
Add new capability to 'qapi/migration.json' schema. Update migrate_caps_check() to validate enabled capability set against introduced one. Perform checks for required kernel features and compatibility with guest memory backends. Signed-off-by: Andrey Gruzdev --- migration/migration.c | 116

[PATCH v5 3/4] migration: support UFFD write fault processing in ram_save_iterate()

2020-12-04 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v5 4/4] migration: implementation of background snapshot thread

2020-12-04 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v4 5/6] the rest of write tracking migration code

2020-11-26 Thread Andrey Gruzdev via
Implementation of bg_migration- iteration/completion/finish. Signed-off-by: Andrey Gruzdev --- migration/migration.c | 74 +-- 1 file changed, 72 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index

[PATCH v4 4/6] implementation of background snapshot thread

2020-11-26 Thread Andrey Gruzdev via
Introducing implementation of 'background' snapshot thread which in overall follows the logic of precopy migration while internally utilizes completely different mechanism to 'freeze' vmstate at the start of snapshot creation. This mechanism is based on userfault_fd with wr-protection support and

[PATCH v4 3/6] support UFFD write fault processing in ram_save_iterate()

2020-11-26 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v4 6/6] introduce simple linear scan rate limiting mechanism

2020-11-26 Thread Andrey Gruzdev via
Since reading UFFD events and saving paged data are performed from the same thread, write fault latencies are sensitive to migration stream stalls. Limiting total page saving rate is a method to reduce amount of noticiable fault resolution latencies. Migration bandwidth limiting is achieved via

[PATCH v4 2/6] introduce UFFD-WP low-level interface helpers

2020-11-26 Thread Andrey Gruzdev via
Implemented support for the whole RAM block memory protection/un-protection. Introduced higher level ram_write_tracking_start() and ram_write_tracking_stop() to start/stop tracking guest memory writes. Signed-off-by: Andrey Gruzdev --- include/exec/memory.h | 7 ++

[PATCH v4 1/6] introduce 'background-snapshot' migration capability

2020-11-26 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 67 +++ migration/migration.h | 1 + qapi/migration.json | 7 - 3 files changed, 74 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index

[PATCH v4 0/6] UFFD write-tracking migration/snapshots

2020-11-26 Thread Andrey Gruzdev via
This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in his series '[PATCH v0 0/4] migration: add background snapshot'. Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that

[PATCH v3 5/7] implementation of vm_start() BH

2020-11-19 Thread Andrey Gruzdev via
To avoid saving updated versions of memory pages we need to start tracking RAM writes before we resume operation of vCPUs. This sequence is especially critical for virtio device backends whos VQs are mapped to main memory and accessed directly not using MMIO callbacks. One problem is that

[PATCH v3 2/7] introduce UFFD-WP low-level interface helpers

2020-11-19 Thread Andrey Gruzdev via
Implemented support for the whole RAM block memory protection/un-protection. Introduced higher level ram_write_tracking_start() and ram_write_tracking_stop() to start/stop tracking guest memory writes. Signed-off-by: Andrey Gruzdev --- include/exec/memory.h | 7 ++ migration/ram.c | 267

[PATCH v3 6/7] the rest of write tracking migration code

2020-11-19 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 72 +-- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index dba388f8bd..ccb451b238 100644 --- a/migration/migration.c +++

[PATCH v3 7/7] introduce simple linear scan rate limiting mechanism

2020-11-19 Thread Andrey Gruzdev via
Since reading UFFD events and saving paged data are performed from the same thread, write fault latencies are sensitive to migration stream stalls. Limiting total page saving rate is a method to reduce amount of noticiable fault resolution latencies. Migration bandwidth limiting is achieved via

[PATCH v3 4/7] implementation of write-tracking migration thread

2020-11-19 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 168 +- migration/migration.h | 3 + migration/savevm.c| 1 - migration/savevm.h| 2 + 4 files changed, 171 insertions(+), 3 deletions(-) diff --git a/migration/migration.c

[PATCH v3 3/7] support UFFD write fault processing in ram_save_iterate()

2020-11-19 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v3 1/7] introduce 'track-writes-ram' migration capability

2020-11-19 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 96 +++ migration/migration.h | 1 + qapi/migration.json | 7 +++- 3 files changed, 103 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index

[PATCH v3 0/7] UFFD write-tracking migration/snapshots

2020-11-19 Thread Andrey Gruzdev via
Changes with v3: * coding style fixes to pass checkpatch test * qapi/migration.json: change 'track-writes-ram' capability * supported version to 6.0 * fixes to commit message format This patch series is a kind of 'rethinking' of Denis Plotnikov's ideas he's implemented in

[PATCH v2 4/7] Implementation of write-tracking migration thread.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 168 +- migration/migration.h | 3 + migration/savevm.c| 1 - migration/savevm.h| 2 + 4 files changed, 171 insertions(+), 3 deletions(-) diff --git a/migration/migration.c

[PATCH v2 5/7] Implementation of vm_start() BH.

2020-11-18 Thread Andrey Gruzdev via
To avoid saving updated versions of memory pages we need to start tracking RAM writes before we resume operation of vCPUs. This sequence is especially critical for virtio device backends whos VQs are mapped to main memory and accessed directly not using MMIO callbacks. One problem is that

[PATCH v2 6/7] The rest of write tracking migration code.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 72 +-- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index dba388f8bd..ccb451b238 100644 --- a/migration/migration.c +++

[PATCH v2 3/7] Support UFFD write fault processing in ram_save_iterate().

2020-11-18 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v2 0/7] UFFD write-tracking migration/snapshots

2020-11-18 Thread Andrey Gruzdev via
Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that it tends to produce a lot of page duplicates while running VM goes on updating already saved pages. That leads to the fact that vmstate image size is

[PATCH v2 2/7] Introduced UFFD-WP low-level interface helpers. Implemented support for the whole RAM block memory protection/un-protection. Higher level ram_write_tracking_start() and ram_write_tracki

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- include/exec/memory.h | 7 ++ migration/ram.c | 267 ++ migration/ram.h | 4 + 3 files changed, 278 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 0f3e6bcd5e..3d798fce16

[PATCH v2 1/7] Introduce 'track-writes-ram' migration capability.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 96 +++ migration/migration.h | 1 + qapi/migration.json | 7 +++- 3 files changed, 103 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index

[PATCH v2 7/7] Introduced simple linear scan rate limiting mechanism for write tracking migration.

2020-11-18 Thread Andrey Gruzdev via
Since reading UFFD events and saving paged data are performed from the same thread, write fault latencies are sensitive to migration stream stalls. Limiting total page saving rate is a method to reduce amount of noticiable fault resolution latencies. Migration bandwidth limiting is achieved via

[PATCH v1 6/7] The rest of write tracking migration code.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 72 +-- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index dba388f8bd..48793b748b 100644 --- a/migration/migration.c +++

[PATCH v1 0/7] UFFD write-tracking migration/snapshots

2020-11-18 Thread Andrey Gruzdev via
Currently the only way to make (external) live VM snapshot is using existing dirty page logging migration mechanism. The main problem is that it tends to produce a lot of page duplicates while running VM goes on updating already saved pages. That leads to the fact that vmstate image size is

[PATCH v1 2/7] Introduced UFFD-WP low-level interface helpers. Implemented support for the whole RAM block memory protection/un-protection. Higher level ram_write_tracking_start() and ram_write_tracki

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- include/exec/memory.h | 7 ++ migration/ram.c | 267 ++ migration/ram.h | 4 + 3 files changed, 278 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 0f3e6bcd5e..3d798fce16

[PATCH v1 3/7] Support UFFD write fault processing in ram_save_iterate().

2020-11-18 Thread Andrey Gruzdev via
In this particular implementation the same single migration thread is responsible for both normal linear dirty page migration and procesing UFFD page fault events. Processing write faults includes reading UFFD file descriptor, finding respective RAM block and saving faulting page to the migration

[PATCH v1 1/7] Introduce 'track-writes-ram' migration capability.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 96 +++ migration/migration.h | 1 + qapi/migration.json | 7 +++- 3 files changed, 103 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index

[PATCH v1 5/7] Implementation of vm_start() BH.

2020-11-18 Thread Andrey Gruzdev via
To avoid saving updated versions of memory pages we need to start tracking RAM writes before we resume operation of vCPUs. This sequence is especially critical for virtio device backends whos VQs are mapped to main memory and accessed directly not using MMIO callbacks. One problem is that

[PATCH v1 7/7] Introduced simple linear scan rate limiting mechanism for write tracking migration.

2020-11-18 Thread Andrey Gruzdev via
Since reading UFFD events and saving paged data are performed from the same thread, write fault latencies are sensitive to migration stream stalls. Limiting total page saving rate is a method to reduce amount of noticiable fault resolution latencies. Migration bandwidth limiting is achieved via

[PATCH v1 4/7] Implementation of write-tracking migration thread.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 168 +- migration/migration.h | 3 + migration/savevm.c| 1 - migration/savevm.h| 2 + 4 files changed, 171 insertions(+), 3 deletions(-) diff --git a/migration/migration.c

[PATCH 2/7] Introduced UFFD-WP low-level interface helpers. Implemented support for the whole RAM block memory protection/un-protection. Higher level ram_write_tracking_start() and ram_write_tracking_

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- include/exec/memory.h | 7 ++ migration/ram.c | 267 ++ migration/ram.h | 4 + 3 files changed, 278 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 0f3e6bcd5e..3d798fce16

[PATCH 5/7] Implementation of vm_start() BH.

2020-11-18 Thread Andrey Gruzdev via
To avoid saving updated versions of memory pages we need to start tracking RAM writes before we resume operation of vCPUs. This sequence is especially critical for virtio device backends whos VQs are mapped to main memory and accessed directly not using MMIO callbacks. One problem is that

[PATCH 1/7] Introduce 'track-writes-ram' migration capability.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 96 +++ migration/migration.h | 1 + qapi/migration.json | 7 +++- 3 files changed, 103 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index

[PATCH 6/7] The rest of write tracking migration code.

2020-11-18 Thread Andrey Gruzdev via
Signed-off-by: Andrey Gruzdev --- migration/migration.c | 72 +-- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 381da4a4d3..92bcb9105d 100644 --- a/migration/migration.c +++

[PATCH 7/7] Introduced simple linear scan rate limiting mechanism for write tracking migration.

2020-11-18 Thread Andrey Gruzdev via
Since reading UFFD events and saving paged data are performed from the same thread, write fault latencies are sensitive to migration stream stalls. Limiting total page saving rate is a method to reduce amount of noticiable fault resolution latencies. Migration bandwidth limiting is achieved via

  1   2   >