From: "Dr. David Alan Gilbert" <dgilb...@redhat.com>
Hi,
This is the 3rd cut of my postcopy implementation.
The two largest changes since V2 are:
* The source side return path handler is now a thread, and that
allows the source side fd to be blocking. (I'm using shutdown(2)
to make that thread joinable at the end). I've not changed the
destination side yet, I'm still concerned that we could end
up blocking the HMP/QMP if we're not careful.
* The dirty bitmaps are no longer sync'ed after we switch into postcopy
We no longer need to do it, because in principal nothing should change,
and indeed if it does change, the dirtying caused the page to be resent
and the destination would reject it (as it should since it can have been
changing itself). In practice I'm seeing occasional cases where dirtying
was happening, I'm suspecting this might be related to the networking
code still being live (see the current patches on the list to stop that)
but I haven't dug too deep on this.
There are also a bunch of other fixes:
Docs update (Paolo's comments, and Eric's)
postcopy test: fix error path (zhanghailiang's comment)
More clean up of exit cases; a lot less TODO's
More bisectable - but not there yet
I've also included the QEMUSizedBuffer/QEMUFile patches in this set for those
who want to patch from the mail; but still intend to update them separately.
This code is also now in a public GIT repo; see
https://github.com/orbitfp7/qemu/tree/wp3-postcopy
This version is tag: wp3-postcopy-v3 on the wp3-postcopy branch.
Postcopy requires the kernel modifications from Andrea posted here:
http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00525.html
Current TODO:
1) It's not bisectable yet
2) There are no testsuite additions (although I have a virt-test modification
I've been using).
3) Not all the code is there for systems with hostpagesize!=qemupagesize
4) xbzrle needs disabling once in postcopy
5) RDMA needs some rework
6) The latency measurements are now pretty consistent, no very large spikes,
but they're a bit higher than expected, I need to look at rate limiting
just the background scan.
7) Conversion of destination side return-path to blocking fd needs
investigation
(as per discussion with Paolo)
8) Andrea has suggestions on ways to avoid some of the huge-page splitting
that occurs during the discard phase after precopy.
9) I'd like to format the data on the return path in a more structured way
(i.e. maybe using stuff from my BER world).
10) The ACPI fix in 2.1 that allowed migrating RAMBlocks to be larger than
the source feels like it needs looking at for postcopy.
11) I've got an occasional (1/100 ~ 1/500ish) failure on migration of idle VMs
Dave
Dr. David Alan Gilbert (47):
QEMUSizedBuffer/QEMUFile
Tests: QEMUSizedBuffer/QEMUBuffer
Start documenting how postcopy works.
qemu_ram_foreach_block: pass up error value, and down the ramblock
name
improve DPRINTF macros, add to savevm
Add qemu_get_counted_string to read a string prefixed by a count byte
Create MigrationIncomingState
socket shutdown
Return path: Open a return path on QEMUFile for sockets
Return path: socket_writev_buffer: Block even on non-blocking fd's
Migration commands
Return path: Control commands
Return path: Send responses from destination to source
Return path: Source handling of return path
qemu_loadvm errors and debug
ram_debug_dump_bitmap: Dump a migration bitmap as text
Rework loadvm path for subloops
Add migration-capability boolean for postcopy-ram.
Add wrappers and handlers for sending/receiving the postcopy-ram
migration messages.
QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream
migrate_init: Call from savevm
Allow savevm handlers to state whether they could go into postcopy
postcopy: OS support test
migrate_start_postcopy: Command to trigger transition to postcopy
MIG_STATE_POSTCOPY_ACTIVE: Add new migration state
qemu_savevm_state_complete: Postcopy changes
Postcopy: Maintain sentmap during postcopy pre phase
Postcopy page-map-incoming (PMI) structure
postcopy: Add incoming_init/cleanup functions
postcopy: Incoming initialisation
postcopy: ram_enable_notify to switch on userfault
Postcopy: postcopy_start
Postcopy: Rework migration thread for postcopy mode
mig fd_connect: open return path
Postcopy: Create a fault handler thread before marking the ram as
userfault
Page request: Add MIG_RPCOMM_REQPAGES reverse command
Page request: Process incoming page request
Page request: Consume pages off the post-copy queue
Add assertion to check migration_dirty_pages
postcopy_ram.c: place_page and helpers
Postcopy: Use helpers to map pages during migration
qemu_ram_block_from_host
Don't sync dirty bitmaps in postcopy
Postcopy; Handle userfault requests
Start up a postcopy/listener thread ready for incoming page data
postcopy: Wire up loadvm_postcopy_ram_handle_{run,end} commands
End of migration for postcopy
Makefile.objs | 2 +-
arch_init.c | 499 +++++++++++++++++++--
docs/migration.txt | 188 ++++++++
exec.c | 66 ++-
hmp-commands.hx | 15 +
hmp.c | 7 +
hmp.h | 1 +
include/exec/cpu-common.h | 8 +-
include/migration/migration.h | 127 ++++++
include/migration/postcopy-ram.h | 89 ++++
include/migration/qemu-file.h | 47 ++
include/migration/vmstate.h | 2 +-
include/qemu/sockets.h | 1 +
include/qemu/typedefs.h | 8 +-
include/sysemu/sysemu.h | 41 +-
migration-rdma.c | 4 +-
migration.c | 680 ++++++++++++++++++++++++++--
postcopy-ram.c | 924 +++++++++++++++++++++++++++++++++++++++
qapi-schema.json | 14 +-
qemu-file.c | 552 ++++++++++++++++++++++-
qmp-commands.hx | 19 +
savevm.c | 848 ++++++++++++++++++++++++++++++++---
tests/Makefile | 2 +-
tests/test-vmstate.c | 73 ++--
util/qemu-sockets.c | 28 ++
25 files changed, 4069 insertions(+), 176 deletions(-)
create mode 100644 include/migration/postcopy-ram.h
create mode 100644 postcopy-ram.c
--
1.9.3
