On 09/17/2015 06:39 PM, Greg Kurz wrote: > On Thu, 17 Sep 2015 11:06:01 +0200 > Cornelia Huck <cornelia.h...@de.ibm.com> wrote: > >> Try to cover the basics of virtio migration. >> >> Signed-off-by: Cornelia Huck <cornelia.h...@de.ibm.com> >> --- >> It might help if we add some documentation; at the very least, it will >> prevent myself getting a headache everytime I look at that code :) >> >> Feedback welcome. > Excellent ! This is a very good to start with. > > Reviewed-by: Greg Kurz <gk...@linux.vnet.ibm.com> > > Just a thought below... > >> --- >> docs/virtio-migration.txt | 101 >> ++++++++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 101 insertions(+) >> create mode 100644 docs/virtio-migration.txt >> >> diff --git a/docs/virtio-migration.txt b/docs/virtio-migration.txt >> new file mode 100644 >> index 0000000..9c575e6 >> --- /dev/null >> +++ b/docs/virtio-migration.txt >> @@ -0,0 +1,101 @@ >> +Virtio devices and migration >> +============================ >> + >> +Saving and restoring the state of virtio devices is a bit of a twisty maze, >> +for several reasons: >> +- state is distributed between several parts: >> + - virtio core, for common fields like features, number of queues, ... >> + - virtio transport (pci, ccw, ...), for the different proxy devices and >> + transport specific state (msix vectors, indicators, ...) >> + - virtio device (net, blk, ...), for the different device types and their >> + state (mac address, request queue, ...) >> +- most fields are saved via the stream interface; subsequently, subsections >> + have been added to make cross-version migration possible >> + >> +This file attempts to document the current procedure and point out some >> +caveats. >> + >> + >> +Save state procedure >> +==================== >> + >> +virtio core virtio transport virtio device >> +----------- ---------------- ------------- >> + >> + save() function >> registered >> + via register_savevm() >> +virtio_save() <---------- >> + ------> save_config() >> + - save proxy device >> + - save transport-specific >> + device fields >> +- save common device >> + fields >> +- save common virtqueue >> + fields >> + ------> save_queue() >> + - save transport-specific >> + virtqueue fields >> + ------> save_device() >> + - save device-specific >> + fields >> +- save subsections >> + - device endianness, >> + if changed from >> + default endianness >> + - 64 bit features, if >> + any high feature bit >> + is set >> + - virtio-1 virtqueue >> + fields, if VERSION_1 >> + is set >> + >> + >> +Load state procedure >> +==================== >> + >> +virtio core virtio transport virtio device >> +----------- ---------------- ------------- >> + >> + load() function >> registered >> + via register_savevm() >> +virtio_load() <---------- >> + ------> load_config() >> + - load proxy device >> + - load transport-specific >> + device fields >> +- load common device >> + fields >> +- load common virtqueue >> + fields >> + ------> load_queue() >> + - load transport-specific >> + virtqueue fields >> +- notify guest >> + ------> load_device() >> + - load device-specific >> + fields >> +- load subsections >> + - device endianness >> + - 64 bit features >> + - virtio-1 virtqueue >> + fields >> +- sanitize endianness >> +- sanitize features >> +- virtqueue index sanity >> + check >> + - feature-dependent setup >> + >> + >> +Implications of this setup >> +========================== >> + >> +Devices need to be careful in their state processing during load: The >> +load_device() procedure is invoked by the core before subsections have >> +been loaded. Any code that depends on information transmitted in subsections >> +therefore has to be invoked in the device's load() function _after_ >> +virtio_load() returned (like e.g. code depending on features). >> + > From a VMState standpoint, such code would land in a post-load callback most > of > the time. Would that help comprehension if we introduce a virtio_post_load() > function ?
I think we could. (With calling a device specific method in virtio_post_load()). >> +Any extension of the state being migrated should be done in subsections >> +added to the core for compatibility reasons. If transport or device specific >> +state is added, core needs to invoke a callback from the new subsection.
