On Thu, Mar 8, 2018 at 8:58 PM, Dr. David Alan Gilbert (git) <dgilb...@redhat.com> wrote: > From: "Dr. David Alan Gilbert" <dgilb...@redhat.com> > > Add some notes to the migration documentation for shared memory > postcopy. > > Signed-off-by: Dr. David Alan Gilbert <dgilb...@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lur...@redhat.com> > --- > docs/devel/migration.rst | 41 +++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 41 insertions(+) > > diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst > index 9d1b7657f0..e32b087f6e 100644 > --- a/docs/devel/migration.rst > +++ b/docs/devel/migration.rst > @@ -577,3 +577,44 @@ Postcopy now works with hugetlbfs backed memory: > hugepages works well, however 1GB hugepages are likely to be problematic > since it takes ~1 second to transfer a 1GB hugepage across a 10Gbps > link, > and until the full page is transferred the destination thread is > blocked. > + > +Postcopy with shared memory > +--------------------------- > + > +Postcopy migration with shared memory needs explicit support from the other > +processes that share memory and from QEMU. There are restrictions on the > type of > +memory that userfault can support shared. > + > +The Linux kernel userfault support works on `/dev/shm` memory and on > `hugetlbfs` > +(although the kernel doesn't provide an equivalent to > `madvise(MADV_DONTNEED)` > +for hugetlbfs which may be a problem in some configurations). > + > +The vhost-user code in QEMU supports clients that have Postcopy support, > +and the `vhost-user-bridge` (in `tests/`) and the DPDK package have changes > +to support postcopy. > + > +The client needs to open a userfaultfd and register the areas > +of memory that it maps with userfault. The client must then pass the > +userfaultfd back to QEMU together with a mapping table that allows > +fault addresses in the clients address space to be converted back to > +RAMBlock/offsets. The client's userfaultfd is added to the postcopy > +fault-thread and page requests are made on behalf of the client by QEMU. > +QEMU performs 'wake' operations on the client's userfaultfd to allow it > +to continue after a page has arrived. > + > +.. note:: > + There are two future improvements that would be nice: > + a) Some way to make QEMU ignorant of the addresses in the clients > + address space > + b) Avoiding the need for QEMU to perform ufd-wake calls after the > + pages have arrived > + > +Retro-fitting postcopy to existing clients is possible: > + a) A mechanism is needed for the registration with userfault as above, > + and the registration needs to be coordinated with the phases of > + postcopy. In vhost-user extra messages are added to the existing > + control channel. > + b) Any thread that can block due to guest memory accesses must be > + identified and the implication understood; for example if the > + guest memory access is made while holding a lock then all other > + threads waiting for that lock will also be blocked. > -- > 2.14.3 > > -- Marc-André Lureau
