On Tue, Feb 19, 2013 at 10:41:08AM -0500, Michael R. Hines wrote: > On 02/18/2013 03:24 AM, Orit Wasserman wrote: > >Hi Michael, The guest device state is quite small (~100K probably > >less) especially when compared to the guest memory and we already > >are pinning the guest memory for RDMA any way I was actually > >wondering about the memory pinning, do we pin all guest memory > >pages as migration starts or on demand? > > The patch supports both methods. There's a function called > "rdma_server_prepare()" which optionally pins all the memory in > advanced. > > We prefer on-demand pinning, of course, because we want to preserve > the ability to do ballooning and the occasional madvise() calls. > > The patch defaults to pinning everything right now for performance > evaluation.......... later I'll make sure to switch that off when > we've coalesced on a solution for the actual RDMA transfer itself.
How well does the incremental pinning work? The pin everything approach does not look very useful because it conflicts with memory overcommit. > >For the guest memory pages, sending the pages directly without > >QemuFile (that does buffering) is better, I would suggest > >implementing an QemuRDMAFile for this. It will have a new API for > >the memory pages (zero copy) so instead of using qemu_put_buffer > >we will call qemu_rdma_buffer or it can reimplement > >qemu_put_buffer (you need to add offset). As for device state > >which is sent in the last phase and is small you can modify the > >current implementation. (well Paolo sent patches that are changing > >this but I think buffering is still an option) The current > >migration implementation copies the device state into a buffer and > >than send the data from the buffer (QemuBufferedFile). You only > >need to pin this buffer, and RDMA it after all the device state > >was written into it. Regards, Orit > I like it .......... can you help me understand - how much different > is this design than the "QEMUFileOps" design that Paulo suggested? > > Or it basically the same? ....reimplementing > qemu_put_buffer/get_buffer() for RDMA purposes...... > > - Michael >
