On Sat, Jun 30, 2018 at 1:52 AM, Mark Johnston <ma...@freebsd.org> wrote: > On Fri, Jun 29, 2018 at 11:58:31AM +0300, Elena Mihailescu wrote: >> Hello, >> >> I am interested if there is a method to inspect what pages/objects were >> created after a vm_object (the vm_map_entry associated with the object) is >> marked as copy-on-write. More specifically, I'm interested only in the >> pages that were copied when a write operation was proceed on a page that >> belongs to the object marked copy-on-write. >> >> I need this for a live migration feature for bhyve in order to send the >> pages that were modified between the iterations in which I migrate the >> guest's memory(the guest's memory will be migrated in rounds - firstly, all >> memory will be sent remote, then, only the pages that were modified and so >> on). >> >> What I want to implement is the following: >> Step 1: Given a vm_object *obj, mark its associated vm_map_entry *entry as >> copy-on-write. >> Step 2: After a while (a non-deterministic amount of time), >> inspect/retrieve the pages that were created based on information existent >> in the object. >> >> What I tried until now: >> >> I implemented a function in kernel that: >> - gets the vmspace structure pointer for the current process >> - gets the vm_map structure pointer for the vmspace >> - iterates through each vm_map_entry and based on the vm_offset_start and >> vm_offset_end determines vm_map_entry that contains the object I am >> interested in. >> - for this object, it prints some debug information such as: shadow_count, >> ref_count, whether if it has a backing_object or not. >> The code written is similar with the code from here (the way in which I get >> vmspace for the current process and the way I am iterating through >> vm_map_entry and objects): >> [0] >> https://github.com/FreeBSD-UPB/freebsd/blob/projects/bhyve_migration/sys/amd64/vmm/vmm_dev.c#L979 >> >> I have read the following documentation about FreeBSD's implementation for >> virtual memory: >> [1] https://www.freebsd.org/doc/en/books/arch-handbook/vm.html >> [2] >> https://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/article.html >> [3] https://people.freebsd.org/~neel/bhyve/bhyve_nested_paging.pdf >> [4] http://www.cse.chalmers.se/edu/year/2011/course/EDA203/unix4.pdf >> >> As far as I could tell after reading the documentation presented above, I >> should look for the object that the object I am interested in is a shadow >> of or an object that my object is shadow for. > > Right. When a copy-on-write fault results in the creation of a new > object, the new object is said to shadow the original object, which > becomes the backing object for the shadow object. When faults in the > corresponding map entry occur, the fault handler first searches for a > page in the map entry's object, and then falls back to the backing > object if necessary. > >> To do that, I should inspect the following fields from the vm_object >> structure (among others)( >> https://github.com/freebsd/freebsd/blob/master/sys/vm/vm_object.h#L98) : >> >> - int shadow_count; /* how many objects that this is a shadow for */ >> - struct vm_object *backing_object; /* object that I'm a shadow of */ >> >> But in all my tests, for the object I am interested in, the shadow_count is >> 0 and the backing_object is NULL. >> >> The code I use to mark the vm_map_entry for the object I am interested in >> copy-on-write is here: >> [5] >> https://github.com/FreeBSD-UPB/freebsd/blob/projects/bhyve_migration/sys/amd64/vmm/vmm_dev.c#L949 > > MAP_ENTRY_NEEDS_COPY needs to be set in order for the copy-on-write > machinery to work the way you expect. Take a look at vm_map_lookup(): > when it sees that flag and the caller is attempting a write fault, it > creates a shadow object, updates the entry and clears > MAP_ENTRY_NEEDS_COPY, leaving MAP_ENTRY_COW set. > >> Is there anything I am doing wrong? Maybe I misunderstood something about >> the way the virtual memory works in FreeBSD. > > I'll note that inspecting and manipulating vm_map_entry and vm_object > structures in the bhyve code constitutes something of an abstraction > violation, though it's reasonable to proceed this way while working on a > prototype of the feature. That is, I think you should keep trying your > current approach, but just be aware that you are using the copy-on-write > mechanism in a way that the VM system isn't really expecting. >
Can you point out the right approach in our case? Thanks, Mihai >> There is another way I could inspect what pages were created between the >> moment I mark an object (its vm_map_entry) as copy-on-write and a later >> moment? > _______________________________________________ > freebsd-virtualizat...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" _______________________________________________ freebsd-amd64@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-amd64 To unsubscribe, send any mail to "freebsd-amd64-unsubscr...@freebsd.org"
