Hanna Czenczek <hre...@redhat.com> wrote: > On 06.02.23 22:02, Juan Quintela wrote: >> Stefan Hajnoczi <stefa...@gmail.com> wrote: >>> On Mon, 6 Feb 2023 at 07:36, Hanna Czenczek <hre...@redhat.com> wrote: >>>> Hi Stefan, >>>> >>>> For true virtio-fs migration, we need to migrate the daemon’s (back >>>> end’s) state somehow. I’m addressing you because you had a talk on this >>>> topic at KVM Forum 2021. :) >>>> >>>> As far as I understood your talk, the only standardized way to migrate a >>>> vhost-user back end’s state is via dbus-vmstate. I believe that >>>> interface is unsuitable for our use case, because we will need to >>>> migrate more than 1 MB of state. Now, that 1 MB limit has supposedly >>>> been chosen arbitrarily, but the introducing commit’s message says that >>>> it’s based on the idea that the data must be supplied basically >>>> immediately anyway (due to both dbus and qemu migration requirements), >>>> and I don’t think we can meet that requirement. >>> Yes, dbus-vmstate is the available today. It's independent of >>> vhost-user and VIRTIO. >> Once that we are here: >> - typical size of your starte (either vhost-user or whatever) > > Difficult to say, completely depends on the use case. When > identifying files by path and organizing them in a tree structure, > probably ~48 bytes per indexed file, plus, say, 16 bytes per open > file. > > So for a small shared filesystem, the state can be very small, but > we’ll also have to prepare for cases where it is in the range of > several MB.
That is not two bad. Anything below a few tens megabytes is easy to manage. Anything in the hundred of megabytes or more really need thought. > The main problem isn’t size but that (when identifying files by path) > we’ll probably want to construct the paths when migrating, which won’t > be done instantaneously. > >> - what are the posibilities that you can enter the iterative stage >> negotiation (i.e. that you can create a dirty bitmap about your state) > > Very good. We should know when parts of the state (information about > a specific indexed or open file) changes. (Exceptions apply, but they > mostly come down to whether indexed files are identified by path or > file handle, which is a choice the user will probably need to make. > Either one comes with caveats.) That is good. >> How much is migration to file used in practice? >> I would like to have some information here. >> It could be necessary probably to be able to encrypt it. And that is a >> (different) whole can of worms. > > I don’t think virtio-fs state needs to be encrypted any more than any > other state. It’ll basically just map FUSE inode IDs to a file in the > shared directory, either via path or file handle; and then also track > open(2) flags for opened files. (At least that’s what’s currently on > my radar.) That information should actually be replicated in the > guest, too (because it too will have mapped the filesystem paths to > FUSE inode IDs), so isn’t more security relevant than guest memory. Oh, that was not about virtio-fs at all. Is because you talked about file migration. Right now, we need to use exec migration to do this. but it is clearly suboptimal. Basically we just do a normal migration, but that means that we have a lot of duplicated pages on the destination. But we can do better. Just create a file that is as big as the memory, and write everypage in its own place. So loading is going to be really fast. (yes holes on RAM is a different issue, but we can ignore that for now). And the other thing is that we have to really encrypt it somehow, so I guess that a block cipher should work, but encryption is not my field of expertise at all. In vhost-user-fs case, I fully agree with you that if you are "exporting" part of the local filesystem, encryption don't buy you anything. Later, Juan.
