在 2021/7/16 上午5:18, Michael S. Tsirkin 写道:
On Thu, Jul 15, 2021 at 09:38:55AM +0800, Jason Wang wrote:
在 2021/7/15 上午12:22, Max Gurtovoy 写道:
On 7/14/2021 6:07 PM, Stefan Hajnoczi wrote:
It requires much more works than the simple virtqueue interface:
(the main
issues is that the function is not self-contained in a single function)
1) how to interact with the existing device status state machine?
2) how to make it work in a nested environment?
3) how to migrate the PF?
4) do we need to allow more control other than just stop/freeze
the device
in the admin virtqueue? If yes, how to handle the concurrent
access from PF
and VF?
5) how it is expected to work with non-PCI virtio device?
I guess your device splitting proposal addresses some of these things?
Max probably has the most to say about these points.
If you want more input I can try to answer too, but I personally am not
developing devices that need this right now, so I might not be the best
person to propose solutions.
I think we mentioned this in the past and agreed that the only common
entity between my solution for virtio VF migration to this proposal is
the new admin control queue.
I can prepare some draft for this.
In our solution the PF will manage migration process for it's VFs using
the PF admin queue. PF is not migratable.
That limits the use cases.
I don't know who is using nested environments in production so don't
know if it worth talking about that.
There should be plenty users for the nested case.
But, if you would like to implement it for testing, no problem. The VF
in level n, probably seen as PF in level n+1. So it can manage the
migration process for its nested VFs.
The PF dependency makes the design almost impossible to be used in a nested
environment.
Can't we have an emulated PF? This would be the most reasonable thing to
do.
Isn't this much more complicated?
- VMM needs to emulate PF
- it means we need the support for PF live migration
- if we want to live migrate VF in L(x) we need PF(physical or emulated)
from L0 to L(x-1)
- complicate the management stack
If we really want to go with admin + PF, it should be used with the per
VF function as well. That is hide the admin vq + PF from any of the
above layers.
L0 virtio device: admin + PF
L1 virtio device: L0 VMM presents the function that is self contained in
a single function which is implemented vai (admin + PF)
...
Lx virtio device: L(x-1) VMM presents the function that is self
contained in a single function
In another way, if all the function is self contained, we get even more
simplified model
Lx virtio device: L(x-1) VMM presents the function that is self
contained in a single function
But the OS should hide the implementation details on how the device is
mediated and expose a unified uAPI for the userspace VMM(Qemu). That is
what vDPA tries to achieve, provides a single device centric uAPI. This
is another call for having a self contained per device(function)
interface for all those stuffs. It would be very complicated to
introduce uAPI for PF (or what ever it was called), and actually we
can't mandate this even if we invent them. We've already had vendors
that implements the virtqueue state via VF BAR(capability).
Thanks
For question 5) what non-PCI devices are interesting in live migration ?
Why not? Virtio support transport other than PCI (CCW, MMIO).
Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
