On Thu, Oct 02, 2025 at 07:37:45PM +0200, Danilo Krummrich wrote: > > The driver to operate the function in "vGPU" mode as indicated by the > > register has to be in nova-core, since there is only one device ID. > > Yes, the PF driver on the host and the PF (from VM perspective) driver in the > VM > have to be that same. But the VF driver on the host can still be a seaparate > one.
In most cases it is going to be bound to a vfio driver.. However, if you actually want a DRM subsystem device on the VF without a VM I don't know why you'd use a different driver than the one used by the VM on the very same VF, with the very same register programming model.. > > I think it would be good to have Zhi clarify more of this, but from > > what I understand are at least three activites comingled all together: > > > > 1) Boot the PF in "vGPU" mode so it can enable SRIOV > > Ok, this might be where the confusion above comes from. When I talk about > nova-core in vGPU mode I mean nova-core running in the VM on the (from VM > perspective) PF. I would call this nova-core running on a VF (assigned to a VM) Not sure "vgpu" is a helpful word here, lets try to talk about what .ko's and struct device_drivers's the various codes should live in.. > But you seem to mean nova-core running on the host PF with vGPU on top? That > of > course has to be in nova-core. Yes, #1 would be implemented as part of nova-core.ko and it's pci_driver. As I understand it around firmware loading nova-core has to tell the FW if it wants to enable "vGPU" mode or not. If it doesn't then the sriov_configure op should be inhibited and #2 disabled. If it does then sriov_configure should work, #2 is enabled, and DRM on the PF is disabled. > > 2) Enable SRIOV and profile VFs to allocate HW resources to them > > I think that's partially in nova-core and partially in vGPU; nova-core > providing > the abstraction of the corresponding firmware / hardware interfaces and vGPU > controlling the semantics of the resource handling? > This is what I thought vGPU has a secondary part for where it binds to > nova-core > through the auxiliary bus, i.e. vGPU consisting out of two drivers actually; > the > VFIO parts and a "per VF resource controller". This is certainly one option, you can put #2 in an aux driver of the PF in a nova-sriov.ko module that is fully divorced from VFIO. It might go along with a nova-fwctl.ko module too. You could also just embed it in nova-core.ko and have it activate when the PF is booted in "vGPU" mode. Broadly I would suggest the latter. aux devices make most sense to cross subsystems. Micro splitting a single driver with aux devices will make more of a mess than required. Though a good motivating reason would be if nova-srvio.ko is large. > > 3) VFIO variant driver to convert the VF into a "VM PF" with whatever > > mediation and enhancement needed > > That should be vGPU only land. I think it is clear this part should be in a vfio-pci-nova.ko Then you have two more: 4) A PCI driver in a VM that creates a DRM subsystem device This is nova-core.ko + nova-drm.ko 5) A VF driver that creates a DRM subsystem device without a VM Zhi says the device can't do this, but lets assume it could, then I would expect this to be nova-core.ko + nova-drm.ko, same as #4. Jason
