On Thu, 18 Apr 2019 15:50:43 -0300 Flavio Leitner <[email protected]> wrote:
> On Thu, Apr 18, 2019 at 12:06:57PM -0600, Alex Williamson wrote: > > On Thu, 18 Apr 2019 13:56:23 -0300 > > Flavio Leitner <[email protected]> wrote: > > > > > On Thu, Apr 18, 2019 at 10:43:11AM -0600, Alex Williamson wrote: > > > > On Thu, 18 Apr 2019 13:23:54 -0300 > > > > Flavio Leitner <[email protected]> wrote: > > > > > > > > > On Thu, Apr 18, 2019 at 11:05:28AM -0400, Aaron Conole wrote: > > > > > > On some systems, it's possible that the initialization of the misc > > > > > > chardev > > > > > > associated with /dev/vfio/vfio is delayed. This happens on > > > > > > machines with > > > > > > large numbers of cores (at least 88+). If this delay exceeds the > > > > > > time > > > > > > required for ovs-vswitchd to call the dpdk initialization routine, > > > > > > the > > > > > > permissions won't be updated and the open call will return EACCES. > > > > > > > > > > > > To fix this, we explicitly allow global open. This means any user > > > > > > may > > > > > > open() the vfio device before the vfio modules are initialized, thus > > > > > > triggering a module load. The applications (including ovs-vswitchd) > > > > > > would be pended while the module loads. This should be safe, as > > > > > > any user > > > > > > may open the vfio device once the module is loaded anyway, since > > > > > > the. > > > > > > module rewrites the permissions as 0666. > > > > > > > > > > > > Signed-off-by: Aaron Conole <[email protected]> > > > > > > --- > > > > > > rhel/usr_lib_udev_rules.d_91-vfio.rules | 1 + > > > > > > 1 file changed, 1 insertion(+) > > > > > > > > > > > > diff --git a/rhel/usr_lib_udev_rules.d_91-vfio.rules > > > > > > b/rhel/usr_lib_udev_rules.d_91-vfio.rules > > > > > > index 8e34b2a2b..c0504ab5a 100644 > > > > > > --- a/rhel/usr_lib_udev_rules.d_91-vfio.rules > > > > > > +++ b/rhel/usr_lib_udev_rules.d_91-vfio.rules > > > > > > @@ -1 +1,2 @@ > > > > > > ACTION=="add", SUBSYSTEM=="vfio*", GROUP="hugetlbfs", MODE="0660" > > > > > > +ACTION=="add", SUBSYSTEM=="misc", KERNEL=="vfio", MODE="0666" > > > > > > > > > > Wouldn't be better to fix this at the module to fix the permissions > > > > > before whatever is taking long to initialize? > > > > > > > > The stronger security policy is to not allow unprivileged users to > > > > trigger an operation which loads a module, so no, upper level system > > > > policy should relax that as necessary, but not the module. VFIO is > > > > unavailable until enabled through privileged entities, so there's some > > > > degree that ovs is "jumping the gun" in making use of it, afaict. > > > > > > Ok, but if the machine has few cores, then the permission is set to > > > 0666 and I assumed that it was done by the module itself. Assuming > > > that it's true, then why can't the module fix the permission before? > > > > Because if the file were initially mode 0666 then an unprivileged user > > accessing the file will load the module, which as discussed above is a > > weaker security policy. > > Agreed, but I was referring to after the proper trigger. > > > Any case of the permissions being "fixed" by > > the module load, directed by a privileged entity, asynchronous to ovs > > using this interface is a race, no matter how early the module manages > > to fix those permissions. > > True. > > > > Another thing is that when the module is ready and the event is sent > > > out, what holds OVS for not trying to open and get EACCESS before > > > udev is triggered to fix the device permission? > > > > If there were a race, could ovs ever run before udev on system > > startup? Probably not. > > It does wait, but only for the udev to settle, which means if the > module has not triggered an event until that time, OVS will not wait > and we still have a race. But udev isn't waiting on the module to trigger an event, the module contains a MODULE_ALIAS, so I believe it's just the static processing of the modules.alias that triggers the event. > > Ideally perhaps a cleaner solution might be an > > explicit dependency on the vfio module specific to ovs startup rather > > than changing a system policy, but it really depends on the context and > > use cases. Thanks, > > It does have. The driverctl will bind the devices to vfio-pci but > the problem is that which signal we should rely on to know when > the vfio module is still initializing, or failed or finished. What signal/mechanism is being used currently? If driverctl is asked to set a driver override it does: 1) if module is not loaded, modprobe 2) unbinds device from existing driver, if any 3) sets driver_override 4) triggers drivers_probe 5) tests if device is bound to a driver, any driver There are certainly some deficiencies here, unbinding the device before setting the driver_override leaves the device open to getting bound by the wrong driver, and the verification in the last step could be more specific in testing for binding to the correct driver, but step #1 is the modprobe of the driver, which should be a synchronous operation. We shouldn't be able to complete a 'driverctl set-override $DEV vfio-pci' without vfio being initialized, afaict. Thanks, Alex _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
