Chris, thanks for taking a look.
Am 14.05.20 um 23:13 schrieb Chris Hofstaedtler | Deduktiva: > * Uwe Sauter <uwe.sauter...@gmail.com> [200514 22:23]: > [...] >> More details: >> >> I followed these two instructions: >> >> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm--ethernet-x >> >> https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connect-ib-connectx-4-with-kvm--infiniband-x >> >> At about halfway through the second site I can't go on because the mentioned >> paths are not available in /sys: >> /sys/class/infiniband/mlx5_0/device/sriov/0/policy >> /sys/class/infiniband/mlx5_0/device/sriov/0/node >> /sys/class/infiniband/mlx5_0/device/sriov/0/port > > Disclaimer: I don't have the hardware you're talking about. > > However, I took a quick look at the mainline kernel drivers and the > driver sources from mellanox.com - the mainline kernel does _NOT_ > have the code for these files. > > I guess if you want to use that, you'll have to install the ofed > driver from mellanox.com (preferably by starting with the sources > for Ubuntu 20.04). As I mentioned I tried to install Mellanox OFED (for Debian) but it wants to uninstall many PVE related packages. If anyone has successfully installed MOFED on PVE 6 and can provide instructions, I'd be happy. > PS: [1] looks like up to date documentation > [1] > https://docs.mellanox.com/pages/viewpage.action?pageId=25146819#SingleRootIOVirtualization(SR-IOV)-settingupsr-iov > This same document also mentions: """ Write to the sysfs file the number of Virtual Functions you need to create for the PF. You can use one of the following equivalent files: You can use one of the following equivalent files: - A standard Linux kernel generated file that is available in the new kernels. echo [num_vfs] > /sys/class/infiniband/mlx5_0/device/sriov_numvfs Note: This file will be generated only if IOMMU is set in the grub.conf file (by adding intel_iommu=on, as seen in the fourth step under “Setting Up SR-IOV”). """ which I can see and following: """ - A file generated by the mlx5_core driver with the same functionality as the kernel generated one. echo [num_vfs] > /sys/class/infiniband/mlx5_0/device/mlx5_num_vfs Note: This file is used by old kernels that do not support the standard file. In such kernels, using sriov_numvfs results in the following error: “bash: echo: write error: Function not implemented”. """ which I don't see. When I follow this instructions with echo 15 > /sys/class/infiniband/mlx5_0/device/sriov_numvfs I get entries in the form of /sys/class/infiniband/mlx5_0/device/virtfn{NUMBER_FROM_0_to_14} -> ../0000:5e:00.1 (and so forth) but those directories don't have files to set the necessary node and port GUIDs. Regards, Uwe _______________________________________________ pve-user mailing list pve-user@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
