I have finished my "home work". issue reproduced with 4.0.0 kernel and latest 4.1.1 ixgbe driver.
Same error in dmesg: [ 221.676388] ixgbe 0000:00:05.0 0000:00:05.0 (uninitialized): Failed to enable PCI sriov: -38 On Fri, Jun 19, 2015 at 12:58 PM, Pavel Odintsov <pavel.odint...@gmail.com> wrote: > Hello! > > After some investigation of ixgbe and Linux kernel code. > > I found function which return this "error 38", it's sriov_enable from > Linux Kernel > (http://lxr.free-electrons.com/source/drivers/pci/iov.c?v=3.16#L173). > > And error 38 means return of ENOSYS > (http://lxr.free-electrons.com/source/include/uapi/asm-generic/errno.h?v=3.16#L9) > from it. Which means "Function not implemented". > > Finally, Linux Kernel really sure about virtual nature of this bypassed NIC. > > if (!dev->is_physfn) > return -ENOSYS; > > So, we need dig into kernel code. > > On Fri, Jun 19, 2015 at 12:05 PM, Pavel Odintsov > <pavel.odint...@gmail.com> wrote: >> Sorry, for the mess. It was very hard day and I post fe silly posts >> due to my inattention (and incorrectly selected ssh window). >> >> Finally, I have got only single issue with ixgbe driver there. >> >> I can't create VF inside KVM VM. Not with max_vfs param, not with echo >> 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs. >> >> Could you help me? >> >> On Fri, Jun 19, 2015 at 11:58 AM, Pavel Odintsov >> <pavel.odint...@gmail.com> wrote: >>> Hello! >>> >>> Please ignre this: >>> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >>> -bash: echo: write error: Function not implemented >>> >>> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >>> -bash: echo: write error: Function not implemented >>> >>> >>> Now it works fine. But broken name which refer to /proc/* instead of >>> /sys/* still an issue. >>> >>> >>> On Fri, Jun 19, 2015 at 11:55 AM, Pavel Odintsov >>> <pavel.odint...@gmail.com> wrote: >>>> Hello! >>>> >>>> Some details. >>>> >>>> Error: >>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: Permission denied >>>> >>>> Definitely related with non existent proc fs path: >>>> find /proc/|grep sriov_numvfs|wc -l >>>> 0 >>>> >>>> It's definitely a bug. >>>> >>>> Because I could find this pseudo files in /sys: >>>> root@filter ~ # find /sys/|grep vfs >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalvfs >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalvfs >>>> root@filter ~ # find /proc/|grep vfs >>>> /proc/sys/vm/vfs_cache_pressure >>>> >>>> But I still can't use sriov_numvfs from there: >>>> >>>> cat /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalvfs >>>> 63 >>>> >>>> cat /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalvfs >>>> 63 >>>> >>>> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >>>> -bash: echo: write error: Function not implemented >>>> >>>> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >>>> -bash: echo: write error: Function not implemented >>>> >>>> dmesg|tail -n 100 >>>> [3004270.392052] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver >>>> - version 3.19.1-k >>>> [3004270.392316] ixgbe: Copyright (c) 1999-2014 Intel Corporation. >>>> [3004295.119478] vfio-pci 0000:03:00.0: Driver doesn't support SRIOV >>>> configuration via sysfs >>>> [3004299.761902] vfio-pci 0000:03:00.1: Driver doesn't support SRIOV >>>> configuration via sysfs >>>> >>>> :( >>>> >>>> On Fri, Jun 19, 2015 at 11:37 AM, Pavel Odintsov >>>> <pavel.odint...@gmail.com> wrote: >>>>> Hello, folks! >>>>> >>>>> I have KVM VM with bypassed PCI-E 82599 two port NIC. >>>>> >>>>> Driver version: 4.0.3 at Debian Jessie 3.16 kernel. >>>>> >>>>> lspci -v|grep Eth >>>>> 00:02.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >>>>> RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 20) >>>>> 00:05.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >>>>> SFI/SFP+ Network Connection (rev 01) >>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 >>>>> 00:06.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >>>>> SFI/SFP+ Network Connection (rev 01) >>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 >>>>> >>>>> And I want to use VF inside this KVM VM. >>>>> >>>>> And I doing following: >>>>> rmmod ixgbe ixgbevf >>>>> modprobe ixgbe max_vfs=2,2 >>>>> modprobe ixgbevf >>>>> >>>>> But after this operation I haven't saw virtual functions in lspci output: >>>>> lspci -tv >>>>> -[0000:00]-+-00.0 Intel Corporation 440FX - 82441FX PMC [Natoma] >>>>> +-01.0 Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] >>>>> +-01.1 Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] >>>>> +-01.3 Intel Corporation 82371AB/EB/MB PIIX4 ACPI >>>>> +-02.0 Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 >>>>> PCI Fast Ethernet Adapter >>>>> +-03.0 Intel Corporation 82801I (ICH9 Family) USB UHCI >>>>> Controller #1 >>>>> +-03.1 Intel Corporation 82801I (ICH9 Family) USB UHCI >>>>> Controller #2 >>>>> +-03.2 Intel Corporation 82801I (ICH9 Family) USB UHCI >>>>> Controller #3 >>>>> +-03.7 Intel Corporation 82801I (ICH9 Family) USB2 EHCI >>>>> Controller #1 >>>>> +-04.0 Red Hat, Inc Virtio memory balloon >>>>> +-05.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ >>>>> Network Connection >>>>> +-06.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ >>>>> Network Connection >>>>> \-07.0 Cirrus Logic GD 5446 >>>>> >>>>> And haven't saw any additional interfaces: >>>>> >>>>> ip link show >>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN >>>>> mode DEFAULT group default >>>>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast >>>>> state UP mode DEFAULT group default qlen 1000 >>>>> link/ether 52:54:00:ec:be:23 brd ff:ff:ff:ff:ff:ff >>>>> 19: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode >>>>> DEFAULT group default qlen 1000 >>>>> link/ether 90:e2:ba:83:3f:24 brd ff:ff:ff:ff:ff:ff >>>>> 20: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP >>>>> mode DEFAULT group default qlen 1000 >>>>> link/ether 90:e2:ba:83:3f:25 brd ff:ff:ff:ff:ff:ff >>>>> >>>>> >>>>> I saw this error in dmesg: >>>>> >>>>> [848550.356391] ixgbe 0000:00:06.0: Enabling SR-IOV VFs using the >>>>> max_vfs module parameter is deprecated. >>>>> [848550.356398] ixgbe 0000:00:06.0: Please use the pci sysfs interface >>>>> instead. Ex: >>>>> [848550.356403] ixgbe 0000:00:06.0: echo '2' > >>>>> /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>>> [848705.672383] ixgbe 0000:00:06.0 (unregistered net_device): Failed >>>>> to enable PCI sriov: -38 <------- COULD BE IMPORTANT >>>>> >>>>> And try another approach: >>>>> rmmod ixgbe ixgbevf >>>>> >>>>> modprobe ixgbe >>>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>>> echo '2' > /sys/bus/pci/devices/0000:00:05.0/sriov_numvfs >>>>> >>>>> And it's not working at all: >>>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: Permission denied >>>>> >>>>> Is it possible to use VF inside KVM VM? >>>>> >>>>> -- >>>>> Sincerely yours, Pavel Odintsov >>>> >>>> >>>> >>>> -- >>>> Sincerely yours, Pavel Odintsov >>> >>> >>> >>> -- >>> Sincerely yours, Pavel Odintsov >> >> >> >> -- >> Sincerely yours, Pavel Odintsov > > > > -- > Sincerely yours, Pavel Odintsov -- Sincerely yours, Pavel Odintsov ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
