Hello! After some investigation of ixgbe and Linux kernel code.
I found function which return this "error 38", it's sriov_enable from Linux Kernel (http://lxr.free-electrons.com/source/drivers/pci/iov.c?v=3.16#L173). And error 38 means return of ENOSYS (http://lxr.free-electrons.com/source/include/uapi/asm-generic/errno.h?v=3.16#L9) from it. Which means "Function not implemented". Finally, Linux Kernel really sure about virtual nature of this bypassed NIC. if (!dev->is_physfn) return -ENOSYS; So, we need dig into kernel code. On Fri, Jun 19, 2015 at 12:05 PM, Pavel Odintsov <pavel.odint...@gmail.com> wrote: > Sorry, for the mess. It was very hard day and I post fe silly posts > due to my inattention (and incorrectly selected ssh window). > > Finally, I have got only single issue with ixgbe driver there. > > I can't create VF inside KVM VM. Not with max_vfs param, not with echo > 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs. > > Could you help me? > > On Fri, Jun 19, 2015 at 11:58 AM, Pavel Odintsov > <pavel.odint...@gmail.com> wrote: >> Hello! >> >> Please ignre this: >> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >> -bash: echo: write error: Function not implemented >> >> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >> -bash: echo: write error: Function not implemented >> >> >> Now it works fine. But broken name which refer to /proc/* instead of >> /sys/* still an issue. >> >> >> On Fri, Jun 19, 2015 at 11:55 AM, Pavel Odintsov >> <pavel.odint...@gmail.com> wrote: >>> Hello! >>> >>> Some details. >>> >>> Error: >>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: Permission denied >>> >>> Definitely related with non existent proc fs path: >>> find /proc/|grep sriov_numvfs|wc -l >>> 0 >>> >>> It's definitely a bug. >>> >>> Because I could find this pseudo files in /sys: >>> root@filter ~ # find /sys/|grep vfs >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalvfs >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalvfs >>> root@filter ~ # find /proc/|grep vfs >>> /proc/sys/vm/vfs_cache_pressure >>> >>> But I still can't use sriov_numvfs from there: >>> >>> cat /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalvfs >>> 63 >>> >>> cat /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalvfs >>> 63 >>> >>> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >>> -bash: echo: write error: Function not implemented >>> >>> echo 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >>> -bash: echo: write error: Function not implemented >>> >>> dmesg|tail -n 100 >>> [3004270.392052] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver >>> - version 3.19.1-k >>> [3004270.392316] ixgbe: Copyright (c) 1999-2014 Intel Corporation. >>> [3004295.119478] vfio-pci 0000:03:00.0: Driver doesn't support SRIOV >>> configuration via sysfs >>> [3004299.761902] vfio-pci 0000:03:00.1: Driver doesn't support SRIOV >>> configuration via sysfs >>> >>> :( >>> >>> On Fri, Jun 19, 2015 at 11:37 AM, Pavel Odintsov >>> <pavel.odint...@gmail.com> wrote: >>>> Hello, folks! >>>> >>>> I have KVM VM with bypassed PCI-E 82599 two port NIC. >>>> >>>> Driver version: 4.0.3 at Debian Jessie 3.16 kernel. >>>> >>>> lspci -v|grep Eth >>>> 00:02.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >>>> RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 20) >>>> 00:05.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >>>> SFI/SFP+ Network Connection (rev 01) >>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 >>>> 00:06.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit >>>> SFI/SFP+ Network Connection (rev 01) >>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 >>>> >>>> And I want to use VF inside this KVM VM. >>>> >>>> And I doing following: >>>> rmmod ixgbe ixgbevf >>>> modprobe ixgbe max_vfs=2,2 >>>> modprobe ixgbevf >>>> >>>> But after this operation I haven't saw virtual functions in lspci output: >>>> lspci -tv >>>> -[0000:00]-+-00.0 Intel Corporation 440FX - 82441FX PMC [Natoma] >>>> +-01.0 Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] >>>> +-01.1 Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] >>>> +-01.3 Intel Corporation 82371AB/EB/MB PIIX4 ACPI >>>> +-02.0 Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 >>>> PCI Fast Ethernet Adapter >>>> +-03.0 Intel Corporation 82801I (ICH9 Family) USB UHCI >>>> Controller #1 >>>> +-03.1 Intel Corporation 82801I (ICH9 Family) USB UHCI >>>> Controller #2 >>>> +-03.2 Intel Corporation 82801I (ICH9 Family) USB UHCI >>>> Controller #3 >>>> +-03.7 Intel Corporation 82801I (ICH9 Family) USB2 EHCI >>>> Controller #1 >>>> +-04.0 Red Hat, Inc Virtio memory balloon >>>> +-05.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ >>>> Network Connection >>>> +-06.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ >>>> Network Connection >>>> \-07.0 Cirrus Logic GD 5446 >>>> >>>> And haven't saw any additional interfaces: >>>> >>>> ip link show >>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN >>>> mode DEFAULT group default >>>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast >>>> state UP mode DEFAULT group default qlen 1000 >>>> link/ether 52:54:00:ec:be:23 brd ff:ff:ff:ff:ff:ff >>>> 19: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode >>>> DEFAULT group default qlen 1000 >>>> link/ether 90:e2:ba:83:3f:24 brd ff:ff:ff:ff:ff:ff >>>> 20: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP >>>> mode DEFAULT group default qlen 1000 >>>> link/ether 90:e2:ba:83:3f:25 brd ff:ff:ff:ff:ff:ff >>>> >>>> >>>> I saw this error in dmesg: >>>> >>>> [848550.356391] ixgbe 0000:00:06.0: Enabling SR-IOV VFs using the >>>> max_vfs module parameter is deprecated. >>>> [848550.356398] ixgbe 0000:00:06.0: Please use the pci sysfs interface >>>> instead. Ex: >>>> [848550.356403] ixgbe 0000:00:06.0: echo '2' > >>>> /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>> [848705.672383] ixgbe 0000:00:06.0 (unregistered net_device): Failed >>>> to enable PCI sriov: -38 <------- COULD BE IMPORTANT >>>> >>>> And try another approach: >>>> rmmod ixgbe ixgbevf >>>> >>>> modprobe ixgbe >>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>> echo '2' > /sys/bus/pci/devices/0000:00:05.0/sriov_numvfs >>>> >>>> And it's not working at all: >>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: Permission denied >>>> >>>> Is it possible to use VF inside KVM VM? >>>> >>>> -- >>>> Sincerely yours, Pavel Odintsov >>> >>> >>> >>> -- >>> Sincerely yours, Pavel Odintsov >> >> >> >> -- >> Sincerely yours, Pavel Odintsov > > > > -- > Sincerely yours, Pavel Odintsov -- Sincerely yours, Pavel Odintsov ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
