Hello, folks! Thanks for answer! Really important information! I will ask my equipment provider.
But could I ask. Do you have this drivers in open source somewhere? On Fri, Jun 19, 2015 at 6:42 PM, Ronciak, John <john.ronc...@intel.com> wrote: > In addition Pavel you should go back to the Intel Field person you got the > NIC's from to get the enabling code. Most people who buy these NIC's are > building their own kernels which have the modified driver which includes this > code. That code can't be pushed upstream as there is no generic > kernel/driver interface for such a feature. > > Cheers, > John > >> -----Original Message----- >> From: Skidmore, Donald C [mailto:donald.c.skidm...@intel.com] >> Sent: Friday, June 19, 2015 8:31 AM >> To: Pavel Odintsov; e1000-devel@lists.sourceforge.net >> Subject: Re: [E1000-devel] VF inside VM with bypassed PCI-E ixgbe >> >> Hey Pavel, >> >> In order to support the bypass drive you need a special driver. I'm >> surprised >> you weren't supplied with one when you bought the NIC. Without it you >> can't communicate with the uC that handles all the bypass functionality. >> Likewise the SDP aren't connected to what the upstream ixgbe driver thinks >> they should be so "undefined" things could happen. >> >> Thanks, >> -Don Skidmore <donald.c.skidm...@intel.com> >> >> >> > -----Original Message----- >> > From: Pavel Odintsov [mailto:pavel.odint...@gmail.com] >> > Sent: Friday, June 19, 2015 3:43 AM >> > To: e1000-devel@lists.sourceforge.net >> > Subject: Re: [E1000-devel] VF inside VM with bypassed PCI-E ixgbe >> > >> > I have finished my "home work". >> > >> > issue reproduced with 4.0.0 kernel and latest 4.1.1 ixgbe driver. >> > >> > Same error in dmesg: >> > [ 221.676388] ixgbe 0000:00:05.0 0000:00:05.0 (uninitialized): Failed >> > to enable PCI sriov: -38 >> > >> > On Fri, Jun 19, 2015 at 12:58 PM, Pavel Odintsov >> > <pavel.odint...@gmail.com> wrote: >> > > Hello! >> > > >> > > After some investigation of ixgbe and Linux kernel code. >> > > >> > > I found function which return this "error 38", it's sriov_enable >> > > from Linux Kernel (http://lxr.free- >> > electrons.com/source/drivers/pci/iov.c?v=3.16#L173). >> > > >> > > And error 38 means return of ENOSYS >> > > (http://lxr.free-electrons.com/source/include/uapi/asm-generic/errno >> > > .h >> > > ?v=3.16#L9) from it. Which means "Function not implemented". >> > > >> > > Finally, Linux Kernel really sure about virtual nature of this bypassed >> > > NIC. >> > > >> > > if (!dev->is_physfn) >> > > return -ENOSYS; >> > > >> > > So, we need dig into kernel code. >> > > >> > > On Fri, Jun 19, 2015 at 12:05 PM, Pavel Odintsov >> > > <pavel.odint...@gmail.com> wrote: >> > >> Sorry, for the mess. It was very hard day and I post fe silly posts >> > >> due to my inattention (and incorrectly selected ssh window). >> > >> >> > >> Finally, I have got only single issue with ixgbe driver there. >> > >> >> > >> I can't create VF inside KVM VM. Not with max_vfs param, not with >> > >> echo >> > >> 2 > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs. >> > >> >> > >> Could you help me? >> > >> >> > >> On Fri, Jun 19, 2015 at 11:58 AM, Pavel Odintsov >> > >> <pavel.odint...@gmail.com> wrote: >> > >>> Hello! >> > >>> >> > >>> Please ignre this: >> > >>> echo 2 > >> > >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >> > >>> -bash: echo: write error: Function not implemented >> > >>> >> > >>> echo 2 > >> > >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >> > >>> -bash: echo: write error: Function not implemented >> > >>> >> > >>> >> > >>> Now it works fine. But broken name which refer to /proc/* instead >> > >>> of >> > >>> /sys/* still an issue. >> > >>> >> > >>> >> > >>> On Fri, Jun 19, 2015 at 11:55 AM, Pavel Odintsov >> > >>> <pavel.odint...@gmail.com> wrote: >> > >>>> Hello! >> > >>>> >> > >>>> Some details. >> > >>>> >> > >>>> Error: >> > >>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >> > >>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: Permission >> > >>>> denied >> > >>>> >> > >>>> Definitely related with non existent proc fs path: >> > >>>> find /proc/|grep sriov_numvfs|wc -l >> > >>>> 0 >> > >>>> >> > >>>> It's definitely a bug. >> > >>>> >> > >>>> Because I could find this pseudo files in /sys: >> > >>>> root@filter ~ # find /sys/|grep vfs >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalvfs >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalvfs >> > >>>> root@filter ~ # find /proc/|grep vfs >> > >>>> /proc/sys/vm/vfs_cache_pressure >> > >>>> >> > >>>> But I still can't use sriov_numvfs from there: >> > >>>> >> > >>>> cat >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalvfs >> > >>>> 63 >> > >>>> >> > >>>> cat >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalvfs >> > >>>> 63 >> > >>>> >> > >>>> echo 2 > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs >> > >>>> -bash: echo: write error: Function not implemented >> > >>>> >> > >>>> echo 2 > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs >> > >>>> -bash: echo: write error: Function not implemented >> > >>>> >> > >>>> dmesg|tail -n 100 >> > >>>> [3004270.392052] ixgbe: Intel(R) 10 Gigabit PCI Express Network >> > >>>> Driver >> > >>>> - version 3.19.1-k >> > >>>> [3004270.392316] ixgbe: Copyright (c) 1999-2014 Intel Corporation. >> > >>>> [3004295.119478] vfio-pci 0000:03:00.0: Driver doesn't support >> > >>>> SRIOV configuration via sysfs [3004299.761902] vfio-pci >> > >>>> 0000:03:00.1: Driver doesn't support SRIOV configuration via >> > >>>> sysfs >> > >>>> >> > >>>> :( >> > >>>> >> > >>>> On Fri, Jun 19, 2015 at 11:37 AM, Pavel Odintsov >> > >>>> <pavel.odint...@gmail.com> wrote: >> > >>>>> Hello, folks! >> > >>>>> >> > >>>>> I have KVM VM with bypassed PCI-E 82599 two port NIC. >> > >>>>> >> > >>>>> Driver version: 4.0.3 at Debian Jessie 3.16 kernel. >> > >>>>> >> > >>>>> lspci -v|grep Eth >> > >>>>> 00:02.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >> > >>>>> RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 20) >> > >>>>> 00:05.0 Ethernet controller: Intel Corporation 82599ES >> > >>>>> 10-Gigabit SFI/SFP+ Network Connection (rev 01) >> > >>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 >> > >>>>> 00:06.0 Ethernet controller: Intel Corporation 82599ES >> > >>>>> 10-Gigabit SFI/SFP+ Network Connection (rev 01) >> > >>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 >> > >>>>> >> > >>>>> And I want to use VF inside this KVM VM. >> > >>>>> >> > >>>>> And I doing following: >> > >>>>> rmmod ixgbe ixgbevf >> > >>>>> modprobe ixgbe max_vfs=2,2 >> > >>>>> modprobe ixgbevf >> > >>>>> >> > >>>>> But after this operation I haven't saw virtual functions in lspci >> output: >> > >>>>> lspci -tv >> > >>>>> -[0000:00]-+-00.0 Intel Corporation 440FX - 82441FX PMC [Natoma] >> > >>>>> +-01.0 Intel Corporation 82371SB PIIX3 ISA >> > >>>>> [Natoma/Triton II] >> > >>>>> +-01.1 Intel Corporation 82371SB PIIX3 IDE >> > >>>>> [Natoma/Triton II] >> > >>>>> +-01.3 Intel Corporation 82371AB/EB/MB PIIX4 ACPI >> > >>>>> +-02.0 Realtek Semiconductor Co., Ltd. >> > >>>>> RTL-8100/8101L/8139 PCI Fast Ethernet Adapter >> > >>>>> +-03.0 Intel Corporation 82801I (ICH9 Family) USB >> > >>>>> UHCI >> > Controller #1 >> > >>>>> +-03.1 Intel Corporation 82801I (ICH9 Family) USB >> > >>>>> UHCI >> > Controller #2 >> > >>>>> +-03.2 Intel Corporation 82801I (ICH9 Family) USB >> > >>>>> UHCI >> > Controller #3 >> > >>>>> +-03.7 Intel Corporation 82801I (ICH9 Family) USB2 >> > >>>>> EHCI Controller #1 >> > >>>>> +-04.0 Red Hat, Inc Virtio memory balloon >> > >>>>> +-05.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ >> > >>>>> Network Connection >> > >>>>> +-06.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ >> > >>>>> Network Connection >> > >>>>> \-07.0 Cirrus Logic GD 5446 >> > >>>>> >> > >>>>> And haven't saw any additional interfaces: >> > >>>>> >> > >>>>> ip link show >> > >>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state >> > >>>>> UNKNOWN mode DEFAULT group default >> > >>>>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >> > >>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc >> > >>>>> pfifo_fast state UP mode DEFAULT group default qlen 1000 >> > >>>>> link/ether 52:54:00:ec:be:23 brd ff:ff:ff:ff:ff:ff >> > >>>>> 19: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state >> > DOWN >> > >>>>> mode DEFAULT group default qlen 1000 >> > >>>>> link/ether 90:e2:ba:83:3f:24 brd ff:ff:ff:ff:ff:ff >> > >>>>> 20: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc >> > mq >> > >>>>> state UP mode DEFAULT group default qlen 1000 >> > >>>>> link/ether 90:e2:ba:83:3f:25 brd ff:ff:ff:ff:ff:ff >> > >>>>> >> > >>>>> >> > >>>>> I saw this error in dmesg: >> > >>>>> >> > >>>>> [848550.356391] ixgbe 0000:00:06.0: Enabling SR-IOV VFs using >> > >>>>> the max_vfs module parameter is deprecated. >> > >>>>> [848550.356398] ixgbe 0000:00:06.0: Please use the pci sysfs >> > >>>>> interface instead. Ex: >> > >>>>> [848550.356403] ixgbe 0000:00:06.0: echo '2' > >> > >>>>> /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >> > >>>>> [848705.672383] ixgbe 0000:00:06.0 (unregistered net_device): >> > >>>>> Failed to enable PCI sriov: -38 <------- COULD BE IMPORTANT >> > >>>>> >> > >>>>> And try another approach: >> > >>>>> rmmod ixgbe ixgbevf >> > >>>>> >> > >>>>> modprobe ixgbe >> > >>>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >> > >>>>> echo '2' > /sys/bus/pci/devices/0000:00:05.0/sriov_numvfs >> > >>>>> >> > >>>>> And it's not working at all: >> > >>>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs >> > >>>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: >> > >>>>> Permission denied >> > >>>>> >> > >>>>> Is it possible to use VF inside KVM VM? >> > >>>>> >> > >>>>> -- >> > >>>>> Sincerely yours, Pavel Odintsov >> > >>>> >> > >>>> >> > >>>> >> > >>>> -- >> > >>>> Sincerely yours, Pavel Odintsov >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Sincerely yours, Pavel Odintsov >> > >> >> > >> >> > >> >> > >> -- >> > >> Sincerely yours, Pavel Odintsov >> > > >> > > >> > > >> > > -- >> > > Sincerely yours, Pavel Odintsov >> > >> > >> > >> > -- >> > Sincerely yours, Pavel Odintsov >> > >> > ---------------------------------------------------------------------- >> > -------- _______________________________________________ >> > E1000-devel mailing list >> > E1000-devel@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/e1000-devel >> > To learn more about Intel® Ethernet, visit >> > http://communities.intel.com/community/wired >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> E1000-devel mailing list >> E1000-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/e1000-devel >> To learn more about Intel® Ethernet, visit >> http://communities.intel.com/community/wired -- Sincerely yours, Pavel Odintsov ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
