Re: bhyve, ixgbe and debian 8

[Rajil Saraswat]

On 11/09/2016 01:38 PM, Peter Grehan wrote:
> Hi Rajil,
>
>> Is this specific to hardware or linux distribution ?
>
>  Possibly both.
>
>  The PCI device reset change that I mentioned earlier is
> https://svnweb.freebsd.org/base?view=revision=306520
>
>  If you upgrade to 11-stable you should be able to pick it up and see
> if it helps.
>
> later,
>
> Peter.
>

I tried the 'HP NC360T 2-Port Gigabit PCIe Adapter Card' which is based
on the em driver. This card survives the reboot in debian vm on FreeBSD
11-Release.

Cheers,
Rajil
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve, ixgbe and debian 8

[Peter Grehan]

Hi Rajil,


Is this specific to hardware or linux distribution ?


 Possibly both.

 The PCI device reset change that I mentioned earlier is
https://svnweb.freebsd.org/base?view=revision=306520

 If you upgrade to 11-stable you should be able to pick it up and see 
if it helps.


later,

Peter.

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve, ixgbe and debian 8

[Rajil Saraswat]

On Nov 7, 2016 2:01 PM, "Peter Grehan"  wrote:
>
> Hi Rajil,
>
>
>>> I tried with the second onboard NIC and it worked the first time. iperf
>>> showed 10g speeds. After restarting the VM, i get an error for both
>>> NICs.
>
>
>  What's the FreeBSD host version ? There was work done recently to reset
passthru devices pre/post running a guest which hopefully should have fixed
this.
>
> later,
>
> Peter.
>

Peter,
Is this specific to hardware or linux distribution ?
Thanks
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve, ixgbe and debian 8

[Rajil Saraswat]

On Nov 7, 2016 2:01 PM, "Peter Grehan"  wrote:
>
> Hi Rajil,
>
>
>>> I tried with the second onboard NIC and it worked the first time. iperf
>>> showed 10g speeds. After restarting the VM, i get an error for both
>>> NICs.
>
>
>  What's the FreeBSD host version ? There was work done recently to reset
passthru devices pre/post running a guest which hopefully should have fixed
this.
>
> later,
>
> Peter.
>

Peter,

$ uname -a
FreeBSD box.localdomain 11.0-RELEASE-p2 FreeBSD 11.0-RELEASE-p2 #0: Mon Oct
24 06:55:27 UTC 2016
r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
amd64

Cheers,
Rajil
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve, ixgbe and debian 8

[Peter Grehan]

Hi Rajil,


I tried with the second onboard NIC and it worked the first time. iperf
showed 10g speeds. After restarting the VM, i get an error for both
NICs.


 What's the FreeBSD host version ? There was work done recently to 
reset passthru devices pre/post running a guest which hopefully should 
have fixed this.


later,

Peter.

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve, ixgbe and debian 8

[Rajil Saraswat]

On 11/07/2016 07:41 AM, Rajil Saraswat wrote:
> On 11/07/2016 07:08 AM, Rajil Saraswat wrote:
>> Hello,
>>
>> I have PCI passthrough working now for the Intel X540-AT2 NIC and is
>> seen inside debian 8 vm.
>>
>> # lspci
>>
>> 00:00.0 Host bridge: Network Appliance Corporation Device 1275
>> 00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device
>> 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
>> 00:06.0 Ethernet controller: Intel Corporation Ethernet Controller
>> 10-Gigabit X540-AT2 (rev 01)
>> 00:1f.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
>>
>>
>> However, on loading the module the kernel gives the error
>>
>> #modprobe ixgbe
>>
>> [   34.929546] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver -
>> version 3.19.1-k
>> [   34.929549] ixgbe: Copyright (c) 1999-2014 Intel Corporation.
>> [   34.929706] ixgbe :00:06.0: can't derive routing for PCI INT A
>> [   34.929708] ixgbe :00:06.0: PCI INT A: no GSI
>> [   34.930354] ixgbe :00:06.0: Adapter removed
>> [   34.931710] ixgbe :00:06.0: can't derive routing for PCI INT A
>> [   34.931717] ixgbe: probe of :00:06.0 failed with error -5
>>
>> Any idea what could be the issue?
>>
>> Thanks,
>>
>> Rajil
>>
> I tried with the second onboard NIC and it worked the first time. iperf
> showed 10g speeds. After restarting the VM, i get an error for both
> NICs. This a dual CPU motherboard Supermicro X10DriT with a pair of
> onboard NICs.
>
> [0.628758] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver -
> version 3.19.1-k
> [0.628760] ixgbe: Copyright (c) 1999-2014 Intel Corporation.
> [0.628899] ixgbe :00:06.0: can't derive routing for PCI INT A
> [0.628901] ixgbe :00:06.0: PCI INT A: no GSI
> [0.629519] ixgbe :00:06.0: Adapter removed
> [0.630143] ixgbe :00:06.0: can't derive routing for PCI INT A
> [0.630147] ixgbe: probe of :00:06.0 failed with error -5
> [0.630199] ixgbe :00:06.1: can't derive routing for PCI INT B
> [0.630200] ixgbe :00:06.1: PCI INT B: no GSI
> [0.630789] ixgbe :00:06.1: Adapter removed
> [0.639469] ixgbe :00:06.1: can't derive routing for PCI INT B
> [0.639473] ixgbe: probe of :00:06.1 failed with error -5
>
After a cold reboot of the server, the devices are again seen in debian.
I guess somehow the devices become unusable when the vm is restarted.

[1.146946] ixgbe :00:06.0: PCI Express bandwidth of 32GT/s available
[1.146949] ixgbe :00:06.0: (Speed:5.0GT/s, Width: x8, Encoding
Loss:20%)
[1.146953] ixgbe :00:06.0: This is not sufficient for optimal
performance of this card.
[1.146955] ixgbe :00:06.0: For optimal performance, at least
60GT/s of bandwidth is required.
[1.146956] ixgbe :00:06.0: A slot with more lanes and/or higher
speed is suggested.
[1.189458] ixgbe :00:06.0: MAC: 3, PHY: 3, PBA No: 030C00-000
[1.189461] ixgbe :00:06.0: 0c:c4:7a:de:cc:c0
[1.339231] ixgbe :00:06.0: Intel(R) 10 Gigabit Network Connection
[1.339364] ixgbe :00:06.1: can't derive routing for PCI INT B
[1.339366] ixgbe :00:06.1: PCI INT B: no GSI
[1.348140] input: PS/2 Generic Mouse as
/devices/platform/i8042/serio1/input/input2
[1.566630] tsc: Refined TSC clocksource calibration: 2198.264 MHz
[1.793929] ixgbe :00:06.1: irq 47 for MSI/MSI-X
[1.793950] ixgbe :00:06.1: irq 48 for MSI/MSI-X
[1.795079] ixgbe :00:06.1: Multiqueue Disabled: Rx Queue count =
1, Tx Queue count = 1
[1.859258] ixgbe :00:06.1: PCI Express bandwidth of 32GT/s available
[1.859261] ixgbe :00:06.1: (Speed:5.0GT/s, Width: x8, Encoding
Loss:20%)
[1.859262] ixgbe :00:06.1: This is not sufficient for optimal
performance of this card.
[1.859264] ixgbe :00:06.1: For optimal performance, at least
60GT/s of bandwidth is required.
[1.859265] ixgbe :00:06.1: A slot with more lanes and/or higher
speed is suggested.
[1.902549] ixgbe :00:06.1: MAC: 3, PHY: 3, PBA No: 030C00-000
[1.902554] ixgbe :00:06.1: 0c:c4:7a:de:cc:c1
[2.055631] ixgbe :00:06.1: Intel(R) 10 Gigabit Network Connection

# cat /proc/interrupts
   CPU0  
  0: 45   IO-APIC-edge  timer
  1:  9   IO-APIC-edge  i8042
  4:   3756   IO-APIC-edge  serial
  8:  0   IO-APIC-edge  rtc0
  9:  0   IO-APIC-fasteoi   acpi
 12:138   IO-APIC-edge  i8042
 40:  0   PCI-MSI-edge  virtio0-config
 41:   5667   PCI-MSI-edge  virtio0-requests
 42:  0   PCI-MSI-edge  virtio1-config
 43:139   PCI-MSI-edge  virtio1-input.0
 44:  1   PCI-MSI-edge  virtio1-output.0
 45:120   PCI-MSI-edge  eth1-TxRx-0
 46:  0   PCI-MSI-edge  eth1
 47:   1170   PCI-MSI-edge  eth2-TxRx-0
 48:  3   PCI-MSI-edge  eth2
NMI:  0   Non-maskable interrupts
LOC:   

Re: bhyve, ixgbe and debian 8

[Rajil Saraswat]

On 11/07/2016 07:08 AM, Rajil Saraswat wrote:
> Hello,
>
> I have PCI passthrough working now for the Intel X540-AT2 NIC and is
> seen inside debian 8 vm.
>
> # lspci
>
> 00:00.0 Host bridge: Network Appliance Corporation Device 1275
> 00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device
> 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
> 00:06.0 Ethernet controller: Intel Corporation Ethernet Controller
> 10-Gigabit X540-AT2 (rev 01)
> 00:1f.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
>
>
> However, on loading the module the kernel gives the error
>
> #modprobe ixgbe
>
> [   34.929546] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver -
> version 3.19.1-k
> [   34.929549] ixgbe: Copyright (c) 1999-2014 Intel Corporation.
> [   34.929706] ixgbe :00:06.0: can't derive routing for PCI INT A
> [   34.929708] ixgbe :00:06.0: PCI INT A: no GSI
> [   34.930354] ixgbe :00:06.0: Adapter removed
> [   34.931710] ixgbe :00:06.0: can't derive routing for PCI INT A
> [   34.931717] ixgbe: probe of :00:06.0 failed with error -5
>
> Any idea what could be the issue?
>
> Thanks,
>
> Rajil
>
I tried with the second onboard NIC and it worked the first time. iperf
showed 10g speeds. After restarting the VM, i get an error for both
NICs. This a dual CPU motherboard Supermicro X10DriT with a pair of
onboard NICs.

[0.628758] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver -
version 3.19.1-k
[0.628760] ixgbe: Copyright (c) 1999-2014 Intel Corporation.
[0.628899] ixgbe :00:06.0: can't derive routing for PCI INT A
[0.628901] ixgbe :00:06.0: PCI INT A: no GSI
[0.629519] ixgbe :00:06.0: Adapter removed
[0.630143] ixgbe :00:06.0: can't derive routing for PCI INT A
[0.630147] ixgbe: probe of :00:06.0 failed with error -5
[0.630199] ixgbe :00:06.1: can't derive routing for PCI INT B
[0.630200] ixgbe :00:06.1: PCI INT B: no GSI
[0.630789] ixgbe :00:06.1: Adapter removed
[0.639469] ixgbe :00:06.1: can't derive routing for PCI INT B
[0.639473] ixgbe: probe of :00:06.1 failed with error -5

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

bhyve, ixgbe and debian 8

[Rajil Saraswat]

Hello,

I have PCI passthrough working now for the Intel X540-AT2 NIC and is
seen inside debian 8 vm.

# lspci

00:00.0 Host bridge: Network Appliance Corporation Device 1275
00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device
00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
00:06.0 Ethernet controller: Intel Corporation Ethernet Controller
10-Gigabit X540-AT2 (rev 01)
00:1f.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]


However, on loading the module the kernel gives the error

#modprobe ixgbe

[   34.929546] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver -
version 3.19.1-k
[   34.929549] ixgbe: Copyright (c) 1999-2014 Intel Corporation.
[   34.929706] ixgbe :00:06.0: can't derive routing for PCI INT A
[   34.929708] ixgbe :00:06.0: PCI INT A: no GSI
[   34.930354] ixgbe :00:06.0: Adapter removed
[   34.931710] ixgbe :00:06.0: can't derive routing for PCI INT A
[   34.931717] ixgbe: probe of :00:06.0 failed with error -5

Any idea what could be the issue?

Thanks,

Rajil

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"