Re: [pfSense] substantial packet loss on em interfaces (Superserver 5015A-EHF-D525)

2015-01-16 Thread Vick Khera
On Fri, Jan 16, 2015 at 3:35 AM, Tim Jansen tim...@byte-site.de wrote:

 some SuperMicro systems (and yours as well) have an IPMI interface running
 via the 1st onboard NIC, which means IPMI shares the phys. NIC with the
 typically LAN configuration on OS level while the IPMI interface is
 configured within the Bios.


This has caused issues for me too. If you have the IPMI interface enabled,
make sure that the sharing mode for the ethernet port is suitable for your
configuration. I personally always put the LAN interface on the shared port
as that causes the fewest problems for me. I usually set the interfaces to
share mode with the IPMI.
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Re: [pfSense] substantial packet loss on em interfaces (Superserver 5015A-EHF-D525)

2015-01-16 Thread Geoff Nordli

On 15-01-16 07:34 AM, Vick Khera wrote:


On Fri, Jan 16, 2015 at 3:35 AM, Tim Jansen tim...@byte-site.de 
mailto:tim...@byte-site.de wrote:


some SuperMicro systems (and yours as well) have an IPMI interface
running via the 1st onboard NIC, which means IPMI shares the phys.
NIC with the typically LAN configuration on OS level while the
IPMI interface is configured within the Bios.


This has caused issues for me too. If you have the IPMI interface 
enabled, make sure that the sharing mode for the ethernet port is 
suitable for your configuration. I personally always put the LAN 
interface on the shared port as that causes the fewest problems for 
me. I usually set the interfaces to share mode with the IPMI.


Hi Vick.

I think you are on to something there.  The part that really confused me 
is I have two of those servers.  One was working OK and the other was 
failing miserably.  It is quite possible the working server had the IPMI 
interface on the LAN port.


The intermittent failure was enough to drive me crazy!!

thanks,

Geoff




___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Re: [pfSense] substantial packet loss on em interfaces (Superserver 5015A-EHF-D525)

2015-01-15 Thread Chris Buechler
On Thu, Jan 15, 2015 at 6:56 PM, Geoff Nordli geo...@gnaa.net wrote:
 Hi.

 We have a Superserver 5015A-EHF-D525
 (http://www.supermicro.com/products/system/1u/5015/sys-5015a-ehf-d525.cfm)
 running  pfsense 2.1.5-RELEASE (amd64) with 2GB of RAM.

 Which has this board in it:
 http://www.supermicro.com/products/motherboard/ATOM/ICH9/X7SPE-HF-D525.cfm

 In this chassis we also have a 4 port Intel NIC which shows up as igb
 interfaces.

 We were experiencing substantial packet loss when using the em interfaces,
 but since we switched over to the igb interfaces things have been good.

 I have both Hardware TCP Segmentation Offloading and Hardware Large Receive
 Offloading disabled.

 This is not a heavily used firewall.

 Anyone else experiencing packet loss on the em interfaces.  Are there any
 other settings I should look at?


My best guess with those symptoms, if it isn't a hardware problem, is
one or more of the affected NICs ending up on the same IRQ as a USB
controller or something else that's causing issues. Most of the time
that's no big deal, on occasion with certain systems with several NICs
it can cause packet loss or performance issues. If that's the case,
may find a BIOS update that fixes it, or may be able to muck with BIOS
settings to make it go away.



 It is possible it is a hardware failure, but I want to see what other
 experience is out there.

 When looking at the network interface statistics, there are zero errors.


 thanks,

 Geoff


 ___
 pfSense mailing list
 https://lists.pfsense.org/mailman/listinfo/list
 Support the project with Gold! https://pfsense.org/gold
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


Re: [pfSense] substantial packet loss on em interfaces (Superserver 5015A-EHF-D525)

2015-01-15 Thread Geoff Nordli

On 15-01-15 11:13 PM, Chris Buechler wrote:

On Thu, Jan 15, 2015 at 6:56 PM, Geoff Nordli geo...@gnaa.net wrote:

Hi.

We have a Superserver 5015A-EHF-D525
(http://www.supermicro.com/products/system/1u/5015/sys-5015a-ehf-d525.cfm)
running  pfsense 2.1.5-RELEASE (amd64) with 2GB of RAM.

Which has this board in it:
http://www.supermicro.com/products/motherboard/ATOM/ICH9/X7SPE-HF-D525.cfm

In this chassis we also have a 4 port Intel NIC which shows up as igb
interfaces.

We were experiencing substantial packet loss when using the em interfaces,
but since we switched over to the igb interfaces things have been good.

I have both Hardware TCP Segmentation Offloading and Hardware Large Receive
Offloading disabled.

This is not a heavily used firewall.

Anyone else experiencing packet loss on the em interfaces.  Are there any
other settings I should look at?


My best guess with those symptoms, if it isn't a hardware problem, is
one or more of the affected NICs ending up on the same IRQ as a USB
controller or something else that's causing issues. Most of the time
that's no big deal, on occasion with certain systems with several NICs
it can cause packet loss or performance issues. If that's the case,
may find a BIOS update that fixes it, or may be able to muck with BIOS
settings to make it go away.




thanks Chris.

Anything visible I can see on the local machine -- without going to 
bios.  Doing a vmstat -i shows the msi interrupts being used for those 
controllers.


interrupt  total   rate
irq18: ehci0 uhci5 2  0
irq19: uhci2 uhci4+  2649947  6
cpu0: timer761654301   1992
irq256: igb0:que 0   9765149 25
irq257: igb0:que 1   2747978  7
irq258: igb0:que 2   2562344  6
irq259: igb0:que 3   2387427  6
irq260: igb0:link  2  0
irq261: igb1:que 0  13670253 35
irq262: igb1:que 1   7013567 18
irq263: igb1:que 2   7502120 19
irq264: igb1:que 3   4520889 11
irq265: igb1:link  2  0
irq266: igb2:que 0  17501710 45
irq267: igb2:que 1   8228974 21
irq268: igb2:que 2   6685269 17
irq269: igb2:que 3   5603615 14
irq270: igb2:link  2  0
irq279: em1:rx 0  233940  0
irq280: em1:tx 0  215119  0
irq281: em1:link  15  0
cpu1: timer761634299   1991
cpu2: timer761634319   1991
cpu3: timer761634321   1991
Total 3137845564   8206


I can definitely try a bios update next time I am close to the machine.

Geoff



___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold