At 07:29 PM 11/18/2009, Jack Vogel wrote:
Hey Mike,
Can you check if you see the same behavior on RELENG 8?
Hi Jack,
Yes, I will reboot the hardware with a RELENG_8 image tomorrow to test
Not sure why this happens on Hartwell (82574) and not on 82571, that's
an interesting bit, the 82574 is the ONLY interface in the em driver that
has MSIX support, unfortunately its kinda hacked in, but it did not really
fit into the igb driver either for various technical reasons.
Is this the "FILTER" vs "ITHREAD" ? Is there a way to force this
chipset to use the same logic as 82571s ?
# dmesg |grep ^em
em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xbc00-0xbc1f
mem 0xface0000-0xfacfffff,0xfacc0000-0xfacdffff irq 16 at device 0.0 on pci1
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:15:17:78:e6:e0
em1: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xb880-0xb89f
mem 0xfac80000-0xfac9ffff,0xfac60000-0xfac7ffff irq 17 at device 0.1 on pci1
em1: Using MSI interrupt
em1: [FILTER]
em1: Ethernet address: 00:15:17:78:e6:e1
em2: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xcc00-0xcc1f
mem 0xfade0000-0xfadfffff,0xfadc0000-0xfaddffff irq 16 at device 0.0 on pci3
em2: Using MSI interrupt
em2: [FILTER]
em2: Ethernet address: 00:15:17:cf:26:de
em3: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xc880-0xc89f
mem 0xfad80000-0xfad9ffff,0xfad60000-0xfad7ffff irq 17 at device 0.1 on pci3
em3: Using MSI interrupt
em3: [FILTER]
em3: Ethernet address: 00:15:17:cf:26:df
em4: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xdc00-0xdc1f
mem 0xfaee0000-0xfaefffff,0xfaedc000-0xfaedffff irq 16 at device 0.0 on pci6
em4: Using MSIX interrupts
em4: [ITHREAD]
em4: [ITHREAD]
em4: [ITHREAD]
em4: Ethernet address: 00:30:48:d6:ef:12
em5: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f
mem 0xfafe0000-0xfaffffff,0xfafdc000-0xfafdffff irq 17 at device 0.0 on pci7
em5: Using MSIX interrupts
em5: [ITHREAD]
em5: [ITHREAD]
em5: [ITHREAD]
em5: Ethernet address: 00:30:48:d6:ef:13
What if you boot up, then do NOT ping or anything until the interface is
assigned an address (and so init is run), and the cable is plugged in. If
that happens first does it work?
yes. If I have the cables plugged in and reboot the box, its ok. I
am pretty sure all is ok if I boot it up, with no address assigned,
plug the cables in, and then assign addr.
I havent tested it out yet, but not sure how things play out when
the ports are connected to a switch that is not in portfast mode, so
the carrier does not always come up right away. The other thing I saw
was that the NIC was getting stuck with the carrier showing up, even
though cable was unplugged. However, I was not able to find the
exact conditions this happened.
Do let me know if you can check on 8, if not I can have my validation
engineer try this.
I will report back tomorrow.
---Mike
Best regards,
Jack
On Wed, Nov 18, 2009 at 1:30 PM, Mike Tancsa
<<mailto:[email protected]>[email protected]> wrote:
On two Intel chipset Supermicro boards (X8STi and X8STE-0) using the
onboard em nics (dmesg info below), I seem to have run into an issue
where if I boot the box up with the cables unplugged, I cannot get
the NICS to properly work post boot up. This is quite repeatable
for me. So at boot time, I have
# ifconfig em5
em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
ether 00:30:48:d6:ef:13
inet 3.3.3.3 netmask 0xffffff00 broadcast 3.3.3.255
media: Ethernet autoselect
status: no carrier
I then ping something that would be across the wire while the nic is
down. e.g. ping 3.3.3.1
I then plug in the cable so that the other side has 3.3.3.1
ifconfig shows all looks good
# ifconfig em5
em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
ether 00:30:48:d6:ef:13
inet 3.3.3.3 netmask 0xffffff00 broadcast 3.3.3.255
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
I try and ping 3.3.3.1 which is on xover (via a switch shows the
same behaviour), and no response to the pings.... BUT, I do see the
MAC addr show up
# ping -c 2 -S 3.3.3.3 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.3>3.3.3.3: 56 data bytes
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
# arp -na
? (3.3.3.1) at 00:30:48:94:88:20 on em5 [ethernet]
? (3.3.3.3) at 00:30:48:d6:ef:13 on em5 permanent [ethernet]
I can see its mac addr ?!?
Furthermore, if I do
# ifconfig em5 <http://3.3.3.55/32>3.3.3.55/32 alias
On the other side, I see
0(ich10)# tcpdump -nei igb0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), capture size 96 bytes
16:16:03.380886 00:30:48:d6:ef:13 > ff:ff:ff:ff:ff:ff, ethertype ARP
(0x0806), length 60: Request who-has 3.3.3.55 tell 3.3.3.55, length 46
and I can ping if I specify the alias as the source IP
# ping -S 3.3.3.55 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.55>3.3.3.55: 56 data bytes
64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=0 ttl=64 time=0.184 ms
64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=2 ttl=64 time=0.055 ms
16:17:01.603345 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype ARP
(0x0806), length 60: Reply 3.3.3.55 is-at 00:30:48:d6:ef:13, length 46
16:17:01.603349 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype
IPv4 (0x0800), length 98: 3.3.3.1 > <http://3.3.3.55>3.3.3.55: ICMP
echo reply, id 7946, seq 0, length 64
16:17:02.603497 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype
IPv4 (0x0800), length 98: 3.3.3.55 > <http://3.3.3.1>3.3.3.1: ICMP
echo request, id 7946, seq 1, length 64
16:17:02.603502 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype
IPv4 (0x0800), length 98: 3.3.3.1 > <http://3.3.3.55>3.3.3.55: ICMP
echo reply, id 7946, seq 1, length 64
16:17:03.604510 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype
IPv4 (0x0800), length 98: 3.3.3.55 > <http://3.3.3.1>3.3.3.1: ICMP
echo request, id 7946, seq 2, length 64
16:17:03.604516 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype
IPv4 (0x0800), length 98: 3.3.3.1 > <http://3.3.3.55>3.3.3.55: ICMP
echo reply, id 7946, seq 2, length 64
but not using the initial IP addr
0[iolite3A]# ping -S 3.3.3.3 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.3>3.3.3.3: 56 data bytes
^C
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
#
Yet,
# ping -S 3.3.3.3 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.3>3.3.3.3: 56 data bytes
^C
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
# ping -S 3.3.3.4 3.3.3.1
PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.4>3.3.3.4: 56 data bytes
64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=0 ttl=64 time=0.176 ms
64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=1 ttl=64 time=0.050 ms
^C
--- 3.3.3.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.050/0.113/0.176/0.063 ms
Strange, eh ?
e...@pci0:6:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086
rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
e...@pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086
rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
em4: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xdc00-0xdc1f
mem 0xfaee0000-0xfaefffff,0xfaedc000-0xfaedffff irq 16 at device 0.0 on pci6
em4: Using MSIX interrupts
em4: [ITHREAD]
em4: [ITHREAD]
em4: [ITHREAD]
em4: Ethernet address: 00:30:48:d6:ef:12
pcib7: <ACPI PCI-PCI bridge> irq 16 at device 28.1 on pci0
pci7: <ACPI PCI bus> on pcib7
em5: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f
mem 0xfafe0000-0xfaffffff,0xfafdc000-0xfafdffff irq 17 at device 0.0 on pci7
em5: Using MSIX interrupts
em5: [ITHREAD]
em5: [ITHREAD]
em5: [ITHREAD]
em5: Ethernet address: 00:30:48:d6:ef:13
The same does NOT happen with my external 2 port pcie nics (it says
HP, but they are intel branded)
eg
e...@pci0:1:0:0: class=0x020000 card=0x115e8086 chip=0x105e8086
rev=0x06 hdr=0x00
vendor = 'Intel Corporation'
device = 'HP NC360T PCIe DP Gigabit Server Adapter (n1e5132)'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x4(x4)
e...@pci0:1:0:1: class=0x020000 card=0x115e8086 chip=0x105e8086
rev=0x06 hdr=0x00
vendor = 'Intel Corporation'
device = 'HP NC360T PCIe DP Gigabit Server Adapter (n1e5132)'
class = network
subclass = ethernet
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x4(x4)
---Mike
--------------------------------------------------------------------
Mike Tancsa, tel +1 519 651 3400
Sentex
Communications,
<mailto:[email protected]>[email protected]
Providing Internet since
1994 <http://www.sentex.net>www.sentex.net
Cambridge, Ontario
Canada <http://www.sentex.net/mike>www.sentex.net/mike
--------------------------------------------------------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, [email protected]
Providing Internet since 1994 www.sentex.net
Cambridge, Ontario Canada www.sentex.net/mike
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[email protected]"
