Re: Intel 10GbE SFP+ (82599) and vlan
On 15.4.2011 12:49, Reyk Floeter wrote: On Thu, Apr 14, 2011 at 04:37:31PM +, Stuart Henderson wrote: 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. well, it works fine on the 82598 (heavily tested and used in production here) but seems to be broken on the 82599. it is either a hardware bug on the 82599 or related to the fact that it uses slighlty different advanced descriptors. this should be a more accurate workaround for now (until we're able to fix it on the 82599): #if NVLAN 0 if (hw-mac.type == ixgbe_mac_82598EB) ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 - ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD hello, i have found this datasheet: http://download.intel.com/design/network/datashts/82599_datasheet.pdf maybe it's worth something maybe not ... 7.4.3.1 Adding 802.1q Tags on Transmits Software might instruct the 82599 to insert an 802.1q VLAN tag on a per-packet basis. If the VLE bit in the transmit descriptor is set to 1b, then the 82599 inserts a VLAN tag into the packet that it transmits over the wire. The Tag Protocol Identifier b TPID (VLAN Ether Type) field of the 802.1q tag comes from the DMATXCTL.VT, and the Tag Control Information (TCI) of the 802.1q tag comes from the VLAN field of the legacy transmit descriptor or the VLAN Tag field of the advanced data transmit descriptor. 7.4.3.2 Stripping 802.1q Tags on Receives Software might instruct the 82599 to strip 802.1q VLAN tags from received packets. The policy whether to strip the VLAN tag is configurable per queue. If the RXDCTL.VME bit for a given queue is set to 1b, and the incoming packet is an 802.1q VLAN packet (that is, its Ethernet Type field matched the VLNCTRL.VET), then the 82599 strips the 4-byte VLAN tag from the packet, and stores the TCI in the VLAN Tag field of the receive descriptor. The 82599 also sets the VP bit in the receive descriptor to indicate that the packet had a VLAN tag that was stripped. If the RXDCTL.VME bit is not set, the 802.1q packets can still be received if they pass the receive filter, but the VLAN tag is not stripped and the VP bit is not set. -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 15.4.2011 12:49, Reyk Floeter wrote: On Thu, Apr 14, 2011 at 04:37:31PM +, Stuart Henderson wrote: 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. well, it works fine on the 82598 (heavily tested and used in production here) but seems to be broken on the 82599. it is either a hardware bug on the 82599 or related to the fact that it uses slighlty different advanced descriptors. this should be a more accurate workaround for now (until we're able to fix it on the 82599): #if NVLAN 0 if (hw-mac.type == ixgbe_mac_82598EB) ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif hello, here is tcpdump from directly connected ix interfaces on rtr1 and rtr2 with applied patch, it's still the same rtr1 - 10.4.4.1 # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 22:04:57.558393 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:04:58.565373 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:04:59.575372 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:05:00.585378 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:05:01.595384 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:05:09.367744 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:10.370152 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:11.380153 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:12.390167 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:13.400168 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 rtr2 - 10.4.4.2 # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 22:04:57.557884 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:04:58.564864 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:04:59.574863 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:05:00.584868 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:05:01.594875 802.1Q vid 123 pri 0 arp who-has 10.4.4.2 tell 10.4.4.1 22:05:09.367257 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:10.369665 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:11.379665 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:12.389679 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 22:05:13.399681 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.4.4.1 tell 10.4.4.2 without vlans on ix interface ip4 and ip6 is working # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB tcpdump: WARNING: compensating for unaligned libpcap packets 22:15:26.094897 fe80::21b:21ff:fe9e:6ea1 fe80::21b:21ff:fe9e:6c99: icmp6: echo request 22:15:26.095030 fe80::21b:21ff:fe9e:6c99 fe80::21b:21ff:fe9e:6ea1: icmp6: echo reply 22:15:26.128965 fe80::21b:21ff:fe9e:6c99 fe80::21b:21ff:fe9e:6ea1: icmp6: echo request 22:15:26.128982 fe80::21b:21ff:fe9e:6ea1 fe80::21b:21ff:fe9e:6c99: icmp6: echo reply tcpdump: WARNING: compensating for unaligned libpcap packets 22:15:26.168977 10.5.5.1 10.5.5.2: icmp: echo request 22:15:26.168988 10.5.5.2 10.5.5.1: icmp: echo reply 22:15:27.094904 fe80::21b:21ff:fe9e:6ea1 fe80::21b:21ff:fe9e:6c99: icmp6: echo request 22:15:27.095038 fe80::21b:21ff:fe9e:6c99 fe80::21b:21ff:fe9e:6ea1: icmp6: echo reply 22:15:27.128972 fe80::21b:21ff:fe9e:6c99 fe80::21b:21ff:fe9e:6ea1: icmp6: echo request 22:15:27.128988 fe80::21b:21ff:fe9e:6ea1 fe80::21b:21ff:fe9e:6c99: icmp6: echo reply 22:15:27.178969 10.5.5.1 10.5.5.2: icmp: echo request 22:15:27.178980 10.5.5.2 10.5.5.1: icmp: echo reply tcpbench (mtu 9014) # tcpbench 10.5.5.2 elapsed_ms bytes mbps bwidth 1000 153454782 1227.638 100.00% Conn: 1 Mbps: 1227.638 Peak Mbps: 1227.638 Avg Mbps: 1227.638 2000 240513194 1924.106 100.00% Conn: 1 Mbps: 1924.106 Peak Mbps: 1924.106 Avg Mbps: 1924.106 3000 291981960 2335.856 100.00% Conn: 1 Mbps: 2335.856 Peak Mbps: 2335.856 Avg Mbps: 2335.856 4000 322381064 2579.049 100.00% Conn: 1 Mbps: 2579.049 Peak Mbps: 2579.049 Avg Mbps: 2579.049 5001 430417974 3443.344 100.00% Conn: 1 Mbps: 3443.344 Peak Mbps: 3443.344 Avg Mbps: 3443.344 6001 441683208 3533.466 100.00% Conn: 1 Mbps: 3533.466 Peak Mbps: 3533.466 Avg Mbps: 3533.466 7001 440813894 3526.511 100.00% Conn: 1 Mbps: 3526.511 Peak Mbps: 3533.466 Avg Mbps: 3526.511 8001 440753406 3526.027 100.00% Conn: 1 Mbps: 3526.027 Peak Mbps: 3533.466 Avg Mbps: 3526.027 9002 440598806 3524.790
Re: Intel 10GbE SFP+ (82599) and vlan
On Thu, Apr 14, 2011 at 04:37:31PM +, Stuart Henderson wrote: 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. well, it works fine on the 82598 (heavily tested and used in production here) but seems to be broken on the 82599. it is either a hardware bug on the 82599 or related to the fact that it uses slighlty different advanced descriptors. this should be a more accurate workaround for now (until we're able to fix it on the 82599): #if NVLAN 0 if (hw-mac.type == ixgbe_mac_82598EB) ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 - ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD
Re: Intel 10GbE SFP+ (82599) and vlan
On 15.4.2011 12:49, Reyk Floeter wrote: On Thu, Apr 14, 2011 at 04:37:31PM +, Stuart Henderson wrote: 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. well, it works fine on the 82598 (heavily tested and used in production here) but seems to be broken on the 82599. it is either a hardware bug on the 82599 or related to the fact that it uses slighlty different advanced descriptors. this should be a more accurate workaround for now (until we're able to fix it on the 82599): #if NVLAN 0 if (hw-mac.type == ixgbe_mac_82598EB) ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif hello, i tried this patch and it's the same thing tcpdump on ix0 - interface connected to switch # tcpdump -i ix0 tcpdump: listening on ix0, link-type EN10MB 15:38:30.408371 802.1Q vid 0 pri 0 802.1Q vid 5 pri 0 SSTP RSTP config flags=7cLEARNING,FORWARDING,AGREED role=DESIGNATED root=1005.8:17:f4:24:af:0 rootcost=2000 bridge=2005.8:17:f4:24:b9:0 port=17 ifcost=128 age=1/0 max=20/0 hello=2/0 fwdelay=15/0 15:38:30.408847 802.1Q vid 0 pri 0 802.1Q vid 10 pri 0 SSTP RSTP config flags=7cLEARNING,FORWARDING,AGREED role=DESIGNATED root=100a.8:17:f4:24:af:0 rootcost=2000 bridge=200a.8:17:f4:24:b9:0 port=17 ifcost=128 age=1/0 max=20/0 hello=2/0 fwdelay=15/0 15:38:30.409304 802.1Q vid 0 pri 0 802.1Q vid 11 pri 0 SSTP RSTP config flags=7cLEARNING,FORWARDING,AGREED role=DESIGNATED root=100b.8:17:f4:24:af:0 rootcost=2000 bridge=200b.8:17:f4:24:b9:0 port=17 ifcost=128 age=1/0 max=20/0 hello=2/0 fwdelay=15/0 15:38:30.409776 802.1Q vid 0 pri 0 802.1Q vid 12 pri 0 SSTP RSTP config flags=7cLEARNING,FORWARDING,AGREED role=DESIGNATED root=100c.8:17:f4:24:af:0 rootcost=2000 bridge=200c.8:17:f4:24:b9:0 port=17 ifcost=128 age=1/0 max=20/0 hello=2/0 fwdelay=15/0 15:38:30.410235 802.1Q vid 0 pri 0 802.1Q vid 13 pri 0 SSTP RSTP config flags=7cLEARNING,FORWARDING,AGREED role=DESIGNATED root=100d.8:17:f4:24:af:0 rootcost=2000 bridge=200d.8:17:f4:24:b9:0 port=17 ifcost=128 age=1/0 max=20/0 hello=2/0 fwdelay=15/0 15:38:30.410710 802.1Q vid 0 pri 0 802.1Q vid 14 pri 0 SSTP RSTP config flags=7cLEARNING,FORWARDING,AGREED role=DESIGNATED root=100e.8:17:f4:24:af:0 rootcost=2000 bridge=200e.8:17:f4:24:b9:0 port=17 ifcost=128 age=1/0 max=20/0 hello=2/0 fwdelay=15/0 every inbound traffic is tagged with vlan0, outbound traffic is not tagged .. will post more tcpdumps later ... OpenBSD 4.9-current (GENERIC) #0: Fri Apr 15 14:52:58 CEST 2011 r...@rtr1.she.lan:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 6428266496 (6130MB) avail mem = 6243127296 (5953MB) mainbus0 at root thank you .. -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 14.4.2011 2:03, Kapetanakis Giannis wrote: Try to do ifconfig ix1 up (up in /etc/hostname.ix1) I've seen vlans not coming up until I do this on parent interface, although they appear active in ifconfig. Giannis hi, tried with hostname.ix1 and hostname.vlan123 but it's not working either # cat hostname.ix1 up # cat hostname.vlan123 vlan 123 vlandev ix1 inet 10.6.6.1 255.255.255.0 NONE # ping 10.6.6.2 PING 10.6.6.2 (10.6.6.2): 56 data bytes ping: sendto: Host is down ping: wrote 10.6.6.2 64 chars, ret=-1 ping: sendto: Host is down ping: wrote 10.6.6.2 64 chars, ret=-1 --- 10.6.6.2 ping statistics --- 2 packets transmitted, 0 packets received, 100.0% packet loss tcpdump is still the same # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 10:39:34.520816 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:35.530822 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:56.740949 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:57.750952 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:58.760958 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 will try to find details of ix card ... -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 14.4.2011 10:47, Hrvoje Popovski wrote: On 14.4.2011 2:03, Kapetanakis Giannis wrote: Try to do ifconfig ix1 up (up in /etc/hostname.ix1) I've seen vlans not coming up until I do this on parent interface, although they appear active in ifconfig. Giannis hi, tried with hostname.ix1 and hostname.vlan123 but it's not working either # cat hostname.ix1 up # cat hostname.vlan123 vlan 123 vlandev ix1 inet 10.6.6.1 255.255.255.0 NONE # ping 10.6.6.2 PING 10.6.6.2 (10.6.6.2): 56 data bytes ping: sendto: Host is down ping: wrote 10.6.6.2 64 chars, ret=-1 ping: sendto: Host is down ping: wrote 10.6.6.2 64 chars, ret=-1 --- 10.6.6.2 ping statistics --- 2 packets transmitted, 0 packets received, 100.0% packet loss tcpdump is still the same # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 10:39:34.520816 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:35.530822 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:56.740949 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:57.750952 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 10:39:58.760958 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.6.6.2 tell 10.6.6.1 will try to find details of ix card ... name: Intel Ethernet Server Adapter X520-SR2 (X520-2) Intel Part Number E10G42BFSR http://www.intel.com/support/network/sb/cs-012904.htm Board ID:E68785-xxx VendorID:8086 DevID:10FB part number from intel driver properties (i don't know what is that part number) E81283-003 -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 14/04/11 02:43, Hrvoje Popovski wrote: hello eveyone, problem is that when i enable vlan on ix interface i can't ping other side. servers are identical and cross connected with twinax SFP+ cable. tried thru switches with other ix interface but same result. card is dual 10GbE intel SFP+ SR, but i'm not sure is it X520-D2 or X520-SR2 or something else ... will try boot something else to see, server are not near me ... whitout vlans network connectivity is ok. I'm not sure what you're trying to do, but a) are you sure you have enabled dot1q vlan tagging on the switch port and vlan is in the list of allowed VLANs? b) If you are not using switch (you say something about cross connected) why do you need vlan(4) separation?? In this case you might need to define null vlan 0 and maybe vlan 1 as well to make it behave like a switch trunk port. Never done b) on systems Giannis [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
Re: Intel 10GbE SFP+ (82599) and vlan
On 2011-04-13, Hrvoje Popovski hrv...@srce.hr wrote: problem is that when i enable vlan on ix interface i can't ping other side. 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 - ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD
Re: Intel 10GbE SFP+ (82599) and vlan
On 14.4.2011 18:37, Stuart Henderson wrote: On 2011-04-13, Hrvoje Popovskihrv...@srce.hr wrote: problem is that when i enable vlan on ix interface i can't ping other side. 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 - ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD hello, i have commented that line and compiled kernel as you said and same thing happens is there any other info that i can provide thank you for diff ... -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 14.4.2011 18:18, Kapetanakis Giannis wrote: I'm not sure what you're trying to do, but i'm trying to understand why vlans aren't working on ix interface i need vlans on ix0 (interface connected to switch) for lan's and ix1 (interface directly connected to other openbsd server) for ospf other two bnx interfaces would be for pfsync and ptp link to isp a) are you sure you have enabled dot1q vlan tagging on the switch port and vlan is in the list of allowed VLANs? yap i have and for that reason just to avoid switching problems, arp cache and stuff like that i enabled vlans on directly connected ix1 interface, just for testing purposes on bnx interfaces everything is working ok b) If you are not using switch (you say something about cross connected) why do you need vlan(4) separation?? In this case you might need to define null vlan 0 and maybe vlan 1 as well to make it behave like a switch trunk port. tried with vlan0 but that i get this: vlan0 with ip address, pinging thru vlan0 # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 20:51:32.240914 802.1Q vid 0 pri 0 802.1Q vid 0 pri 0 arp who-has 10.8.8.2 tell 10.8.8.1 20:51:33.250920 802.1Q vid 0 pri 0 802.1Q vid 0 pri 0 arp who-has 10.8.8.2 tell 10.8.8.1 :) tcpdump on vlan0 without ip address, pinging thru vlan123 # tcpdump -i vlan0 tcpdump: listening on vlan0, link-type EN10MB 20:57:32.022784 802.1Q vid 123 pri 0 arp who-has 10.5.5.2 tell 10.5.5.1 20:57:33.032790 802.1Q vid 123 pri 0 arp who-has 10.5.5.2 tell 10.5.5.1 interesting thing is that i can't see any icmp traffic only arp -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 14.4.2011 20:28, Hrvoje Popovski wrote: On 14.4.2011 18:37, Stuart Henderson wrote: On 2011-04-13, Hrvoje Popovskihrv...@srce.hr wrote: problem is that when i enable vlan on ix interface i can't ping other side. 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 - ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD hello, i have commented that line and compiled kernel as you said and same thing happens is there any other info that i can provide thank you for diff ... maybe this thread will help .. http://www.mail-archive.com/freebsd-net@freebsd.org/msg34997.html -- /hrvoje
Re: Intel 10GbE SFP+ (82599) and vlan
On 2011-04-14, Hrvoje Popovski hrv...@srce.hr wrote: On 14.4.2011 18:37, Stuart Henderson wrote: On 2011-04-13, Hrvoje Popovskihrv...@srce.hr wrote: problem is that when i enable vlan on ix interface i can't ping other side. 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 -ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD hello, i have commented that line and compiled kernel as you said and same thing happens is there any other info that i can provide thank you for diff ... Did you try it on the transmitting machine or the receiving machine (or both)? http://lists.freebsd.org/pipermail/freebsd-net/2010-November/027114.html suggests that disabling this should help.
Re: Intel 10GbE SFP+ (82599) and vlan
On 14.4.2011 23:34, Stuart Henderson wrote: On 2011-04-14, Hrvoje Popovskihrv...@srce.hr wrote: On 14.4.2011 18:37, Stuart Henderson wrote: On 2011-04-13, Hrvoje Popovskihrv...@srce.hr wrote: problem is that when i enable vlan on ix interface i can't ping other side. 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 your config is OK, something is broken there. I guess this will make it function but it's not a correct fix. Index: if_ix.c === RCS file: /cvs/src/sys/dev/pci/if_ix.c,v retrieving revision 1.50 diff -u -p -r1.50 if_ix.c --- if_ix.c 13 Apr 2011 00:14:18 - 1.50 +++ if_ix.c 14 Apr 2011 16:36:58 - @@ -1453,7 +1453,7 @@ ixgbe_setup_interface(struct ix_softc *s ifp-if_capabilities = IFCAP_VLAN_MTU; #if NVLAN 0 - ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; +// ifp-if_capabilities |= IFCAP_VLAN_HWTAGGING; #endif #ifdef IX_CSUM_OFFLOAD hello, i have commented that line and compiled kernel as you said and same thing happens is there any other info that i can provide thank you for diff ... Did you try it on the transmitting machine or the receiving machine (or both)? i tried diff on both machines and vlans on ix0 (interface connected to switch) and ix1 (directly connected to other machine) will double check everything maybe i'm doing something fundamentally wrong with vlans... http://lists.freebsd.org/pipermail/freebsd-net/2010-November/027114.html suggests that disabling this should help. i just read that but need more time to understand :) -- /hrvoje
Intel 10GbE SFP+ (82599) and vlan
hello eveyone, problem is that when i enable vlan on ix interface i can't ping other side. servers are identical and cross connected with twinax SFP+ cable. tried thru switches with other ix interface but same result. card is dual 10GbE intel SFP+ SR, but i'm not sure is it X520-D2 or X520-SR2 or something else ... will try boot something else to see, server are not near me ... whitout vlans network connectivity is ok. ix1 = directly connected intrface on both machines ... rtr1: # ifconfig ix1 10.2.2.1/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6c:99 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet 10.2.2.1 netmask 0xff00 broadcast 10.2.2.255 inet6 fe80::21b:21ff:fe9e:6c99%ix1 prefixlen 64 scopeid 0x2 rtr2: # ifconfig ix1 10.2.2.2/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6e:a1 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet 10.2.2.2 netmask 0xff00 broadcast 10.2.2.255 inet6 fe80::21b:21ff:fe9e:6ea1%ix1 prefixlen 64 scopeid 0x2 rtr1: # ping 10.2.2.2 PING 10.2.2.2 (10.2.2.2): 56 data bytes 64 bytes from 10.2.2.2: icmp_seq=0 ttl=255 time=0.169 ms 64 bytes from 10.2.2.2: icmp_seq=1 ttl=255 time=0.157 ms rtr1: # ifconfig vlan123 vlan 123 vlandev ix1 10.3.3.1/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6c:99 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet6 fe80::21b:21ff:fe9e:6c99%ix1 prefixlen 64 scopeid 0x2 # ifconfig vlan123 vlan123: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6c:99 priority: 0 vlan: 123 priority: 0 parent interface: ix1 groups: vlan status: active inet6 fe80::21b:21ff:fe9e:6c99%vlan123 prefixlen 64 scopeid 0x8 inet 10.3.3.1 netmask 0xff00 broadcast 10.3.3.255 rtr2: # ifconfig vlan123 vlan 123 vlandev ix1 10.3.3.2/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6e:a1 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet6 fe80::21b:21ff:fe9e:6ea1%ix1 prefixlen 64 scopeid 0x2 # ifconfig vlan123 vlan123: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6e:a1 priority: 0 vlan: 123 priority: 0 parent interface: ix1 groups: vlan status: active inet6 fe80::21b:21ff:fe9e:6ea1%vlan123 prefixlen 64 scopeid 0x8 inet 10.3.3.2 netmask 0xff00 broadcast 10.3.3.255 rtr1: # ping 10.3.3.2 PING 10.3.3.2 (10.3.3.2): 56 data bytes ping: sendto: Host is down ping: wrote 10.3.3.2 64 chars, ret=-1 ping: sendto: Host is down ping: wrote 10.3.3.2 64 chars, ret=-1 ping: sendto: Host is down rtr2: # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:39.566708 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:40.576713 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:41.586718 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:42.596727 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:21:03.806848 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 i'm not sure but it's seems that this is some kind of qnq ? can i somehow strip vlan 0 and make things work ? :) on bnx interface with vlans, tcpdump is little different :) # tcpdump -i bnx1 tcpdump: listening on bnx1, link-type EN10MB 01:28:53.869575 802.1Q vid 111 pri 0 10.5.5.1 10.5.5.2: icmp: echo request 01:28:53.869589 802.1Q vid 111 pri 0 10.5.5.2 10.5.5.1: icmp: echo reply 01:28:54.879580 802.1Q vid 111 pri 0 10.5.5.1 10.5.5.2: icmp: echo request 01:28:54.879593 802.1Q vid 111 pri 0 10.5.5.2 10.5.5.1: icmp: echo reply Thank you OpenBSD 4.9-current (GENERIC.MP) #35: Wed Apr 13 01:09:23 MDT 2011 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 6428266496 (6130MB) avail mem = 6243102720 (5953MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (78 entries) bios0: vendor Dell Inc. version 1.5.3 date 10/22/2010 bios0: Dell Inc. PowerEdge R410 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 32 (boot processor) cpu0: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.37 MHz cpu0:
Re: Intel 10GbE SFP+ (82599) and vlan
Try to do ifconfig ix1 up (up in /etc/hostname.ix1) I've seen vlans not coming up until I do this on parent interface, although they appear active in ifconfig. Giannis On 14/04/11 02:43, Hrvoje Popovski wrote: hello eveyone, problem is that when i enable vlan on ix interface i can't ping other side. servers are identical and cross connected with twinax SFP+ cable. tried thru switches with other ix interface but same result. card is dual 10GbE intel SFP+ SR, but i'm not sure is it X520-D2 or X520-SR2 or something else ... will try boot something else to see, server are not near me ... whitout vlans network connectivity is ok. ix1 = directly connected intrface on both machines ... rtr1: # ifconfig ix1 10.2.2.1/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6c:99 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet 10.2.2.1 netmask 0xff00 broadcast 10.2.2.255 inet6 fe80::21b:21ff:fe9e:6c99%ix1 prefixlen 64 scopeid 0x2 rtr2: # ifconfig ix1 10.2.2.2/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6e:a1 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet 10.2.2.2 netmask 0xff00 broadcast 10.2.2.255 inet6 fe80::21b:21ff:fe9e:6ea1%ix1 prefixlen 64 scopeid 0x2 rtr1: # ping 10.2.2.2 PING 10.2.2.2 (10.2.2.2): 56 data bytes 64 bytes from 10.2.2.2: icmp_seq=0 ttl=255 time=0.169 ms 64 bytes from 10.2.2.2: icmp_seq=1 ttl=255 time=0.157 ms rtr1: # ifconfig vlan123 vlan 123 vlandev ix1 10.3.3.1/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6c:99 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet6 fe80::21b:21ff:fe9e:6c99%ix1 prefixlen 64 scopeid 0x2 # ifconfig vlan123 vlan123: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6c:99 priority: 0 vlan: 123 priority: 0 parent interface: ix1 groups: vlan status: active inet6 fe80::21b:21ff:fe9e:6c99%vlan123 prefixlen 64 scopeid 0x8 inet 10.3.3.1 netmask 0xff00 broadcast 10.3.3.255 rtr2: # ifconfig vlan123 vlan 123 vlandev ix1 10.3.3.2/24 # ifconfig ix1 ix1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6e:a1 priority: 0 media: Ethernet autoselect (10GbaseSR full-duplex) status: active inet6 fe80::21b:21ff:fe9e:6ea1%ix1 prefixlen 64 scopeid 0x2 # ifconfig vlan123 vlan123: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:1b:21:9e:6e:a1 priority: 0 vlan: 123 priority: 0 parent interface: ix1 groups: vlan status: active inet6 fe80::21b:21ff:fe9e:6ea1%vlan123 prefixlen 64 scopeid 0x8 inet 10.3.3.2 netmask 0xff00 broadcast 10.3.3.255 rtr1: # ping 10.3.3.2 PING 10.3.3.2 (10.3.3.2): 56 data bytes ping: sendto: Host is down ping: wrote 10.3.3.2 64 chars, ret=-1 ping: sendto: Host is down ping: wrote 10.3.3.2 64 chars, ret=-1 ping: sendto: Host is down rtr2: # tcpdump -i ix1 tcpdump: listening on ix1, link-type EN10MB 01:20:38.556705 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:39.566708 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:40.576713 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:41.586718 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:20:42.596727 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 01:21:03.806848 802.1Q vid 0 pri 0 802.1Q vid 123 pri 0 arp who-has 10.3.3.2 tell 10.3.3.1 i'm not sure but it's seems that this is some kind of qnq ? can i somehow strip vlan 0 and make things work ? :) on bnx interface with vlans, tcpdump is little different :) # tcpdump -i bnx1 tcpdump: listening on bnx1, link-type EN10MB 01:28:53.869575 802.1Q vid 111 pri 0 10.5.5.1 10.5.5.2: icmp: echo request 01:28:53.869589 802.1Q vid 111 pri 0 10.5.5.2 10.5.5.1: icmp: echo reply 01:28:54.879580 802.1Q vid 111 pri 0 10.5.5.1 10.5.5.2: icmp: echo request 01:28:54.879593 802.1Q vid 111 pri 0 10.5.5.2 10.5.5.1: icmp: echo reply Thank you OpenBSD 4.9-current (GENERIC.MP) #35: Wed Apr 13 01:09:23 MDT 2011 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 6428266496 (6130MB) avail mem = 6243102720 (5953MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (78 entries) bios0: vendor Dell Inc. version 1.5.3 date 10/22/2010 bios0: Dell Inc. PowerEdge R410 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup
