On Sun, May 24, 2009 at 2:52 PM, Axton <[email protected]> wrote:
> The vlan id for my em0 interface is not reading properly after upgrading to
> 4.5.
>
> Tcpdump shows some wild vid values in the traffic when using em0:
>
> * This traffic should be on vlan2 (lan)
> 00:21:70:c5:3d:4f ff:ff:ff:ff:ff:ff 8100 64: 802.1Q vid 512 pri 0 arp
> who-has 10.107.208.1 tell 10.107.208.50
>
> * This traffic should be on vlan3 (egress vlan)
> 00:1e:be:fe:f3:05 ff:ff:ff:ff:ff:ff 8100 64: 802.1Q vid 768 pri 0 arp
> who-has 98.196.101.152 tell 98.196.100.1
> 00:1e:be:fe:f3:05 ff:ff:ff:ff:ff:ff 8100 64: 802.1Q vid 768 pri 0 arp
> who-has 98.196.88.115 tell 98.196.88.1
>
> * This traffic should be on vlan4, it is correct:
> 00:02:b3:ed:68:89 01:00:5e:7f:ff:fa 8100 308: 802.1Q vid 4 pri 0
> 10.0.0.1.29275 > 239.255.255.250.1900: udp 262 [ttl 1]
> 00:02:b3:ed:68:89 01:00:5e:7f:ff:fa 8100 380: 802.1Q vid 4 pri 0
> 10.0.0.1.29275 > 239.255.255.250.1900: udp 334 [ttl 1]
>
> It seems as though the vlan id is being multiplied by 256 for vlans 2 and
3.
>
>
> When I use the gem0 interface on the same machine, things work:
>
> * This traffic should be on vlan2 (lan), it is correct:
> 00:03:ba:04:b2:1d 00:50:8d:95:39:17 8100 110: 802.1Q vid 2 pri 0
> 10.107.208.1.22 > 10.107.208.102.2692: P 920030:920082(52) ack 11189 win
> 17520 (DF) [tos 0x10]
> 00:03:ba:04:b2:1d 00:50:8d:95:39:17 8100 110: 802.1Q vid 2 pri 0
> 10.107.208.1.22 > 10.107.208.102.2692: P 920082:920134(52) ack 11189 win
> 17520 (DF) [tos 0x10]
>
> * This traffic should be on vlan3 (egress vlan), it is correct:
> 00:1e:be:fe:f3:05 ff:ff:ff:ff:ff:ff 8100 64: 802.1Q vid 3 pri 0 arp who-has
> 98.194.104.216 tell 98.194.104.1
> 00:1e:be:fe:f3:05 ff:ff:ff:ff:ff:ff 8100 64: 802.1Q vid 3 pri 0 arp who-has
> 76.31.110.47 tell 76.31.108.1
>
> * This traffic should be on vlan4, it is correct:
> 00:03:ba:04:b2:1d 01:00:5e:7f:ff:fa 8100 373: 802.1Q vid 4 pri 0
> 10.0.0.1.10117 > 239.255.255.250.1900: udp 327 [ttl 1]
> 00:03:ba:04:b2:1d 01:00:5e:7f:ff:fa 8100 373: 802.1Q vid 4 pri 0
> 10.0.0.1.10117 > 239.255.255.250.1900: udp 327 [ttl 1]
>
>
> The em0 interface worked without an issue using 4.4 as did gem0.
>
>
> Here are my interface configurations using gem0:
> # ifconfig -a
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33160
> priority: 0
> groups: lo
> inet 127.0.0.1 netmask 0xff000000
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
> gem0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu
1500
> lladdr 00:03:ba:04:b2:1d
> priority: 0
> media: Ethernet autoselect (100baseTX full-duplex)
> status: active
> inet6 fe80::203:baff:fe04:b21d%gem0 prefixlen 64 scopeid 0x1
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:02:b3:ed:68:89
> priority: 0
> media: Ethernet autoselect (none)
> status: no carrier
> inet6 fe80::202:b3ff:feed:6889%em0 prefixlen 64 scopeid 0x2
> enc0: flags=0<> mtu 1536
> priority: 0
> vlan2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:03:ba:04:b2:1d
> priority: 0
> vlan: 2 priority: 0 parent interface: gem0
> groups: vlan
> inet6 fe80::203:baff:fe04:b21d%vlan2 prefixlen 64 scopeid 0x5
> inet 10.107.208.1 netmask 0xffffff00 broadcast 10.107.208.255
> vlan3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:03:ba:04:b2:1d
> priority: 0
> vlan: 3 priority: 0 parent interface: gem0
> groups: vlan egress
> inet6 fe80::203:baff:fe04:b21d%vlan3 prefixlen 64 scopeid 0x6
> inet x.x.x.x netmask 0xfffffc00 broadcast 255.255.255.255
> vlan4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:03:ba:04:b2:1d
> priority: 0
> vlan: 4 priority: 0 parent interface: gem0
> groups: vlan
> inet6 fe80::203:baff:fe04:b21d%vlan4 prefixlen 64 scopeid 0x7
> inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
> vlan5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:03:ba:04:b2:1d
> priority: 0
> vlan: 5 priority: 0 parent interface: gem0
> groups: vlan
> inet6 fe80::203:baff:fe04:b21d%vlan5 prefixlen 64 scopeid 0x8
> inet 10.180.16.1 netmask 0xffffff00 broadcast 10.180.16.255
> pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33160
> priority: 0
> groups: pflog
>
> Here are my interface configurations using em0:
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33160
> priority: 0
> groups: lo
> inet 127.0.0.1 netmask 0xff000000
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
> gem0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu
1500
> lladdr 00:03:ba:04:b2:1d
> priority: 0
> media: Ethernet autoselect (none)
> status: no carrier
> inet6 fe80::203:baff:fe04:b21d%gem0 prefixlen 64 scopeid 0x1
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:02:b3:ed:68:89
> priority: 0
> media: Ethernet autoselect (1000baseT full-duplex,master)
> status: active
> inet6 fe80::202:b3ff:feed:6889%em0 prefixlen 64 scopeid 0x2
> enc0: flags=0<> mtu 1536
> priority: 0
> vlan2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:02:b3:ed:68:89
> priority: 0
> vlan: 2 priority: 0 parent interface: em0
> groups: vlan
> inet6 fe80::202:b3ff:feed:6889%vlan2 prefixlen 64 scopeid 0x5
> inet 10.107.208.1 netmask 0xffffff00 broadcast 10.107.208.255
> vlan3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:02:b3:ed:68:89
> priority: 0
> vlan: 3 priority: 0 parent interface: em0
> groups: vlan
> inet6 fe80::202:b3ff:feed:6889%vlan3 prefixlen 64 scopeid 0x6
> vlan4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:02:b3:ed:68:89
> priority: 0
> vlan: 4 priority: 0 parent interface: em0
> groups: vlan
> inet6 fe80::202:b3ff:feed:6889%vlan4 prefixlen 64 scopeid 0x7
> inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
> vlan5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 00:02:b3:ed:68:89
> priority: 0
> vlan: 5 priority: 0 parent interface: em0
> groups: vlan
> inet6 fe80::202:b3ff:feed:6889%vlan5 prefixlen 64 scopeid 0x8
> inet 10.180.16.1 netmask 0xffffff00 broadcast 10.180.16.255
> pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33160
> priority: 0
> groups: pflog
>
> Note that vlan2 does not have an ip because dhclient could not get one.
> dhclient could also not verify one of the leases stored in the lease db.
>
>
> I collected the information above using the following procedure:
>
> 1. configure hostname.if files to use the em0 interface
> # cat hostname.em0
> up media autoselect
> # cat hostname.gem0
> up media autoselect
> # cat hostname.vlan2
> inet 10.107.208.1 255.255.255.0 NONE vlan 2 vlandev em0
> # cat hostname.vlan3
> dhcp vlan 3 vlandev em0
> # cat hostname.vlan4
> inet 10.0.0.1 255.255.255.0 NONE vlan 4 vlandev em0
> # cat hostname.vlan5
> inet 10.180.16.1 255.255.255.0 NONE vlan 5 vlandev em0
>
> 2. Plug the em0 port on the openbsd host into the tagged port on the
switch.
>
> 3. restart the host
>
> 4. dump some packets from the em0 interface and grab the ifconfig output
>
> 5. configure hostname.if files to use the gem0 interface
> # cat hostname.em0
> up media autoselect
> # cat hostname.gem0
> up media autoselect
> # cat hostname.vlan2
> inet 10.107.208.1 255.255.255.0 NONE vlan 2 vlandev gem0
> # cat hostname.vlan3
> dhcp vlan 3 vlandev gem0
> # cat hostname.vlan4
> inet 10.0.0.1 255.255.255.0 NONE vlan 4 vlandev gem0
> # cat hostname.vlan5
> inet 10.180.16.1 255.255.255.0 NONE vlan 5 vlandev gem0
>
> 6. Plug the gem0 port on the openbsd host into the tagged port on the
> switch.
>
> 7. restart the host
>
> 8. dump some packets from the gem0 interface and grab the ifconfig output
>
>
> This machine was first built using 4.4, then upgraded to 4.5 using the
> console/boot from cdrom method. The files were upgraded using diff and
> manual updates for the files listed in the upgrade instructions; packages
> were updated from a mirror. The switch is a dell powerconnect 5324 and is
> configured with one tagged port on all vlans (2,3,4,5) and all the other
> ports assigned untagged to a single vlan.
>
>
> The packages installed on the host are minimal:
> # pkg_info
> gettext-0.17p0 GNU gettext
> igmpproxy-0.1-beta2 Multicast router utilizing IGMP forwarding
> kermit-8.0.211 serial and network communications package
> libiconv-1.12 character set conversion library
> lsof-4.81p0 list information about open files
> malo-firmware-1.4 Firmware binary images for malo driver
> rsync-3.0.5 mirroring/synchronization over low bandwidth links
> wget-1.11.4 retrieve files from the web via HTTP, HTTPS and FTP
>
>
> The kernel is an unmodified generic 4.5 kernel; the 4.4 kernel was
> unmodified as well.
>
>
> The syctl.conf changes are nothing out of the ordinary:
> --- /etc/sysctl.conf Sat May 23 15:49:49 2009
> +++ /root/media/etc45/etc45/sysctl.conf Sat Feb 28 18:09:35 2009
> @@ -4,7 +4,7 @@
> # boot time. See sysctl(3) and sysctl(8) for more information on
> # the many available variables.
> #
> -net.inet.ip.forwarding=1 # 1=Permit forwarding (routing) of IPv4
> packets
> +#net.inet.ip.forwarding=1 # 1=Permit forwarding (routing) of IPv4
> packets
> #net.inet.ip.mforwarding=1 # 1=Permit forwarding (routing) of IPv4
> multicast packets
> #net.inet.ip.multipath=1 # 1=Enable IP multipath routing
> #net.inet6.ip6.forwarding=1 # 1=Permit forwarding (routing) of IPv6
> packets
> @@ -32,6 +32,6 @@
> #kern.nosuidcoredump=2 # 2=Put suid coredumps in /var/crash
> #kern.watchdog.period=32 # >0=Enable hardware watchdog(4) timer if
> available
> #kern.watchdog.auto=0 # 0=Disable automatic watchdog(4)
> retriggering
> -machdep.led_blink=1 # 1=On sparc64, make led(s) blink
> +#machdep.led_blink=1 # 1=On sparc64, make led(s) blink
> #machdep.allowaperture=1 # See xf86(4)
> #machdep.kbdreset=1 # permit console to do a nice halt
>
>
> rc.conf.local starts a few services, nothing out of the ordinary:
> # cat /etc/rc.conf.local
> pf=YES
> dhcpd_flags=""
> named_flags=""
> ntpd_flags=""
> ftpproxy_flags=""
> isakmpd_flags="-K"
> ipsec=YES
>
>
> Let me know if anyone needs any more information or if this is in fact a
bug
> and I will submit a bug report.
>
>
> Here is my dmesg:
>
> console is keyboard/display
> Copyright (c) 1982, 1986, 1989, 1991, 1993
> The Regents of the University of California. All rights reserved.
> Copyright (c) 1995-2009 OpenBSD. All rights reserved.
> http://www.OpenBSD.org
>
> OpenBSD 4.5 (GENERIC) #1898: Sat Feb 28 17:42:44 MST 2009
> [email protected]:/usr/src/sys/arch/sparc64/compile/GENERIC
> real mem = 805306368 (768MB)
> avail mem = 765681664 (730MB)
> mainbus0 at root: Sun Blade 100 (UltraSPARC-IIe)
> cpu0 at mainbus0: SUNW,UltraSPARC-IIe (rev 1.4) @ 502 MHz
> cpu0: physical 16K instruction (32 b/l), 16K data (32 b/l), 256K external
> (64 b/l)
> psycho0 at mainbus0: pci108e,a001, impl 0, version 0, ign 7c0
> psycho0: bus range 0-1, PCI bus 0
> psycho0: dvma map c0000000-dfffffff
> pci0 at psycho0
> ebus0 at pci0 dev 12 function 0 "Sun RIO EBus" rev 0x01
> "flashprom" at ebus0 addr 0-fffff not configured
> clock1 at ebus0 addr 0-1fff: mk48t59
> ebus1 at pci0 dev 7 function 0 "Acer Labs M1533 ISA" rev 0x00
> "dma" at ebus1 addr 0-ffff ivec 0x2a not configured
> power0 at ebus1 addr 800-82f ivec 0x20
> com0 at ebus1 addr 3f8-3ff ivec 0x2b: ns16550a, 16 byte fifo
> com1 at ebus1 addr 2e8-2ef ivec 0x2b: ns16550a, 16 byte fifo
> alipm0 at pci0 dev 3 function 0 "Acer Labs M7101 Power" rev 0x00: 223KHz
> clock
> iic0 at alipm0
> "max1617" at alipm0 addr 0x18 skipped due to alipm0 bugs
> "scm001" at alipm0 addr 0x20 skipped due to alipm0 bugs
> spdmem0 at iic0 addr 0x50: 256MB SDRAM ECC PC133CL2
> spdmem1 at iic0 addr 0x51: 256MB SDRAM ECC PC133CL2
> spdmem2 at iic0 addr 0x52: 256MB SDRAM ECC PC133CL2
> gem0 at pci0 dev 12 function 1 "Sun ERI Ether" rev 0x01: ivec 0x7c6,
address
> 00:03:ba:04:b2:1d
> ukphy0 at gem0 phy 1: Generic IEEE 802.3u media interface, rev. 1: OUI
> 0x0010dd, model 0x0002
> "Sun FireWire" rev 0x01 at pci0 dev 12 function 2 not configured
> ohci0 at pci0 dev 12 function 3 "Sun USB" rev 0x01: ivec 0x7e4, version
1.0,
> legacy support
> autri0 at pci0 dev 8 function 0 "Acer Labs M5451 Audio" rev 0x01: ivec
0x7e3
> ac97: codec id 0x41445348 (Analog Devices AD1881A)
> ac97: codec features headphone, Analog Devices Phat Stereo
> audio0 at autri0
> midi0 at autri0: <4DWAVE MIDI UART>
> pciide0 at pci0 dev 13 function 0 "Acer Labs M5229 UDMA IDE" rev 0xc3: DMA,
> channel 0 configured to native-PCI, channel 1 configured to native-PCI
> pciide0: using ivec 0x7cc for native-PCI interrupt
> wd0 at pciide0 channel 0 drive 0: <WDC WD1600AAJB-56WRA0>
> wd0: 16-sector PIO, LBA48, 152627MB, 312581808 sectors
> wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
> atapiscsi0 at pciide0 channel 1 drive 0
> scsibus0 at atapiscsi0: 2 targets
> cd0 at scsibus0 targ 0 lun 0: <LITEON, CD-ROM LTN486S, YSU1> ATAPI 5/cdrom
> removable
> cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
> ppb0 at pci0 dev 5 function 0 "DEC 21152 PCI-PCI" rev 0x03
> pci1 at ppb0 bus 1
> em0 at pci1 dev 0 function 0 "Intel PRO/1000T (82544GC)" rev 0x02: ivec
> 0x7d9, address 00:02:b3:ed:68:89
> vgafb0 at pci0 dev 19 function 0 "ATI Rage XL" rev 0x27
> wsdisplay0 at vgafb0 mux 1: console (std, sun emulation)
> usb0 at ohci0: USB revision 1.0
> uhub0 at usb0 "Sun OHCI root hub" rev 1.00/1.00 addr 1
> uhidev0 at uhub0 port 4 configuration 1 interface 0 "Sun Microsystems Type
6
> Keyboard" rev 1.00/1.02 addr 2
> uhidev0: iclass 3/1
> ukbd0 at uhidev0: 8 modifier keys, 6 key codes, country code 33
> wskbd0 at ukbd0: console keyboard, using wsdisplay0
> softraid0 at root
> bootpath: /p...@1f,0/i...@d,0/d...@0,0
> root on wd0a swap on wd0b dump on wd0b
>
> Thanks,
> Axton Grams
>
Applied the following patch (compliments to Brad) to 4.5 stable
branch, rebuilt, installed, and things are now working as expected.
Index: if_em.c
===================================================================
RCS file: /cvs/src/sys/dev/pci/if_em.c,v
retrieving revision 1.207
diff -u -p -r1.207 if_em.c
--- if_em.c 27 Jan 2009 09:17:51 -0000 1.207
+++ if_em.c 24 May 2009 21:08:29 -0000
@@ -2752,7 +2752,7 @@ em_rxeof(struct em_softc *sc, int count)
#if NVLAN > 0
if (desc->status & E1000_RXD_STAT_VP) {
m->m_pkthdr.ether_vtag =
- (desc->special &
+ (letoh16(desc->special) &
E1000_RXD_SPC_VLAN_MASK);
m->m_flags |= M_VLANTAG;
}
He had the following comments:
"It looks like a byte swap was missed when re-adding the support for
HW VLAN tagging and stripping so this would affect em(4) with the
reception of VLAN tagged packets when in use on any big endian
architecture such as sparc64 ;)."
Thanks,
Axton Grams
