> Date: Fri, 21 Feb 2020 21:49:33 +1100 > From: Jonathan Gray <[email protected]> > > On Fri, Feb 21, 2020 at 04:45:01PM +1100, Jonathan Gray wrote: > > On Tue, Feb 18, 2020 at 12:37:10AM +0100, alexandre wrote: > > > > > > > On Mon, Feb 17, 2020 at 01:31:08AM +0100, Alexandre wrote: > > > > > Hello, > > > > > > > > > > I am running an OpenBSD/armv7 guest on a QEMU 4.2.0 "virt" machine > > > > > host; > > > > > see the attached file (from_qemu_virt.dts) for the fdt of the guest > > > > > machine, it has only a virtio-mmio bus with a virtio-net device > > > > > attached. > > > > > > > > > > The QEMU command line is also attached, together with source and bin > > > > > for > > > > > the tiny bootloader for the bsd kernel (from the distfiles of 6.6 > > > > > release) > > > > > used as QEMU "bios". > > > > > > > > > > I used the netdev user backend. > > > > > > > > > > Boot is OK (see dmesg.txt). > > > > > > > > > > The vio network interface is configured by dhclient and we have this: > > > > > > > > > > my# ifconfig > > > > > lo0: flags=8049 mtu 32768 > > > > > index 3 priority 0 llprio 3 > > > > > groups: lo > > > > > inet6 ::1 prefixlen 128 > > > > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > > > > > inet 127.0.0.1 netmask 0xff000000 > > > > > vio0: flags=808843 mtu > > > > > 1500 > > > > > lladdr 52:54:00:12:34:56 > > > > > index 1 priority 0 llprio 3 > > > > > groups: egress > > > > > media: Ethernet autoselect > > > > > status: active > > > > > inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255 > > > > > enc0: flags=0<> > > > > > index 2 priority 0 llprio 3 > > > > > groups: enc > > > > > status: active > > > > > pflog0: flags=141 mtu 33168 > > > > > index 4 priority 0 llprio 3 > > > > > groups: pflog > > > > > > > > > > my# route -n show > > > > > Routing tables > > > > > > > > > > Internet: > > > > > Destination Gateway Flags Refs Use Mtu Prio > > > > > Iface > > > > > default 10.0.2.2 UGS 0 0 - 8 vio0 > > > > > 224/4 127.0.0.1 URS 0 0 32768 8 lo0 > > > > > 10.0.2/24 10.0.2.15 UCn 1 0 - 4 vio0 > > > > > 10.0.2.2 52:55:0a0:00:02:02 UHLch 1 4 - 3 > > > > > vio0 > > > > > 10.0.2.15 52:54:00:12:34:56 UHLl 0 46 - 1 vio0 > > > > > 10.0.2.255 10.0.2.15 UHb 0 0 - 1 vio0 > > > > > 127/8 127.0.0.1 UGRS 0 0 32768 8 lo0 > > > > > 127.0.0.1 127.0.0.1 UHhl 1 2 32768 1 lo0 > > > > > [[ edited Inet6 routes ..]] > > > > > > > > > > Ping works OK from the guest to the host ICMP Echo requests are > > > > > correctly > > > > > sent and Echo replies correctly received. It does not work from the > > > > > guest > > > > > to a public IP (but that's fine, it is a known limitation of QEMU net > > > > > user). > > > > > > > > > > UDP packets are OK in the direction guest --> host, but not in > > > > > reverse host > > > > > --> guest. This cause failure of DNS resolution for instance. TCP > > > > > packets > > > > > have the same problem (the guest sends the SYN, which is received by > > > > > the > > > > > host who sends the SYN-ACK, but the SYN-ACK is not "seen" by the OBSD > > > > > guest > > > > > and connect timeouts). > > > > > > > > > > What's surprising (to me !) is that packets are visible on tcpdump on > > > > > the > > > > > guest (with 0 packets "dropped by kernel") > > > > > > > > > > Steps to reproduce: > > > > > > > > > > on guest: > > > > > > > > > > my# tcpdump -w test.dump -p& > > > > > [1] 13288 > > > > > my# tcpdump: listening on vio0, link-type EN10MB > > > > > my# nc -v -u 10.0.2.2 2222 > > > > > Connection to 10.0.2.2 2222 port [udp/*] succeeded! > > > > > hello from guest (<<-- typed on the guest console) > > > > > > > > > > on host: > > > > > > > > > > $ nc -l -v -u -p 2222 > > > > > listening on 0.0.0.0:2222 ... > > > > > connect to 127.0.0.1:2222 from localhost.localdomain:60487 > > > > > (127.0.0.1:60487) > > > > > hello from guest (-->> got this from the guest) > > > > > hello from host (<<-- typed on the host console, NOT shown on the > > > > > guest > > > > > console) > > > > > > > > > > Now the tcpdump -neX on the guest is attached, you can see that the > > > > > reply > > > > > packets are seen by the kernel but forwarded to beyond "to user > > > > > space". I > > > > > also attached tcpdump on the guest, no difference is shown. > > > > > > > > > > I tried the 0x02 flags of vio (see dmesg) with no effect. The same > > > > > with > > > > > 0x100 or by guetting the vio0 interface in promiscuous mode with > > > > > tcpdump. > > > > > > > > > > pf has default rules (block return all, pass all flags S/SA, X11 and > > > > > dpb > > > > > builder blocking). Same problem with pf disabled. When appropriate log > > > > > rules are configured, I see the faulty packets in pflogd journal as > > > > > in the > > > > > guest tcpdump. > > > > > > > > > > > > > > > > > > Maybe netstat -s -p ip and netstat -s -p udp help to find the cause of > > > > the > > > > packet drops. Also check pfctl -si if one of those counters change when > > > > you send UDP packets to the guest. > > > > > > > > I normally used qemu with the tap virtio option (using > > > > -net nic,vlan=$id,macaddr=$mac,model=virtio -net tap,vlan=$id,fd=$fd), > > > > never had issues with that. > > > > > > > > > I confirm it is a problem with in4_cksum on ARM. > > > > > > Networking is OK when I either: > > > > > > * change > > > > > > --- sys/dev/pv/vio.c.old > > > +++ sys/dev/pv/vio.c > > > - m->m_pkthdr.csum_flags = 0; > > > + m->m_pkthdr.csum_flags = M_UDP_CSUM_IN_OK | M_TCP_CSUM_IN_OK; > > > > > > so as to disable checksum verification by udp_input() > > > > > > OR: > > > > > > * use the portable C implementation of in4_cksum instead of the ARM one > > > by doing > > > > I can not reproduce this problem on a cubox with fec(4) but can on qemu > > with vio(4) (by the way you can use qemu_arm/u-boot.bin for bios with > > -M virt,highmem=off). > > > > It turns out the armv5te/xscale path (which was removed) works. > > this is enough to make it work
ok kettenis@ The code is a bit inconsistent in its usage of IP_SRC and IP_DST and whether reading beyond the (pseudo)-header is done or not. But this change is an improvement. > Index: in_cksum_arm.S > =================================================================== > RCS file: /cvs/src/sys/arch/arm/arm/in_cksum_arm.S,v > retrieving revision 1.7 > diff -u -p -r1.7 in_cksum_arm.S > --- in_cksum_arm.S 6 Aug 2018 18:39:13 -0000 1.7 > +++ in_cksum_arm.S 21 Feb 2020 10:44:48 -0000 > @@ -140,7 +140,7 @@ ENTRY(in4_cksum) > > /* 0x02: Data 16-bit aligned */ > ldr r4, [r6, #(IP_SRC - 2)] /* r4 = 10xx */ > - ldr r7, [r6, #(IP_DST - 2)] /* r7 = xx76 */ > + ldrh r7, [r6, #(IP_DST + 2)] /* r7 = ..76 */ > ldr r5, [r6, #(IP_SRC + 2)] /* r5 = 5432 */ > mov r4, r4, lsr #16 /* r4 = ..10 */ > orr r4, r4, r7, lsl #16 /* r4 = 7610 */ > >
