Hello, I am running an OpenBSD/armv7 guest on a QEMU 4.2.0 "virt" machine host; see the attached file (from_qemu_virt.dts) for the fdt of the guest machine, it has only a virtio-mmio bus with a virtio-net device attached.
The QEMU command line is also attached, together with source and bin for
the tiny bootloader for the bsd kernel (from the distfiles of 6.6 release)
used as QEMU "bios".
I used the netdev user backend.
Boot is OK (see dmesg.txt).
The vio network interface is configured by dhclient and we have this:
my# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
index 3 priority 0 llprio 3
groups: lo
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
vio0: flags=808843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF4> mtu
1500
lladdr 52:54:00:12:34:56
index 1 priority 0 llprio 3
groups: egress
media: Ethernet autoselect
status: active
inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
enc0: flags=0<>
index 2 priority 0 llprio 3
groups: enc
status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33168
index 4 priority 0 llprio 3
groups: pflog
my# route -n show
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Prio
Iface
default 10.0.2.2 UGS 0 0 - 8 vio0
224/4 127.0.0.1 URS 0 0 32768 8 lo0
10.0.2/24 10.0.2.15 UCn 1 0 - 4 vio0
10.0.2.2 52:55:0a0:00:02:02 UHLch 1 4 - 3
vio0
10.0.2.15 52:54:00:12:34:56 UHLl 0 46 - 1 vio0
10.0.2.255 10.0.2.15 UHb 0 0 - 1 vio0
127/8 127.0.0.1 UGRS 0 0 32768 8 lo0
127.0.0.1 127.0.0.1 UHhl 1 2 32768 1 lo0
[[ edited Inet6 routes ..]]
Ping works OK from the guest to the host ICMP Echo requests are correctly
sent and Echo replies correctly received. It does not work from the guest
to a public IP (but that's fine, it is a known limitation of QEMU net user).
UDP packets are OK in the direction guest --> host, but not in reverse host
--> guest. This cause failure of DNS resolution for instance. TCP packets
have the same problem (the guest sends the SYN, which is received by the
host who sends the SYN-ACK, but the SYN-ACK is not "seen" by the OBSD guest
and connect timeouts).
What's surprising (to me !) is that packets are visible on tcpdump on the
guest (with 0 packets "dropped by kernel")
Steps to reproduce:
on guest:
my# tcpdump -w test.dump -p&
[1] 13288
my# tcpdump: listening on vio0, link-type EN10MB
my# nc -v -u 10.0.2.2 2222
Connection to 10.0.2.2 2222 port [udp/*] succeeded!
hello from guest (<<-- typed on the guest console)
on host:
$ nc -l -v -u -p 2222
listening on 0.0.0.0:2222 ...
connect to 127.0.0.1:2222 from localhost.localdomain:60487 (127.0.0.1:60487)
hello from guest (-->> got this from the guest)
hello from host (<<-- typed on the host console, NOT shown on the guest
console)
Now the tcpdump -neX on the guest is attached, you can see that the reply
packets are seen by the kernel but forwarded to beyond "to user space". I
also attached tcpdump on the guest, no difference is shown.
I tried the 0x02 flags of vio (see dmesg) with no effect. The same with
0x100 or by guetting the vio0 interface in promiscuous mode with tcpdump.
pf has default rules (block return all, pass all flags S/SA, X11 and dpb
builder blocking). Same problem with pf disabled. When appropriate log
rules are configured, I see the faulty packets in pflogd journal as in the
guest tcpdump.
Best regards,
Alexandre J
qemu-system-arm -M virt -m 512 -bios test.bin -nographic \ -device loader,file=bsd,force-raw=on,addr=0x402ff000 -device loader,file=mine.dtb,addr=0x40100000 \ -device virtio-blk-device,drive=drv -drive if=none,id=drv,file=obsdarm.img \ -netdev user,id=n1 -device virtio-net-device,netdev=n1 \ -object filter-dump,netdev=n1,id=f,file=dump.dat
from_qemu_virt.dts
Description: audio/vnd.dts
.arm eor r0,r0 mov r1,#0x1234 movt r2,#0x4010 movt r3,#0x4030 mov pc,r3
test.bin
Description: Binary data
XXXXOpenBSD 6.6 (GENERIC) #226: Sat Oct 12 08:36:17 MDT 2019
[email protected]:/usr/src/sys/arch/armv7/compile/GENERIC
real mem = 536870912 (512MB)
avail mem = 516952064 (493MB)
warning: no entropy supplied by boot loader
mainbus0 at root: unknown model
cpu0 at mainbus0 mpidr 0: ARM Cortex-A15 r2p1
cpu0: 32KB 64b/line 2-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu0: 2304KB 64b/line 16-way L2 cache
cortex0 at mainbus0
psci0 at mainbus0: PSCI 0.2
ampintc0 at mainbus0 nirq 288, ncpu 1: "intc"
ampintcmsi0 at ampintc0: nspi 64
simplebus0 at mainbus0: "platform"
virtio0 at mainbus0: Virtio Unknown (0) Device
virtio1 at mainbus0: Virtio Unknown (0) Device
virtio2 at mainbus0: Virtio Unknown (0) Device
virtio3 at mainbus0: Virtio Unknown (0) Device
virtio4 at mainbus0: Virtio Unknown (0) Device
virtio5 at mainbus0: Virtio Unknown (0) Device
virtio6 at mainbus0: Virtio Unknown (0) Device
virtio7 at mainbus0: Virtio Unknown (0) Device
virtio8 at mainbus0: Virtio Unknown (0) Device
virtio9 at mainbus0: Virtio Unknown (0) Device
virtio10 at mainbus0: Virtio Unknown (0) Device
virtio11 at mainbus0: Virtio Unknown (0) Device
virtio12 at mainbus0: Virtio Unknown (0) Device
virtio13 at mainbus0: Virtio Unknown (0) Device
virtio14 at mainbus0: Virtio Unknown (0) Device
virtio15 at mainbus0: Virtio Unknown (0) Device
virtio16 at mainbus0: Virtio Unknown (0) Device
virtio17 at mainbus0: Virtio Unknown (0) Device
virtio18 at mainbus0: Virtio Unknown (0) Device
virtio19 at mainbus0: Virtio Unknown (0) Device
virtio20 at mainbus0: Virtio Unknown (0) Device
virtio21 at mainbus0: Virtio Unknown (0) Device
virtio22 at mainbus0: Virtio Unknown (0) Device
virtio23 at mainbus0: Virtio Unknown (0) Device
virtio24 at mainbus0: Virtio Unknown (0) Device
virtio25 at mainbus0: Virtio Unknown (0) Device
virtio26 at mainbus0: Virtio Unknown (0) Device
virtio27 at mainbus0: Virtio Unknown (0) Device
virtio28 at mainbus0: Virtio Unknown (0) Device
virtio29 at mainbus0: Virtio Unknown (0) Device
virtio30 at mainbus0: Virtio Network Device
vio0 at virtio30 RingEventIdx disabled by UKC: address 52:54:00:12:34:56
virtio31 at mainbus0: Virtio Block Device
vioblk0 at virtio31
scsibus0 at vioblk0: 2 targets
sd0 at scsibus0 targ 0 lun 0: <VirtIO, Block Device, >
sd0: 3072MB, 512 bytes/sector, 6291456 sectors
plrtc0 at mainbus0
pluart0 at mainbus0: console
agtimer0 at mainbus0: tick rate 62500 KHz
/dev/ksyms: Symbol table not valid.
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
bootfile: sd0a
boot device: sd0
root on sd0a (db7ead66690094f5.a) swap on sd0b dump on sd0b
00:15:38.765961 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 28
0x0000: 0001 0800 0604 0001 5254 0012 3456 0a00 ........RT..4V..
0x0010: 020f 0000 0000 0000 0a00 0202 ............
00:15:38.765989 ARP, Reply 10.0.2.2 is-at 52:55:0a:00:02:02, length 50
0x0000: 0001 0800 0604 0002 5255 0a00 0202 0a00 ........RU......
0x0010: 0202 5254 0012 3456 0a00 020f 0000 0000 ..RT..4V........
0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0030: 0000 ..
00:15:38.766001 IP 10.0.2.15.7598 > 10.0.2.2.2222: UDP, length 1
0x0000: 4500 001d 1f01 0000 4011 43bf 0a00 020f [email protected].....
0x0010: 0a00 0202 1dae 08ae 0009 696f 58 ..........ioX
00:15:38.767774 IP 10.0.2.15.7598 > 10.0.2.2.2222: UDP, length 1
0x0000: 4500 001d 9bb8 0000 4011 c707 0a00 020f E.......@.......
0x0010: 0a00 0202 1dae 08ae 0009 696f 58 ..........ioX
00:15:38.768171 IP 10.0.2.15.7598 > 10.0.2.2.2222: UDP, length 1
0x0000: 4500 001d 4732 0000 4011 1b8e 0a00 020f E...G2..@.......
0x0010: 0a00 0202 1dae 08ae 0009 696f 58 ..........ioX
00:15:38.768524 IP 10.0.2.15.7598 > 10.0.2.2.2222: UDP, length 1
0x0000: 4500 001d 2008 0000 4011 42b8 0a00 020f [email protected].....
0x0010: 0a00 0202 1dae 08ae 0009 696f 58 ..........ioX
00:15:42.754993 IP 10.0.2.15.7598 > 10.0.2.2.2222: UDP, length 17
0x0000: 4500 002d 3d01 0000 4011 25af 0a00 020f E..-=...@.%.....
0x0010: 0a00 0202 1dae 08ae 0019 943c 6865 6c6c ...........<hell
0x0020: 6f20 6672 6f6d 2067 7565 7374 0a o.from.guest.
00:15:48.703628 IP 10.0.2.2.2222 > 10.0.2.15.7598: UDP, length 16
0x0000: 4500 002c 0038 0000 4011 6279 0a00 0202 E..,[email protected]....
0x0010: 0a00 020f 08ae 1dae 0018 a399 6865 6c6c ............hell
0x0020: 6f20 6672 6f6d 2068 6f73 740a o.from.host.
01:15:39.066108 52:54:00:12:34:56 ff:ff:ff:ff:ff:ff 0806 42: arp who-has 10.0.2.2 tell 10.0.2.15 0000: ffff ffff ffff 5254 0012 3456 0806 0001 ......RT..4V.... 0010: 0800 0604 0001 5254 0012 3456 0a00 020f ......RT..4V.... 0020: 0000 0000 0000 0a00 0202 .......... 01:15:39.066137 52:54:00:12:34:56 52:55:0a:00:02:02 0800 43: 10.0.2.15.7598 > 10.0.2.2.2222: udp 1 0000: 5255 0a00 0202 5254 0012 3456 0800 4500 RU....RT..4V..E. 0010: 001d 1f01 0000 4011 43bf 0a00 020f 0a00 [email protected]....... 0020: 0202 1dae 08ae 0009 696f 58 ........ioX 01:15:39.066529 52:55:0a:00:02:02 52:54:00:12:34:56 0806 64: arp reply 10.0.2.2 is-at 52:55:0a:00:02:02 0000: 5254 0012 3456 5255 0a00 0202 0806 0001 RT..4VRU........ 0010: 0800 0604 0002 5255 0a00 0202 0a00 0202 ......RU........ 0020: 5254 0012 3456 0a00 020f 0000 0000 0000 RT..4V.......... 0030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 01:15:39.067979 52:54:00:12:34:56 52:55:0a:00:02:02 0800 43: 10.0.2.15.7598 > 10.0.2.2.2222: udp 1 0000: 5255 0a00 0202 5254 0012 3456 0800 4500 RU....RT..4V..E. 0010: 001d 9bb8 0000 4011 c707 0a00 020f 0a00 ......@......... 0020: 0202 1dae 08ae 0009 696f 58 ........ioX 01:15:39.068345 52:54:00:12:34:56 52:55:0a:00:02:02 0800 43: 10.0.2.15.7598 > 10.0.2.2.2222: udp 1 0000: 5255 0a00 0202 5254 0012 3456 0800 4500 RU....RT..4V..E. 0010: 001d 4732 0000 4011 1b8e 0a00 020f 0a00 ..G2..@......... 0020: 0202 1dae 08ae 0009 696f 58 ........ioX 01:15:39.068724 52:54:00:12:34:56 52:55:0a:00:02:02 0800 43: 10.0.2.15.7598 > 10.0.2.2.2222: udp 1 0000: 5255 0a00 0202 5254 0012 3456 0800 4500 RU....RT..4V..E. 0010: 001d 2008 0000 4011 42b8 0a00 020f 0a00 .. [email protected]....... 0020: 0202 1dae 08ae 0009 696f 58 ........ioX 01:15:43.055114 52:54:00:12:34:56 52:55:0a:00:02:02 0800 59: 10.0.2.15.7598 > 10.0.2.2.2222: udp 17 0000: 5255 0a00 0202 5254 0012 3456 0800 4500 RU....RT..4V..E. 0010: 002d 3d01 0000 4011 25af 0a00 020f 0a00 .-=...@.%....... 0020: 0202 1dae 08ae 0019 943c 6865 6c6c 6f20 .........<hello 0030: 6672 6f6d 2067 7565 7374 0a from guest. 01:15:49.004206 52:55:0a:00:02:02 52:54:00:12:34:56 0800 58: 10.0.2.2.2222 > 10.0.2.15.7598: udp 16 0000: 5254 0012 3456 5255 0a00 0202 0800 4500 RT..4VRU......E. 0010: 002c 0038 0000 4011 6279 0a00 0202 0a00 .,[email protected]...... 0020: 020f 08ae 1dae 0018 a399 6865 6c6c 6f20 ..........hello 0030: 6672 6f6d 2068 6f73 740a from host.
