Dear Rump kernel community,
I've been working on an Intel gigabit rump kernel for the L4Re system. I
am using a stripped-down version of the POSIX rumpuser and a
custom-developed version of pci-userspace for L4Re.
Now, let me go straight to my issue. I am observing lots of invocations
of rumpcomp_pci_virt_to_mach() on addresses that were allocated by
rumpuser_malloc(). As rumpuser_malloc() allocates ordinary unpinned host
virtual memory, it does not make sense to me to translate it into host
physical memory; and on L4Re this is not possible at all, so the
translation cannot be performed.
I found that this memory is allocated by the pool_page_alloc() as
eventually invoked from wm_add_rxbuf() and the
rumpcomp_pci_virt_to_mach() is attempted from bus_dmamap_load_mbuf()
also invoked from wm_add_rxbuf(). The detailed stack traces follow at
the end of this e-mail.
Now, my question is how to handle this situation. I noticed that both
Hurd and Linux print a warning and return 0 if they cannot find the
translation. But is it even sensible to invoke
rumpcomp_pci_virt_to_mach() on something which was not allocated by
rumpcomp_pci, ie. rumpcomp_pci_dmalloc()? Should an identity mapping be
assumed here, just as in the case of the rump version of vtophys(). I
could also modify rumpuser to always pin the allocated memory, but that
would be a huge overkill for other invocations of rumpuser_malloc().
Thanks in advance for any suggestions.
Cheers,
Jakub
Breakpoint 1, rumpuser_malloc (howmuch=howmuch@entry=4096,
alignment=alignment@entry=4096, memp=memp@entry=0x8000f868) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/lib/rumpuser/src/rumpuser_mem.c:48⏎
48␉·····{⏎
#0 rumpuser_malloc (howmuch=howmuch@entry=4096,
alignment=alignment@entry=4096, memp=memp@entry=0x8000f868) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/lib/rumpuser/src/rumpuser_mem.c:48⏎
#1 0x0000000001147d1b in rump_hypermalloc (howmuch=4096,
alignment=4096, waitok=false, wmsg=0x11aa871 "kmalloc")⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/librump/rumpkern/vm.c:1235⏎
#2 0x0000000001147de1 in uvm_km_kmem_alloc (vm=<optimized out>,
size=<optimized out>, flags=<optimized out>, addr=0x8000f8d8)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/librump/rumpkern/vm.c:803⏎
#3 0x000000000111695d in pool_page_alloc (pp=<optimized out>,
flags=<optimized out>)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:2588⏎
#4 0x0000000001117d84 in pool_allocator_alloc (flags=2, pp=0xf91240) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:2558⏎
#5 pool_grow (pp=pp@entry=0xf91240, flags=flags@entry=2) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:1098⏎
#6 0x0000000001117ad5 in pool_catchup (pp=<optimized out>) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:1245⏎
#7 pool_get (pp=pp@entry=0xf91240, flags=flags@entry=0) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:943⏎
#8 0x0000000001118ec3 in pool_cache_get_slow (cc=cc@entry=0xeb51c0,
s=s@entry=0, objectp=objectp@entry=0x8000fa08, pap=pap@entry=0xf934a0,
flags=flags@entry=0)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:2186⏎
#9 0x000000000111a3bb in pool_cache_get_paddr (pc=0xf91240, flags=0,
pap=0xf934a0) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/lib/librump/../../sys/rump/../kern/subr_pool.c:2276⏎
#10 0x0000000001017bcb in wm_add_rxbuf (rxq=rxq@entry=0x2ed18a8,
idx=idx@entry=2) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:4072⏎
#11 0x000000000101bf8c in wm_init_rx_buffer (sc=0xfd9000, sc=0xfd9000,
rxq=0x2ed18a8) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:5799⏎
#12 wm_init_rx_queue (rxq=0x2ed18a8, wmq=0x2ed1000, sc=0xfd9000) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:5844⏎
#13 wm_init_txrx_queues (sc=<optimized out>) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:5868⏎
#14 wm_init_locked (ifp=<optimized out>) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:4584⏎
#15 wm_init (ifp=0xfd9060) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:4483⏎
#16 0x00000000010116ba in wm_ioctl (ifp=0xfd9060, cmd=2156947728,
data=0x8000fd00) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:2909⏎
#17 0x00000000010a6034 in doifioctl (so=so@entry=0xf1ddb0,
cmd=cmd@entry=2156947728, data=data@entry=0x8000fd00, l=0x31f7000)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/net/lib/libnet/../../../../net/if.c:2688⏎
#18 0x000000000102fb68 in wrapifioctl (so=0xf1ddb0,
cmd=cmd@entry=2156947728, data=data@entry=0x8000fd00)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig.c:63⏎
#19 0x000000000102fbfb in chflag (ifname=ifname@entry=0x11ac331 "wm0",
edflag=edflag@entry=0x102fb10 <addup>)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig.c:105⏎
#20 0x000000000102fe9f in rump_netconfig_ifup
(ifname=ifname@entry=0x11ac331 "wm0") at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig.c:112⏎
#21 0x000000000102e079 in rump_pub_netconfig_ifup
(arg1=arg1@entry=0x11ac331 "wm0") at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig_if_wrappers.c:44⏎
#22 0x000000000114fede in main () at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/examples/rumpdemo/rumpdemo.cc:30⏎
(gdb) c⏎
Continuing.⏎
⏎
Breakpoint 4, rumpuser_malloc (howmuch=howmuch@entry=4096,
alignment=alignment@entry=4096, memp=memp@entry=0x8000f868) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/lib/rumpuser/src/rumpuser_mem.c:64⏎
64␉·····␉·······*memp = mem;⏎
$22 = (void *) 0x3a11000⏎
Breakpoint 2, rumpcomp_pci_virt_to_mach (virt=0x3a11800) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/lib/rumppciuser/src/rumppciuser.cc:279⏎
279␉····{⏎
#0 rumpcomp_pci_virt_to_mach (virt=0x3a11800) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/lib/rumppciuser/src/rumppciuser.cc:279⏎
#1 0x00000000010202f4 in bus_dmamap_load_mbuf (t=0x40, map=0x8e620,
m0=0xf93000, flags=257)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/dev/lib/libpci/rumpdev_bus_dma.c:333⏎
#2 0x0000000001017c79 in wm_add_rxbuf (rxq=rxq@entry=0x2ed18a8,
idx=idx@entry=4) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:4084⏎
#3 0x000000000101bf8c in wm_init_rx_buffer (sc=0xfd9000, sc=0xfd9000,
rxq=0x2ed18a8) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:5799⏎
#4 wm_init_rx_queue (rxq=0x2ed18a8, wmq=0x2ed1000, sc=0xfd9000) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:5844⏎
#5 wm_init_txrx_queues (sc=<optimized out>) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:5868⏎
#6 wm_init_locked (ifp=<optimized out>) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:4584⏎
#7 wm_init (ifp=0xfd9060) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:4483⏎
#8 0x00000000010116ba in wm_ioctl (ifp=0xfd9060, cmd=2156947728,
data=0x8000fd00) at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/../dev/pci/if_wm.c:2909⏎
#9 0x00000000010a6034 in doifioctl (so=so@entry=0xf1ddb0,
cmd=cmd@entry=2156947728, data=data@entry=0x8000fd00, l=0x31f7000)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/src-netbsd/sys/rump/net/lib/libnet/../../../../net/if.c:2688⏎
#10 0x000000000102fb68 in wrapifioctl (so=0xf1ddb0,
cmd=cmd@entry=2156947728, data=data@entry=0x8000fd00)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig.c:63⏎
#11 0x000000000102fbfb in chflag (ifname=ifname@entry=0x11ac331 "wm0",
edflag=edflag@entry=0x102fb10 <addup>)⏎
at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig.c:105⏎
#12 0x000000000102fe9f in rump_netconfig_ifup
(ifname=ifname@entry=0x11ac331 "wm0") at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig.c:112⏎
#13 0x000000000102e079 in rump_pub_netconfig_ifup
(arg1=arg1@entry=0x11ac331 "wm0") at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/contrib/buildrump.sh/brlib/libnetconfig/netconfig_if_wrappers.c:44⏎
#14 0x000000000114fede in main () at
/home/jermar/Kernkonzept/software/l4/pkg/rumpkernel/examples/rumpdemo/rumpdemo.cc:30⏎
