On 27/10/2026 19:11, Mike Kelly wrote:
There seems to be an issue with systems having HIGHMEM memory which
causes opening the ld device to lock the system (to a degree
anyway). This is likely to be related to DMA transfers which I'll look
at next.
The cause of this is proving difficult to determine but it doesn't
appear to be directly related to the new NVMe code. I'll show what I
have determined so far in the hope that someone can help me get closer
to a full understanding.
1) There's no evidence that particular physical addresses for the DMA
memory has any connection. Altering gnumach to supply no memory at all
from the HIGHMEM segment (when it has memory available) makes no
difference to the outcome . In any case, this PCI bus supports 64 bit
addressing, so it shouldn't matter anyway.
2) In my virt-manager configuration with 2.8G or more RAM, the physical
address of the PCI region for NVMe is at 0xe00000000. This is mapped at
virtual address 0x1a1000 by libpciaccess (pci_device_map_range()). This
physical address is larger than the 32 bit space, but in similar systems
with lower memory, it is mapped within the 32 bit physical address. This
physical address is perhaps irrelevant.
During boot, there are many successful writes to this PCI region and in
all cases the NVMe device is initialised successfully. With lower memory
systems, these same writes continue to work and the logical disk is read
perfectly. I also enabled Qemu tracing and there are numerous trace
logs showing the PCI writes succeeding:
pci_nvme_mmio_write addr 0x1008 data 0x1 size 4
With a 3G RAM machine, debug trace that I added to the rumpkernel code
shows it reaching the call to 'nvme_write4' (in
nvme_op_sq_leave_locked), and indeed executing that call, but no trace
emerges in the Qemu trace log. I checked that the virtual address (
0x1a2008) matches the expected physical address using the gnumach 'show
map' debugger and also adding my own debugger command to lookup the page
table entry and all seems fine there. I concluded that there must be a
problem with Qemu itself.
I then made use of the qemu-monitor by running the following in a
continuous loop on the 3G machine:
# while true; do virsh qemu-monitor-command rump-nvme2-x64 --hmp 'xp/x
0xe00000000'; done
During Hurd boot, this reports:
0000000e00000000: 0x0f0107ff
But shortly after launching pci-arbiter, acpi etc it then shows:
0000000e00000000: Cannot access memory
It seems to me that Qemu has removed the memory region for the NVMe
device. I think it would be impossible for Hurd/gnumach to do anything
at all that might make that happen, wouldn't it ? If that's true, then
surely it must be some unexpected behaviour or bug in Qemu itself?
Can anyone see something incorrect in my conclusions?
Thanks,
Mike.
