Hi Paul, On 5/23/23 00:51, Paul Boddie wrote:
On Sunday, 23 April 2023 10:09:52 CEST Jakub Jermář wrote:The idea is that you have a dataspace and a DMA space. In order to obtain a DMA address that you can then use to program your I/O device, you need to map a portion of the dataspace into the DMA space. After use you unmap it from the DMA space. See for example the implementation of the Inout buffer in our nvme-driver: https://github.com/kernkonzept/nvme-driver/blob/master/server/src/inout_buffer.h#L46 I also looked at the main.cc file to see how I might obtain a DMA space and assign it to a DMA domain. Obtaining a DMA space appears to involve invoking a factory create operation using the L4RE_PROTO_DMA_SPACE protocol. Here, I found a difference between the L4Re C and C++ libraries, with the latter referencing a "user_factory" but defining it as the memory allocator, so I just used the mem_alloc member of the environment structure in C for the factory.
In C++ mem_alloc and user_factory are nearly the same. Mem_alloc extends the factory protocol with allocation for dataspaces. At the moment, this is effectively a convenience wrapper, as both are using the same capability index and thus talk to the same server. The reason for the user_factory name is to be able to write less confusing code, e.g. where one would use mem_alloc to create some non-memory objects.
With a DMA space, it should supposedly be possible to specify ~0U (or -1, I guess) as the domain identifier when assigning the space to a domain, but this does not work and yields the following warning: IO | vbus vbus does not support a global DMA domain This along with an error indicating "No such object found". The main.cc file in the nvme-driver performs a search for a DMA domain resource, but replicating this myself produces a few issues. First of all, a search from the root device doesn't manage to find a resource offering a DMA domain if I traverse devices and their resources. However, I see that the hw_root_bus.cc file in pkg/io/io/server/src adds root resources for a DMA domain, amongst other things, and if I also investigate the resources of the root device itself, then I do find such resources. Strangely, these resources appear to have an identifier of zero after they have been added to the bus, even though they get set up with expected identifiers such as "DMAD". Meanwhile, in the main.cc file in pkg/io/io/server/src, a DMA domain resource is explicitly created when no iommu capability is provided to the server. This actually appears in the dump of the devices and resources at the system bus level, as produced by Io, but not within the vbus. So, I think I need some suggestions as to how I can actually obtain a domain identifier and maybe how a domain should be made available in the .io file. Sorry if I failed to understand what to do, but none of this is particularly obvious to me.
The path you are on sounds correct. A reason for a missing DMA domain resource on the vbus is (usually) the lack of a DMA capable device on said vbus.
https://github.com/kernkonzept/manifest/wiki/NVMeWithLinux#vbus-configuration-fileshows a vbus configuration file that places all PCI/storage devices on the vbus. I presume your MIPS board lacks PCI, so you have to write the entries for System_bus and the Vbus yourself.
The helpful sections in another tutorial are: https://github.com/kernkonzept/manifest/wiki/HwPassThrough#ios-config-file https://github.com/kernkonzept/manifest/wiki/HwPassThrough#vbus-configuration-fileI found your mercurial instance for the l4re topic but didn't find any io & vbus configuration. Can you point me to your config files?
And/Or send along the Io output with maximum verbosity (-vvvvvvv)? Cheers, Philipp
OpenPGP_signature
Description: OpenPGP digital signature
_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de https://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
