On Wed, 21 May 2025, Julien Grall wrote: > > For static dom0less flows, we can do the same as for xl flows but we have > > the > > additional problem of domU's PCI bus enumeration racing with QEMU. > > On x86, when domU's access a memory range that has not yet got IOREQ's > > connected to it, the accesses succeeds with reads returning 0xFFFFFFFF and > > writes ignored. This makes it easy for guests to wait for IOREQ devices to > > pop up since guests will find an empty bus and can initiate a rescan later > > when QEMU attaches. On ARM, we trap on these accesses. > > > If we on ARM add support for MMIO background regions with a default > read value, > > i.e mmio handlers that have lower priority than IOREQs and that are > > read-const + writes-ignored, we could support the same flow on ARM. > > This may be generally useful for other devices as well (e.g virtio-mmio or > > something else). We could also use this to defer PCI enumeration. > > Regardless what I wrote above, if we are going down the route of returning > 0xFFFFFFFF, I would just do it for every IOs rather than the one specify in > the device-tree. This could still behind a per-domain option, but it would at > least make simpler to setup the system (AFAIU, in your current proposal, we > would need to specify the range in multiple places). This seems like a good idea.
> > So the next versions of this series I was thinking to remove the PCI > > specifics > > and instead add FDT bindings to ARM dom0less enabling setup of user > > configurable (by address-range and read-value) background mmio regions. > > Xen would then support option #2 without any PCI specifics added. > > > > Thoughts? > > > > Cheers, > > Edgar > > > > # References to spec > > > > PCI express base specification: > > 7.5.1.1.1 Vendor ID Register (Offset 00h) > > The Vendor ID register is HwInit and the value in this register identifies > > the manufacturer of the Function. In keeping with > > PCI-SIG procedures, valid vendor identifiers must be allocated by the > > PCI-SIG to ensure uniqueness. Each vendor must > > have at least one Vendor ID. It is recommended that software read the Vendor > > ID register to determine if a Function is > > present, where a value of FFFFh indicates that no Function is present. > > > > PCI Firmware Specification: > > 3.5 Device State at Firmware/Operating System Handoff > > Page 34: > > The operating system is required to configure PCI subsystems: > > During hotplug > > For devices that take too long to come out of reset > > PCI-to-PCI bridges that are at levels below what firmware is designed to > > configure > > > > Page 36: > > Note: The operating system does not have to walk all buses during boot. The > > kernel can > > automatically configure devices on request; i.e., an event can cause a scan > > of I/O on demand. > > I am not sure why you quote this. To me it reads like this is up to the OS to > decide when to access the PCI bus. As we don't control the OS, this doesn't > seem a behavior Xen can rely on. > > > > > FPGA's can be programmed at runtime and appear on the ECAM bus silently. > > An PCI rescan needs to be triggered for the OS to discover the device: > > Intel FPGAs: > > https://www.intel.com/content/www/us/en/docs/programmable/683190/1-3-1/how-to-rescan-bus-and-re-enable-aer.html > > To clarify, you are saying the ECAM bus may be completely empty (e.g. > everything is reading as ~0) and some part of the ECAM will return a non ~0 > value when the FPGA run. >From what I found from my searches on the topic, the PCIe spec says when a device is not present or not initialized, the system typically receives a value of 0xFFFFFFFF on read. Once a device becomes active and responds to configuration accesses, subsequent reads will return valid data. The specifications do not require interrupts or hot-plug events for device detection; polling the configuration space is a compliant method. Which actually points to your suggestion above to do this for every IOs.
