On Tuesday, April 05, 2016 09:14:31 AM Konstantin Belousov wrote: > On Mon, Apr 04, 2016 at 03:45:07PM -0700, John Baldwin wrote: > > However, another question is how to deal with systems that do bus address > > translation (like the arm64 ThunderX boxes) where the values in the PCI > > BAR are not CPU physical addresses. To do this properly we may need some > > sort of bus method akin to my bus_map_resource() WIP but one that instead > > returns a suitable 'struct sglist' for a given 'struct resource *' that > > can be used with OBJT_SG to build a VM object to use for the mapping. > Is there any documentation on the ThunderX PCI access mechanisms ?
The Host-PCI (or really some other bus -> PCI) bridges apply a fixed offset to PCI addresses, so if your BAR has address X written to it, the CPU has to use an address of X + Y (where Y is the PCI window base address) to access it. The bridge device decodes the access to address X + Y and generates a PCI transaction to address X. It's not clear to me how DMA works for these devices (e.g. if all DMA requests from PCI devices have to use an IOMMU built into the bridge). > > (At some point I do think I would like a variant of OBJT_SG that used > > managed pages so that mappings could be revoked when whatever is backing > > the sglist is disabled (e.g. the device is ejected or the BAR is > > relocated, etc.).) > Why cannot you use MGTDEVICE pager already ? Driver would need to > maintain its private list of pages, and from this PoV, a convenience > wrapper around MGTDEVICE which unifies operations on sg lists could be > useful. Still, it could be that the wrapper appear to be single-purpose. Mostly I do not have experience with MGTDEVICE, though I was planning to look at it as a way to implement this. Two things though: 1) there may not be a cdev to associate with, and 2) I know of at least one device driver that would use this in addition to using this for a general "map this BAR" ioctl on /dev/pci. There are other cases in the past where I used OBJT_SG but would have preferred to use a variant that used managed pages so that I could invidate any existing mappings. In particular what I want to do is invalidate an object so that any future uses fail. Alternatively, it might be nice to hook a destructor call into a VM object so that I could know when the object is no longer in use (knowing that all its mappings have been destroyed). When using OBJT_SG objects as aliases for other things (memory allocated via contigmalloc or bus_dma or for resources like PCI BARs), I could keep a reference count on the original "thing" that I increment when creating an OBJT_SG object to return from something like d_mmap_single() or the /dev/pci ioctl and drop the reference count in the destructor hook for that object. -- John Baldwin _______________________________________________ firstname.lastname@example.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"