On Saturday, October 20, 2018 10:14 AM, David Gwynne <da...@gwynne.id.au> wrote:
> > On 20 Oct 2018, at 11:56 am, Joseph Mayer joseph.ma...@protonmail.com wrote: > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Friday, October 19, 2018 5:15 PM, Mark Kettenis mark.kette...@xs4all.nl > > wrote: > > > > > > Date: Fri, 19 Oct 2018 10:22:30 +1000 > > > > From: David Gwynne da...@gwynne.id.au > > > > On Wed, May 10, 2017 at 10:09:59PM +1000, David Gwynne wrote: > > > > > > > > > On Mon, May 08, 2017 at 11:03:58AM +1000, David Gwynne wrote: > > > > > > > > > > > on modern sparc64s (think fire or sparc enterprise Mx000 boxes), > > > > > > setting up and tearing down the translation table entries (TTEs) > > > > > > is very expensive. so expensive that the cost of doing it for disk > > > > > > io has a noticable impact on compile times. > > > > > > now that there's a BUS_DMA_64BIT flag, we can use that to decide > > > > > > to bypass the iommu for devices that set that flag, therefore > > > > > > avoiding the cost of handling the TTEs. > > > > Question for the unintroduced, what's the scope here, TTE is Sparc's > > page table and reconfiguring them at (process) context switch is > > expensive and this suggestion removes the need for TTE:s for hardware > > device access, but those don't change at context switch? > > We're talking about an IOMMU here, not a traditional MMU providing virtual > addresses for programs. An IOMMU sits between physical memory and the devices > in a machine. It allows DMA addresses to mapped to different parts of > physical memory. Mapping physical memory to a DMA virtual address (or dva) is > how a device that only understands 32bit addresses can work in a 64bit > machine. Memory at high addresses gets mapped to a low dva. > > This is done at runtime on OpenBSD when DMA mappings are loaded or unloaded > by populating Translation Table Entries (TTEs). A TTE is effectively a table > or array mapping DVA pages to physical addresses. Generally device drivers > load and unload dma memory for every I/O or packet or so on. > > IOMMUs in sparc64s have some more features than this. Because they really are > between memory and the devices they can act as a gatekeeper for all memory > accesses. They also have a toggle that can allow a device to have direct or > passthru access to physical memory. If passthru is enabled, there's a special > address range that effectively maps all physical memory into a DVA range. > Devices can be pointed at it without having to manage TTEs. When passthru is > disabled, all accesses must go through TTEs. > > Currently OpenBSD disables passthru. The benefit is devices can't blindly > access sensitive memory unless it is explicitly shared. Note that this is how > it is on most architectures anyway. However, the consequence of managing the > TTEs is that it is expensive, and extremely so in some cases. > > dlg Last iteration from me on this one. Why is this not a problem on some other architectures? I'd have thought DMA and hardware being assigned transitory addresses (from memory allocator or other OS subsystem or driver) mostly is a lower level phenomenon and memcpy normally applies on higher levels, isn't it so - for networking for instance, mbuf's take over soon above the driver level. Does OpenBSD have a pool of to-be-mbufs and it asks network drivers to write received ethernet frames directly to them, and similarly transmit ethernet frames directly from mbuf:s? What potentially or clearly sensitive memory would passthru expose, driver-owned structures only or all memory?
