Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-22 Thread Jason Gunthorpe
On Thu, Apr 22, 2021 at 08:34:32AM +, Tian, Kevin wrote: > The shim layer could be considered as a new iommu backend in VFIO, > which connects VFIO iommu ops to the internal helpers in > drivers/ioasid. It may be the best we can do because of SPAPR, but the ideal outcome should be to remove

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-21 Thread Jason Gunthorpe
On Wed, Apr 21, 2021 at 01:33:12PM -0600, Alex Williamson wrote: > > I still expect that VFIO_GROUP_SET_CONTAINER will be used to connect > > /dev/{ioasid,vfio} to the VFIO group and all the group and device > > logic stays inside VFIO. > > But that group and device logic is also tied to the

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-21 Thread Jason Gunthorpe
On Wed, Apr 21, 2021 at 10:54:51AM -0600, Alex Williamson wrote: > That's essentially replacing vfio-core, where I think we're more I am only talking about /dev/vfio here which is basically the IOMMU interface part. I still expect that VFIO_GROUP_SET_CONTAINER will be used to connect

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-21 Thread Jason Gunthorpe
On Wed, Apr 21, 2021 at 01:18:07PM +, Liu, Yi L wrote: > > Ideally this new /dev/ioasid interface, and making use of it as a VFIO > > IOMMU backend, should replace type1. > > yeah, just a double check, I think this also requires a new set of uAPIs > (e.g. new MAP/UNMAP), which means the

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-16 Thread Jason Gunthorpe
On Fri, Apr 16, 2021 at 10:23:32AM -0700, Jacob Pan wrote: > Perhaps similar to cgroup v1 vs v2, it took a long time and with known > limitations in v1. cgroup v2 is still having transition problems, if anything it is a cautionary tale to think really hard about uAPI because transitioning can be

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-16 Thread Jason Gunthorpe
On Fri, Apr 16, 2021 at 04:26:19PM +0200, Auger Eric wrote: > This was largely done during several confs including plumber, KVM forum, > for several years. Also API docs were shared on the ML. I don't remember > any voice was raised at those moments. I don't think anyone objects to the high

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-16 Thread Jason Gunthorpe
On Fri, Apr 16, 2021 at 03:38:02PM +0200, Auger Eric wrote: > The redesign requirement came pretty late in the development process. > The iommu user API is upstream for a while, the VFIO interfaces have > been submitted a long time ago and under review for a bunch of time. > Redesigning

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-15 Thread Jason Gunthorpe
On Thu, Apr 15, 2021 at 03:11:19PM +0200, Auger Eric wrote: > Hi Jason, > > On 4/1/21 6:03 PM, Jason Gunthorpe wrote: > > On Thu, Apr 01, 2021 at 02:08:17PM +, Liu, Yi L wrote: > > > >> DMA page faults are delivered to root-complex via page request message and &

Re: [PATCH 2/2] iommu/sva: Remove mm parameter from SVA bind API

2021-04-15 Thread Jason Gunthorpe
On Thu, Apr 15, 2021 at 01:33:24PM +0800, Lu Baolu wrote: > Hi Jason, > > On 4/14/21 7:26 PM, Jason Gunthorpe wrote: > > On Wed, Apr 14, 2021 at 02:22:09PM +0800, Lu Baolu wrote: > > > > > I still worry about supervisor pasid allocation. > > > > > &g

Re: [PATCH 2/2] iommu/sva: Remove mm parameter from SVA bind API

2021-04-14 Thread Jason Gunthorpe
On Wed, Apr 14, 2021 at 02:22:09PM +0800, Lu Baolu wrote: > I still worry about supervisor pasid allocation. > > If we use iommu_sva_alloc_pasid() to allocate a supervisor pasid, which > mm should the pasid be set? I've ever thought about passing _mm to > iommu_sva_alloc_pasid(). But if you add

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-08 Thread Jason Gunthorpe
On Wed, Apr 07, 2021 at 11:50:02PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, April 7, 2021 8:21 PM > > > > On Wed, Apr 07, 2021 at 02:08:33AM +, Tian, Kevin wrote: > > > > > > Because if you don't then we

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-07 Thread Jason Gunthorpe
On Wed, Apr 07, 2021 at 08:43:50PM +0200, Jean-Philippe Brucker wrote: > * Get a container handle out of /dev/ioasid (or /dev/iommu, really.) > No operation available since we don't know what the device and IOMMU > capabilities are. > > * Attach the handle to a VF. With VFIO that would be >

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-07 Thread Jason Gunthorpe
On Wed, Apr 07, 2021 at 02:08:33AM +, Tian, Kevin wrote: > > Because if you don't then we enter insane world where a PASID is being > > created under /dev/ioasid but its translation path flows through setup > > done by VFIO and the whole user API becomes an incomprehensible mess. > > > > How

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-07 Thread Jason Gunthorpe
On Wed, Apr 07, 2021 at 08:17:50AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, April 6, 2021 8:43 PM > > > > On Tue, Apr 06, 2021 at 09:35:17AM +0800, Jason Wang wrote: > > > > > > VFIO and VDPA has no buisness having map/unma

Re: [PATCH v8 7/9] vfio/mdev: Add iommu related member in mdev_device

2021-04-07 Thread Jason Gunthorpe
On Wed, Apr 07, 2021 at 09:58:05AM +0800, Lu Baolu wrote: > I've ever tried to implement a bus iommu_ops for mdev devices. > > https://lore.kernel.org/lkml/20201030045809.957927-1-baolu...@linux.intel.com/ > > Any comments? You still have the symbol_get, so something continues to be wrong with

Re: [PATCH v8 7/9] vfio/mdev: Add iommu related member in mdev_device

2021-04-06 Thread Jason Gunthorpe
On Mon, Mar 25, 2019 at 09:30:34AM +0800, Lu Baolu wrote: > A parent device might create different types of mediated > devices. For example, a mediated device could be created > by the parent device with full isolation and protection > provided by the IOMMU. One usage case could be found on >

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-06 Thread Jason Gunthorpe
On Tue, Apr 06, 2021 at 09:35:17AM +0800, Jason Wang wrote: > > VFIO and VDPA has no buisness having map/unmap interfaces once we have > > /dev/ioasid. That all belongs in the iosaid side. > > > > I know they have those interfaces today, but that doesn't mean we have > > to keep using them for

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-06 Thread Jason Gunthorpe
On Tue, Apr 06, 2021 at 01:27:15AM +, Tian, Kevin wrote: > > and here is one example why using existing VFIO/VDPA interface makes > sense. say dev1 (w/ sva) and dev2 (w/o sva) are placed in a single VFIO > container. Forget about SVA, it is an irrelevant detail of how a PASID is

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-06 Thread Jason Gunthorpe
On Tue, Apr 06, 2021 at 01:02:05AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, April 6, 2021 7:40 AM > > > > On Fri, Apr 02, 2021 at 07:58:02AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > > > Sent: Thursd

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-06 Thread Jason Gunthorpe
On Tue, Apr 06, 2021 at 12:37:35AM +, Tian, Kevin wrote: > With nested translation it is GVA->GPA->HPA. The kernel needs to > fix fault related to GPA->HPA (managed by VFIO/VDPA) while > handle_mm_fault only handles HVA->HPA. In this case, the 2nd-level > page fault is expected to be

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-05 Thread Jason Gunthorpe
On Fri, Apr 02, 2021 at 08:22:28AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, March 30, 2021 9:29 PM > > > > > > > > First, userspace may use ioasid in a non-SVA scenario where ioasid is > > > bound to specific securit

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-05 Thread Jason Gunthorpe
On Fri, Apr 02, 2021 at 07:58:02AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, April 1, 2021 9:47 PM > > > > On Thu, Apr 01, 2021 at 01:43:36PM +, Liu, Yi L wrote: > > > > From: Jason Gunthorpe > > > > Sent: Thursd

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-05 Thread Jason Gunthorpe
On Fri, Apr 02, 2021 at 07:30:23AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Friday, April 2, 2021 12:04 AM > > > > On Thu, Apr 01, 2021 at 02:08:17PM +, Liu, Yi L wrote: > > > > > DMA page faults are delivered t

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 10:23:55AM -0700, Jacob Pan wrote: > Hi Jason, > > On Wed, 31 Mar 2021 21:37:05 -0300, Jason Gunthorpe wrote: > > > On Wed, Mar 31, 2021 at 04:46:21PM -0700, Jacob Pan wrote: > > > Hi Jason, > > > > > > On Wed, 31 Mar 2021

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 02:08:17PM +, Liu, Yi L wrote: > DMA page faults are delivered to root-complex via page request message and > it is per-device according to PCIe spec. Page request handling flow is: > > 1) iommu driver receives a page request from device > 2) iommu driver parses the

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 01:43:36PM +, Liu, Yi L wrote: > > From: Jason Gunthorpe > > Sent: Thursday, April 1, 2021 9:16 PM > > > > On Thu, Apr 01, 2021 at 01:10:48PM +, Liu, Yi L wrote: > > > > From: Jason Gunthorpe > > > > Sent: Thursda

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 01:38:46PM +, Liu, Yi L wrote: > > From: Jean-Philippe Brucker > > Sent: Thursday, April 1, 2021 8:05 PM > [...] > > > > Also wondering about: > > > > * Querying IOMMU nesting capabilities before binding page tables (which > > page table formats are supported?). We

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 01:10:48PM +, Liu, Yi L wrote: > > From: Jason Gunthorpe > > Sent: Thursday, April 1, 2021 7:47 PM > [...] > > I'm worried Intel views the only use of PASID in a guest is with > > ENQCMD, but that is not consistent with the industry. We ne

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 02:05:00PM +0200, Jean-Philippe Brucker wrote: > On Thu, Apr 01, 2021 at 07:04:01AM +, Liu, Yi L wrote: > > > - how about AMD and ARM's vSVA support? Their PASID allocation and page > > > table > > > happens within guest. They only need to bind the guest PASID table

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 07:04:01AM +, Liu, Yi L wrote: > After reading your reply in > https://lore.kernel.org/linux-iommu/20210331123801.gd1463...@nvidia.com/#t > So you mean /dev/ioasid FD is per-VM instead of per-ioasid, so above skeleton > doesn't suit your idea. You can do it one PASID

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-04-01 Thread Jason Gunthorpe
On Thu, Apr 01, 2021 at 04:38:44AM +, Liu, Yi L wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, March 31, 2021 8:41 PM > > > > On Wed, Mar 31, 2021 at 07:38:36AM +, Liu, Yi L wrote: > > > > > The reason is /dev/ioasid FD is per-VM since the

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-31 Thread Jason Gunthorpe
On Wed, Mar 31, 2021 at 04:46:21PM -0700, Jacob Pan wrote: > Hi Jason, > > On Wed, 31 Mar 2021 09:38:01 -0300, Jason Gunthorpe wrote: > > > > > Get rid of the ioasid set. > > > > > > > > Each driver has its own list of allowed ioasids. &

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-31 Thread Jason Gunthorpe
On Wed, Mar 31, 2021 at 11:20:30AM -0700, Jacob Pan wrote: > Hi Jason, > > On Wed, 31 Mar 2021 14:31:48 -0300, Jason Gunthorpe wrote: > > > > > We should try to avoid hidden behind the scenes kernel > > > > interconnections between subsystems. > >

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-31 Thread Jason Gunthorpe
On Wed, Mar 31, 2021 at 09:34:57AM -0700, Jacob Pan wrote: > "3.3 PASID translation > To support PASID isolation for Shared Work Queues used by VMs, the CPU must > provide a way for the PASID to be communicated to the device in the DMWr > transaction. On Intel CPUs, the CPU provides a PASID

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-31 Thread Jason Gunthorpe
On Wed, Mar 31, 2021 at 07:38:36AM +, Liu, Yi L wrote: > The reason is /dev/ioasid FD is per-VM since the ioasid allocated to > the VM should be able to be shared by all assigned device for the VM. > But the SVA operations (bind/unbind page table, cache_invalidate) should > be per-device. It

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-31 Thread Jason Gunthorpe
On Wed, Mar 31, 2021 at 07:41:40AM +, Liu, Yi L wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, March 30, 2021 9:28 PM > > > > On Tue, Mar 30, 2021 at 04:14:58AM +, Tian, Kevin wrote: > > > > > One correction. The mdev should still const

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-31 Thread Jason Gunthorpe
On Tue, Mar 30, 2021 at 05:10:41PM -0700, Jacob Pan wrote: > Hi Jason, > > On Tue, 30 Mar 2021 10:43:13 -0300, Jason Gunthorpe wrote: > > > > If two mdevs from the same PF dev are assigned to two VMs, the PASID > > > table will be shared. IOASID set ensures

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-30 Thread Jason Gunthorpe
On Tue, Mar 30, 2021 at 03:42:24PM +0200, Jean-Philippe Brucker wrote: > On Tue, Mar 30, 2021 at 10:07:55AM -0300, Jason Gunthorpe wrote: > > On Fri, Mar 26, 2021 at 09:06:42AM +0100, Jean-Philippe Brucker wrote: > > > > > It's not inconceivable to have a contro

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-30 Thread Jason Gunthorpe
On Mon, Mar 29, 2021 at 03:55:26PM -0700, Jacob Pan wrote: > In one of the earlier discussions, I was made aware of some use cases (by > AMD, iirc) where PASID can be used w/o IOMMU. That is why I tried to keep > ioasid a separate subsystem. Other than that, I don't see an issue > combining the

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-30 Thread Jason Gunthorpe
On Tue, Mar 30, 2021 at 01:37:05AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, March 30, 2021 12:32 AM > > > > On Wed, Mar 24, 2021 at 12:05:28PM -0700, Jacob Pan wrote: > > > > > > IMHO a use created PASID is either bound

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-30 Thread Jason Gunthorpe
On Tue, Mar 30, 2021 at 04:14:58AM +, Tian, Kevin wrote: > One correction. The mdev should still construct the list of allowed PASID's as > you said (by listening to IOASID_BIND/UNBIND event), in addition to the > ioasid > set maintained per VM (updated when a PASID is allocated/freed). The

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-30 Thread Jason Gunthorpe
On Tue, Mar 30, 2021 at 02:24:09AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, March 30, 2021 12:32 AM > > > In terms of usage for guest SVA, an ioasid_set is mostly tied to a host > > > mm, > > > the use case is a

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-30 Thread Jason Gunthorpe
On Fri, Mar 26, 2021 at 09:06:42AM +0100, Jean-Philippe Brucker wrote: > It's not inconceivable to have a control queue doing DMA tagged with > PASID. The devices I know either use untagged DMA, or have a choice to use > a PASID. I don't think we should encourage that. A PASID and all the

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-29 Thread Jason Gunthorpe
On Wed, Mar 24, 2021 at 12:05:28PM -0700, Jacob Pan wrote: > > IMHO a use created PASID is either bound to a mm (current) at creation > > time, or it will never be bound to a mm and its page table is under > > user control via /dev/ioasid. > > > True for PASID used in native SVA bind. But for

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-25 Thread Jason Gunthorpe
On Thu, Mar 25, 2021 at 10:02:36AM -0700, Jacob Pan wrote: > Hi Jean-Philippe, > > On Thu, 25 Mar 2021 11:21:40 +0100, Jean-Philippe Brucker > wrote: > > > On Wed, Mar 24, 2021 at 03:12:30PM -0700, Jacob Pan wrote: > > > Hi Jason, > > > > > > On

Re: [RFC PATCH v2 04/11] PCI/P2PDMA: Introduce pci_p2pdma_should_map_bus() and pci_p2pdma_bus_offset()

2021-03-24 Thread Jason Gunthorpe
On Mon, Mar 15, 2021 at 10:27:08AM -0600, Logan Gunthorpe wrote: > In this case the WARN_ON is just to guard against misuse of the > function. It should never happen unless a developer changes the code in > a way that is incorrect. So I think that's the correct use of WARN_ON. > Though I might

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-24 Thread Jason Gunthorpe
On Wed, Mar 24, 2021 at 10:02:46AM -0700, Jacob Pan wrote: > > Also wondering about device driver allocating auxiliary domains for their > > private use, to do iommu_map/unmap on private PASIDs (a clean replacement > > to super SVA, for example). Would that go through the same path as > >

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-22 Thread Jason Gunthorpe
On Fri, Mar 19, 2021 at 11:22:21AM -0700, Jacob Pan wrote: > Hi Jason, > > On Fri, 19 Mar 2021 10:54:32 -0300, Jason Gunthorpe wrote: > > > On Fri, Mar 19, 2021 at 02:41:32PM +0100, Jean-Philippe Brucker wrote: > > > On Fri, Mar 19, 2021 at 09:46:45AM -

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-19 Thread Jason Gunthorpe
On Fri, Mar 19, 2021 at 02:41:32PM +0100, Jean-Philippe Brucker wrote: > On Fri, Mar 19, 2021 at 09:46:45AM -0300, Jason Gunthorpe wrote: > > On Fri, Mar 19, 2021 at 10:58:41AM +0100, Jean-Philippe Brucker wrote: > > > > > Although there is no use for it at the moment

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-03-19 Thread Jason Gunthorpe
On Fri, Mar 19, 2021 at 10:58:41AM +0100, Jean-Philippe Brucker wrote: > Although there is no use for it at the moment (only two upstream users and > it looks like amdkfd always uses current too), I quite like the > client-server model where the privileged process does bind() and programs > the

Re: [RFC PATCH 18/18] ioasid: Add /dev/ioasid for userspace

2021-03-12 Thread Jason Gunthorpe
On Thu, Mar 11, 2021 at 02:55:34PM -0800, Jacob Pan wrote: > Hi Jason, > > Thanks for the review. > > On Wed, 10 Mar 2021 15:23:01 -0400, Jason Gunthorpe wrote: > > > On Sat, Feb 27, 2021 at 02:01:26PM -0800, Jacob Pan wrote: > > > > > +/*

Re: [RFC PATCH v2 11/11] nvme-pci: Convert to using dma_map_sg for p2pdma pages

2021-03-11 Thread Jason Gunthorpe
On Thu, Mar 11, 2021 at 04:31:41PM -0700, Logan Gunthorpe wrote: > Convert to using dma_[un]map_sg() for PCI p2pdma pages. > > This should be equivalent, though support will be somewhat less > (only dma-direct and dma-iommu are currently supported). > > Signed-off-by: Logan Gunthorpe >

Re: [RFC PATCH 18/18] ioasid: Add /dev/ioasid for userspace

2021-03-10 Thread Jason Gunthorpe
On Sat, Feb 27, 2021 at 02:01:26PM -0800, Jacob Pan wrote: > +/* IOCTLs for IOASID file descriptor (/dev/ioasid) */ > + > +/** > + * IOASID_GET_API_VERSION - _IO(IOASID_TYPE, IOASID_BASE + 0) > + * > + * Report the version of the IOASID API. This allows us to bump the entire >

Re: [RFC PATCH 15/18] cgroup: Introduce ioasids controller

2021-03-04 Thread Jason Gunthorpe
On Thu, Mar 04, 2021 at 11:01:44AM -0800, Jacob Pan wrote: > > For something like qemu I'd expect to put the qemu process in a cgroup > > with 1 PASID. Who cares what qemu uses the PASID for, or how it was > > allocated? > > For vSVA, we will need one PASID per guest process. But that is up to

Re: [RFC PATCH 15/18] cgroup: Introduce ioasids controller

2021-03-04 Thread Jason Gunthorpe
On Thu, Mar 04, 2021 at 09:46:03AM -0800, Jacob Pan wrote: > Right, I was assuming have three use cases of IOASIDs: > 1. host supervisor SVA (not a concern, just one init_mm to bind) > 2. host user SVA, either one IOASID per process or perhaps some private > IOASID for private address space > 3.

Re: [Patch v8 04/10] vfio/type1: Support binding guest page tables to PASID

2021-03-04 Thread Jason Gunthorpe
On Thu, Mar 04, 2021 at 07:20:22AM +, Liu, Yi L wrote: > > > However, IOMMU is a system device which has little value to be exposed > > to > > > the userspace. Not to mention the device-IOMMU affinity/topology. VFIO > > > nicely abstracts IOMMU from the userspace, why do we want to reverse > >

Re: [RFC PATCH 15/18] cgroup: Introduce ioasids controller

2021-03-03 Thread Jason Gunthorpe
On Wed, Mar 03, 2021 at 04:02:05PM -0800, Jacob Pan wrote: > > The interface definitely can be reused. But IOASID has a different > > behavior in terms of migration and ownership checking. I guess SEV key > > IDs are not tied to a process whereas IOASIDs are. Perhaps this can be > > solved by

Re: [Patch v8 04/10] vfio/type1: Support binding guest page tables to PASID

2021-03-03 Thread Jason Gunthorpe
On Wed, Mar 03, 2021 at 11:42:12AM -0800, Jacob Pan wrote: > Hi Jason, > > On Tue, 2 Mar 2021 13:15:51 -0400, Jason Gunthorpe wrote: > > > On Tue, Mar 02, 2021 at 09:13:19AM -0800, Jacob Pan wrote: > > > Hi Jason, > > > > > > On Tue, 2 Mar 2021

Re: [Patch v8 04/10] vfio/type1: Support binding guest page tables to PASID

2021-03-02 Thread Jason Gunthorpe
On Tue, Mar 02, 2021 at 09:13:19AM -0800, Jacob Pan wrote: > Hi Jason, > > On Tue, 2 Mar 2021 08:56:28 -0400, Jason Gunthorpe wrote: > > > On Wed, Mar 03, 2021 at 04:35:39AM +0800, Liu Yi L wrote: > > > > > > +static int vfio_dev_bind_gpas

Re: [Patch v8 04/10] vfio/type1: Support binding guest page tables to PASID

2021-03-02 Thread Jason Gunthorpe
On Wed, Mar 03, 2021 at 04:35:39AM +0800, Liu Yi L wrote: > > +static int vfio_dev_bind_gpasid_fn(struct device *dev, void *data) > +{ > + struct domain_capsule *dc = (struct domain_capsule *)data; > + unsigned long arg = *(unsigned long *)dc->data; > + > + return

Re: [Patch v8 03/10] vfio/type1: Report iommu nesting info to userspace

2021-03-02 Thread Jason Gunthorpe
On Wed, Mar 03, 2021 at 04:35:38AM +0800, Liu Yi L wrote: > diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c > index 4bb162c1d649..3a5c84d4f19b 100644 > +++ b/drivers/vfio/vfio_iommu_type1.c > @@ -63,22 +63,24 @@ MODULE_PARM_DESC(dma_entry_limit, >

Re: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory pin

2021-02-10 Thread Jason Gunthorpe
On Tue, Feb 09, 2021 at 10:22:47PM +, Song Bao Hua (Barry Song) wrote: > The problem is that SVA declares we can use any memory of a process > to do I/O. And in real scenarios, we are unable to customize most > applications to make them use the pool. So we are looking for some > extension

Re: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory pin

2021-02-09 Thread Jason Gunthorpe
On Tue, Feb 09, 2021 at 03:01:42AM +, Song Bao Hua (Barry Song) wrote: > On the other hand, wouldn't it be the benefit of hardware accelerators > to have a lower and more stable latency zip/encryption than CPU? No, I don't think so. If this is an important problem then it should apply

Re: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory pin

2021-02-08 Thread Jason Gunthorpe
On Mon, Feb 08, 2021 at 08:35:31PM +, Song Bao Hua (Barry Song) wrote: > > > > From: Jason Gunthorpe [mailto:j...@ziepe.ca] > > Sent: Tuesday, February 9, 2021 7:34 AM > > To: David Hildenbrand > > Cc: Wangzhou (B) ; linux-ker...@vger.kernel.org; > > iom

Re: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory pin

2021-02-08 Thread Jason Gunthorpe
On Mon, Feb 08, 2021 at 09:14:28AM +0100, David Hildenbrand wrote: > People are constantly struggling with the effects of long term pinnings > under user space control, like we already have with vfio and RDMA. > > And here we are, adding yet another, easier way to mess with core MM in the > same

Re: [PATCH 11/12] platform-msi: Add platform check for subdevice irq domain

2021-02-04 Thread Jason Gunthorpe
On Wed, Feb 03, 2021 at 12:56:44PM -0800, Megha Dey wrote: > +bool arch_support_pci_device_ims(struct pci_dev *pdev) > +{ Consistent language please, we are not using IMS elsewhere, this feature is called device_msi in Linux. Jason ___ iommu mailing

Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device

2021-02-01 Thread Jason Gunthorpe
On Fri, Jan 29, 2021 at 10:09:03AM +, Tian, Kevin wrote: > > SVA is not doom to work with IO page fault only. If we have SVA+pin, > > we would get both sharing address and stable I/O latency. > > Isn't it like a traditional MAP_DMA API (imply pinning) plus specifying > cpu_va of the memory

Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device

2021-01-26 Thread Jason Gunthorpe
On Tue, Jan 26, 2021 at 01:26:45AM +, Song Bao Hua (Barry Song) wrote: > > On Mon, Jan 25, 2021 at 11:35:22PM +, Song Bao Hua (Barry Song) wrote: > > > > > > On Mon, Jan 25, 2021 at 10:21:14PM +, Song Bao Hua (Barry Song) > > > > wrote: > > > > > mlock, while certainly be able to

Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 11:35:22PM +, Song Bao Hua (Barry Song) wrote: > > On Mon, Jan 25, 2021 at 10:21:14PM +, Song Bao Hua (Barry Song) wrote: > > > mlock, while certainly be able to prevent swapping out, it won't > > > be able to stop page moving due to: > > > * memory compaction in

Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 10:21:14PM +, Song Bao Hua (Barry Song) wrote: > mlock, while certainly be able to prevent swapping out, it won't > be able to stop page moving due to: > * memory compaction in alloc_pages() > * making huge pages > * numa balance > * memory compaction in CMA Enabling

Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 04:34:56PM +0800, Zhou Wang wrote: > +static int uacce_pin_page(struct uacce_pin_container *priv, > + struct uacce_pin_address *addr) > +{ > + unsigned int flags = FOLL_FORCE | FOLL_WRITE; > + unsigned long first, last, nr_pages; > +

Re: [PATCH v10 12/13] iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind()

2020-11-25 Thread Jason Gunthorpe
On Wed, Nov 25, 2020 at 10:27:49AM +0100, Jean-Philippe Brucker wrote: > > I'm strongly > > trying to discourage static lists matching mm's like smmu_mn is > > doing. This is handled by the core code, don't open code it.. > > We discussed this at v6, which wonkily stored the mn ops in the domain

Re: [PATCH v10 12/13] iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind()

2020-11-24 Thread Jason Gunthorpe
On Fri, Sep 18, 2020 at 12:18:52PM +0200, Jean-Philippe Brucker wrote: > +/* Allocate or get existing MMU notifier for this {domain, mm} pair */ > +static struct arm_smmu_mmu_notifier * > +arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, > + struct mm_struct

Re: remove dma_virt_ops v2

2020-11-17 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 07:19:31PM +0100, Christoph Hellwig wrote: > Hi Jason, > > this series switches the RDMA core to opencode the special case of > devices bypassing the DMA mapping in the RDMA ULPs. The virt ops > have caused a bit of trouble due to the P2P code node working with > them due

Re: iommu/vt-d: Cure VF irqdomain hickup

2020-11-16 Thread Jason Gunthorpe
t which needs quite some other > changes to the way how x86 manages PCI and MSI domains. > > Fixes: 85a8dfc57a0b ("iommm/vt-d: Store irq domain in struct device") > Reported-by: Jason Gunthorpe > Signed-off-by: Thomas Gleixner > --- > drivers/iommu/intel/dmar.c |

Re: remove dma_virt_ops v2

2020-11-12 Thread Jason Gunthorpe
On Thu, Nov 12, 2020 at 06:09:56PM +0100, Christoph Hellwig wrote: > On Thu, Nov 12, 2020 at 12:59:35PM -0400, Jason Gunthorpe wrote: > > RMDA/sw: Don't allow drivers using dma_virt_ops on highmem configs > > I think this one actually is something needed in 5.10 and -stable.

Re: remove dma_virt_ops v2

2020-11-12 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 07:19:31PM +0100, Christoph Hellwig wrote: > Hi Jason, > > this series switches the RDMA core to opencode the special case of > devices bypassing the DMA mapping in the RDMA ULPs. The virt ops > have caused a bit of trouble due to the P2P code node working with > them due

Re: remove dma_virt_ops v2

2020-11-12 Thread Jason Gunthorpe
On Thu, Nov 12, 2020 at 10:40:30AM +0100, Christoph Hellwig wrote: > ping? > > On Fri, Nov 06, 2020 at 07:19:31PM +0100, Christoph Hellwig wrote: > > Hi Jason, > > > > this series switches the RDMA core to opencode the special case of > > devices bypassing the DMA mapping in the RDMA ULPs. The

REGRESSION: Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI

2020-11-12 Thread Jason Gunthorpe
On Wed, Aug 26, 2020 at 01:16:28PM +0200, Thomas Gleixner wrote: > This is the second version of providing a base to support device MSI (non > PCI based) and on top of that support for IMS (Interrupt Message Storm) > based devices in a halfways architecture independent way. Hi Thomas, Our test

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 01:03:26PM -0700, Logan Gunthorpe wrote: > I don't think a function like that will work for the p2pmem use case. In > order to implement proper page freeing I expect I'll need a loop around > the allocator and vm_insert_mixed()... Something roughly like: > > for (addr =

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 12:44:59PM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 12:30 p.m., Jason Gunthorpe wrote: > >> I certainly can't make decisions for code that isn't currently > >> upstream. > > > > The rdma drivers are all upstream, what are

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 11:20:05AM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 11:09 a.m., Jason Gunthorpe wrote: > >> Ah, hmm, yes. I guess the pages have to be hooked and returned to the > >> genalloc through free_devmap_managed_page(). > > > > Th

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:53:45AM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 10:42 a.m., Jason Gunthorpe wrote: > > On Fri, Nov 06, 2020 at 10:28:00AM -0700, Logan Gunthorpe wrote: > >> > >> > >> On 2020-11-06 10:22 a.m., Jason Gunthorpe wrote

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:28:00AM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 10:22 a.m., Jason Gunthorpe wrote: > > On Fri, Nov 06, 2020 at 10:00:35AM -0700, Logan Gunthorpe wrote: > >> Introduce pci_mmap_p2pmem() which is a helper to allocate and mm

Re: [RFC PATCH 15/15] nvme-pci: Allow mmaping the CMB in userspace

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:00:36AM -0700, Logan Gunthorpe wrote: > Allow userspace to obtain CMB memory by mmaping the controller's > char device. The mmap call allocates and returns a hunk of CMB memory, > (the offset is ignored) so userspace does not have control over the > address within the

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:00:35AM -0700, Logan Gunthorpe wrote: > Introduce pci_mmap_p2pmem() which is a helper to allocate and mmap > a hunk of p2pmem into userspace. > > Signed-off-by: Logan Gunthorpe > drivers/pci/p2pdma.c | 104 + >

Re: [PATCH 3/6] RDMA/core: remove use of dma_virt_ops

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 01:52:53PM -0400, Jason Gunthorpe wrote: > On Thu, Nov 05, 2020 at 08:42:02AM +0100, Christoph Hellwig wrote: > > @@ -1341,7 +1322,14 @@ int ib_register_device(struct ib_device *device, > > const char *name, > > if (ret) >

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 06:43:06PM +0100, Christoph Hellwig wrote: > On Thu, Nov 05, 2020 at 01:39:30PM -0400, Jason Gunthorpe wrote: > > Hmm. This works for real devices like mlx5, but it means the three SW > > devices will also resolve to a real PCI device that is not the

Re: [PATCH 3/6] RDMA/core: remove use of dma_virt_ops

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:02AM +0100, Christoph Hellwig wrote: > @@ -1341,7 +1322,14 @@ int ib_register_device(struct ib_device *device, const > char *name, > if (ret) > return ret; > > - setup_dma_device(device, dma_device); > + /* > + * If the caller does

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 06:29:21PM +0100, Christoph Hellwig wrote: > On Thu, Nov 05, 2020 at 01:23:57PM -0400, Jason Gunthorpe wrote: > > But that depends on the calling driver doing this properly, and we > > don't expose an API to get the PCI device of the struct ib_device >

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 06:08:16PM +0100, Christoph Hellwig wrote: > On Thu, Nov 05, 2020 at 10:34:18AM -0400, Jason Gunthorpe wrote: > > The check is removed here, but I didn't see a matching check added to > > the IB side? Something like: > > > > static int rdma_rw_m

Re: [PATCH 1/6] RMDA/sw: don't allow drivers using dma_virt_ops on highmem configs

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:00AM +0100, Christoph Hellwig wrote: > dma_virt_ops requires that all pages have a kernel virtual address. > Introduce a INFINIBAND_VIRT_DMA Kconfig symbol that depends on !HIGHMEM > and a large enough dma_addr_t, and make all three driver depend on the > new symbol.

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:03AM +0100, Christoph Hellwig wrote: > Now that all users of dma_virt_ops are gone we can remove the workaround > for it in the PCI peer to peer code. > > Signed-off-by: Christoph Hellwig > Reviewed-by: Logan Gunthorpe > Acked-by: Bjorn Helgaas >

Re: [PATCH 3/6] RDMA/core: remove use of dma_virt_ops

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:02AM +0100, Christoph Hellwig wrote: > diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h > index 5f8fd7976034e0..661e4a22b3cb81 100644 > +++ b/include/rdma/ib_verbs.h > @@ -3950,6 +3950,8 @@ static inline int ib_req_ncomp_notif(struct ib_cq *cq, > int

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-11-04 Thread Jason Gunthorpe
On Tue, Nov 03, 2020 at 08:14:29PM +0100, j...@8bytes.org wrote: > On Tue, Nov 03, 2020 at 01:48:51PM -0400, Jason Gunthorpe wrote: > > I think the same PCI driver with a small flag to support the PF or > > VF is not the same as two completely different drivers in different

Re: Re: [PATCH 2/5] RDMA/core: remove use of dma_virt_ops

2020-11-04 Thread Jason Gunthorpe
On Wed, Nov 04, 2020 at 03:09:04PM +, Bernard Metzler wrote: > lkey of zero to pass a physical buffer, only allowed for > kernel applications? Very nice idea I think. It already exists, it is called the local_dma_lkey, just set IB_DEVICE_LOCAL_DMA_LKEY and provide the value you want to use

Re: [PATCH 2/5] RDMA/core: remove use of dma_virt_ops

2020-11-04 Thread Jason Gunthorpe
On Wed, Nov 04, 2020 at 05:31:35PM +0100, Christoph Hellwig wrote: > On Wed, Nov 04, 2020 at 11:52:55AM -0400, Jason Gunthorpe wrote: > > It could work, I think a resonable ULP API would be to have some > > > > rdma_fill_ib_sge_from_sgl() > > rdma_map_sge_single

Re: [PATCH 2/5] RDMA/core: remove use of dma_virt_ops

2020-11-04 Thread Jason Gunthorpe
On Wed, Nov 04, 2020 at 03:01:08PM +0100, Christoph Hellwig wrote: > > Sigh. I think the proper fix is to replace addr/length with a > > scatterlist pointer in the struct ib_sge, then have SW drivers > > directly use the page pointer properly. > > The proper fix is to move the DMA mapping into

Re: [PATCH 2/5] RDMA/core: remove use of dma_virt_ops

2020-11-04 Thread Jason Gunthorpe
On Wed, Nov 04, 2020 at 10:50:49AM +0100, Christoph Hellwig wrote: > +int ib_dma_virt_map_sg(struct ib_device *dev, struct scatterlist *sg, int > nents) > +{ > + struct scatterlist *s; > + int i; > + > + for_each_sg(sg, s, nents, i) { > + sg_dma_address(s) =

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-11-03 Thread Jason Gunthorpe
On Tue, Nov 03, 2020 at 05:55:40PM +0100, j...@8bytes.org wrote: > On Tue, Nov 03, 2020 at 11:22:23AM -0400, Jason Gunthorpe wrote: > > This whole thread was brought up by IDXD which has a SVA driver and > > now wants to add a vfio-mdev driver too. SVA devices that want to be >

<    4   5   6   7   8   9   10   11   >