Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-14 Thread Jason Gunthorpe
On Mon, Nov 14, 2022 at 10:21:50AM -0500, Matthew Rosato wrote: > On 11/14/22 9:59 AM, Jason Gunthorpe wrote: > > On Mon, Nov 14, 2022 at 09:55:21AM -0500, Matthew Rosato wrote: > >>>> AFAICT there is no equivalent means to specify > >>>> vfio_iommu_type1.dm

Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-14 Thread Jason Gunthorpe
On Mon, Nov 14, 2022 at 09:55:21AM -0500, Matthew Rosato wrote: > >> AFAICT there is no equivalent means to specify > >> vfio_iommu_type1.dma_entry_limit when using iommufd; looks like > >> we'll just always get the default 65535. > > > > No, there is no arbitary limit on iommufd > > Yeah,

Re: [PATCH v2 11/11] iommufd: Allow iommufd to supply /dev/vfio/vfio

2022-11-14 Thread Jason Gunthorpe
On Fri, Nov 11, 2022 at 12:16:02PM +0800, Yi Liu wrote: > > +#if IS_ENABLED(CONFIG_IOMMUFD_VFIO_CONTAINER) > > +MODULE_ALIAS_MISCDEV(VFIO_MINOR); > > +MODULE_ALIAS("devname:vfio/vfio"); > > will this line also result in systemd to create this devnodes at boot > based on the module info even if

Re: [PATCH v2 07/11] vfio-iommufd: Support iommufd for physical VFIO devices

2022-11-14 Thread Jason Gunthorpe
On Fri, Nov 11, 2022 at 12:12:36PM +0800, Yi Liu wrote: > > +int vfio_iommufd_bind(struct vfio_device *vdev, struct iommufd_ctx *ictx) > > +{ > > + u32 ioas_id; > > + u32 device_id; > > + int ret; > > + > > + lockdep_assert_held(>dev_set->lock); > > + > > + /* > > +* If the driver

Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-14 Thread Jason Gunthorpe
On Mon, Nov 14, 2022 at 08:51:58PM +0800, Yi Liu wrote: > Our side, Yu He, Lixiao Yang has done below tests on Intel platform with > the above kernel, results are: > > 1) GVT-g test suit passed, Intel iGFx passthrough passed. > > 2) NIC passthrough test with different guest memory (1G/4G),

Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-14 Thread Jason Gunthorpe
On Thu, Nov 10, 2022 at 10:01:13PM -0500, Matthew Rosato wrote: > On 11/7/22 7:52 PM, Jason Gunthorpe wrote: > > This series provides an alternative container layer for VFIO implemented > > using iommufd. This is optional, if CONFIG_IOMMUFD is not set then it will >

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-10 Thread Jason Gunthorpe
On Wed, Nov 09, 2022 at 11:28:22AM -0700, Alex Williamson wrote: > > > > I'd be much more comfortable with this as a system wide iommufd flag > > > > if we also tied it to do some demonstration of privilege - eg a > > > > requirement to open iommufd with CAP_SYS_RAWIO for instance. > > > > > >

Re: [PATCH v2 10/11] vfio: Make vfio_container optionally compiled

2022-11-10 Thread Jason Gunthorpe
On Thu, Nov 10, 2022 at 06:57:57AM +, Tian, Kevin wrote: > > + /* > > +* Emulation for NOIMMU is imperfect in that VFIO blocks almost all > > +* other ioctls. We let them keep working but they mostly fail since no > > +* IOAS should exist. > > +*/ > > + if

Re: [PATCH v2 07/11] vfio-iommufd: Support iommufd for physical VFIO devices

2022-11-10 Thread Jason Gunthorpe
On Thu, Nov 10, 2022 at 03:11:16AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, November 8, 2022 8:53 AM > > > > + > > +int vfio_iommufd_bind(struct vfio_device *vdev, struct iommufd_ctx *ictx) > > +{ > > + u32 ioas_

Re: [PATCH v2 10/11] vfio: Make vfio_container optionally compiled

2022-11-09 Thread Jason Gunthorpe
On Wed, Nov 09, 2022 at 10:18:09AM -0700, Alex Williamson wrote: > DPDK supports no-iommu mode. Er? Huh? How? I thought no-iommu was for applications that didn't do DMA? How is DPDK getting packets in/out without DMA? I guess it snoops in /proc/ or something to learn PFNs of mlock'd memory? >

Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-09 Thread Jason Gunthorpe
On Tue, Nov 08, 2022 at 11:18:03PM +0800, Yi Liu wrote: > On 2022/11/8 17:19, Nicolin Chen wrote: > > On Mon, Nov 07, 2022 at 08:52:44PM -0400, Jason Gunthorpe wrote: > > > > > This is on github: > > > https://github.com/jgunthorpe/linux/commits/vfio_iommufd >

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-09 Thread Jason Gunthorpe
On Wed, Nov 09, 2022 at 03:21:29AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, November 9, 2022 9:05 AM > > > > On Tue, Nov 08, 2022 at 03:55:20PM -0700, Alex Williamson wrote: > > > > > > > So why exactly isn't thi

Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-09 Thread Jason Gunthorpe
On Wed, Nov 09, 2022 at 09:03:52AM +, Tian, Kevin wrote: > every mail in this series is shown thrice in lore: > > https://lore.kernel.org/all/0-v2-65016290f146+33e-vfio_iommufd_...@nvidia.com/ > > not sure what caused it but it's annoying to check the conversation there. It is sort of a

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-08 Thread Jason Gunthorpe
On Tue, Nov 08, 2022 at 03:55:20PM -0700, Alex Williamson wrote: > > > So why exactly isn't this an issue for VDPA? Are we just burying our > > > head in the sand that such platforms exists and can still be useful > > > given the appropriate risk vs reward trade-off? > > > > Simply that

Re: [PATCH v2 10/11] vfio: Make vfio_container optionally compiled

2022-11-08 Thread Jason Gunthorpe
On Tue, Nov 08, 2022 at 03:28:31PM -0700, Alex Williamson wrote: > Perhaps this should have been obvious, but I'm realizing that > vfio-noiommu mode is completely missing without VFIO_CONTAINER, which > seems a barrier to deprecating VFIO_CONTAINER and perhaps makes it a Yes, it is the same as

Re: [PATCH v2 07/11] vfio-iommufd: Support iommufd for physical VFIO devices

2022-11-08 Thread Jason Gunthorpe
On Tue, Nov 08, 2022 at 03:41:25PM +0800, Yi Liu wrote: > On 2022/11/8 14:10, Nicolin Chen wrote: > > On Mon, Nov 07, 2022 at 08:52:51PM -0400, Jason Gunthorpe wrote: > > > > > @@ -795,6 +800,10 @@ static int vfio_device_first_open(struct vfio_device > > > *de

Re: [PATCH v2 07/11] vfio-iommufd: Support iommufd for physical VFIO devices

2022-11-08 Thread Jason Gunthorpe
On Mon, Nov 07, 2022 at 10:10:59PM -0800, Nicolin Chen wrote: > > @@ -812,6 +821,7 @@ static int vfio_device_first_open(struct vfio_device > > *device) > > device->kvm = NULL; > > if (device->group->container) > > vfio_group_unuse_container(device->group); > > +

Re: [RFC PATCH v2 1/3] drivers/accel: define kconfig and register a new major

2022-11-08 Thread Jason Gunthorpe
On Tue, Nov 08, 2022 at 06:33:23AM +1000, Dave Airlie wrote: > At plumbers we decided a direction, I think the direction is good, if > there is refactoring to be done, I'd rather it was done in tree with a > clear direction. > > Coming in now and saying we should go down a different path isn't >

[PATCH v2 00/11] Connect VFIO to IOMMUFD

2022-11-07 Thread Jason Gunthorpe
- New patch to fix vfio modaliases when vfio container is disabled - Add a dmesg once when the iommufd provided /dev/vfio/vfio is opened to signal that iommufd is providing this v1: https://lore.kernel.org/r/0-v1-4991695894d8+211-vfio_iommufd_...@nvidia.com Jason Gunthorpe (11): vfio: Move

[PATCH v2 01/11] vfio: Move vfio_device driver open/close code to a function

2022-11-07 Thread Jason Gunthorpe
This error unwind is getting complicated. Move all the code into two pair'd function. The functions should be called when the open_count == 1 after incrementing/before decrementing. Reviewed-by: Kevin Tian Reviewed-by: Yi Liu Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers

[PATCH v2 05/11] vfio: Use IOMMU_CAP_ENFORCE_CACHE_COHERENCY for vfio_file_enforced_coherent()

2022-11-07 Thread Jason Gunthorpe
ill be connected to enforcing domains. Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/vfio/container.c | 5 +++-- drivers/vfio/vfio.h | 2 -- drivers/vfio/vfio_main.c | 29 - 3 files changed, 19 insertions(+), 17 deletions(-) diff --gi

[PATCH v2 07/11] vfio-iommufd: Support iommufd for physical VFIO devices

2022-11-07 Thread Jason Gunthorpe
by providing bind_iommufd/unbind_iommufd/attach_ioas callbacks to implement as required. The core code provides three default callbacks for physical mode using a real iommu_domain. This is suitable for drivers using vfio_register_group_dev() Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe

[PATCH v2 10/11] vfio: Make vfio_container optionally compiled

2022-11-07 Thread Jason Gunthorpe
the device/group and container have been created. Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/vfio/Kconfig | 35 +++ drivers/vfio/Makefile | 4 +-- drivers/vfio/vfio.h | 65 +++ 3 files changed, 91 insertions

[PATCH v2 02/11] vfio: Move vfio_device_assign_container() into vfio_device_first_open()

2022-11-07 Thread Jason Gunthorpe
be closed. Thus we do not need to check this on every device FD open, just the first. Reorganize the code so that only the first open and last close manages the container. Reviewed-by: Kevin Tian Reviewed-by: Yi Liu Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/vfio

[PATCH v2 08/11] vfio-iommufd: Support iommufd for emulated VFIO devices

2022-11-07 Thread Jason Gunthorpe
-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/kvmgt.c | 3 + drivers/s390/cio/vfio_ccw_ops.c | 3 + drivers/s390/crypto/vfio_ap_ops.c | 3 + drivers/vfio/container.c | 110 +--- drivers/vfio/iommufd.c| 58 +++ drivers/vfio/vfio.h

[PATCH v2 09/11] vfio: Move container related MODULE_ALIAS statements into container.c

2022-11-07 Thread Jason Gunthorpe
The miscdev is in container.c, so should these related MODULE_ALIAS statements. This is necessary for the next patch to be able to fully disable /dev/vfio/vfio. Fixes: cdc71fe4ecbf ("vfio: Move container code into drivers/vfio/container.c") Reported-by: "Liu, Yi L" Signed-off

[PATCH v2 04/11] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-07 Thread Jason Gunthorpe
allow_unsafe_interrupts is in the current state of the world. If vfio container is disabled then this option will not be available to the user. Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.h | 2 ++ drivers/vfio/vfio_iommu_type1.c | 5 ++--- drivers/vfio/vfio_main.c

[PATCH v2 03/11] vfio: Rename vfio_device_assign/unassign_container()

2022-11-07 Thread Jason Gunthorpe
These functions don't really assign anything anymore, they just increment some refcounts and do a sanity check. Call them vfio_group_[un]use_container() Reviewed-by: Kevin Tian Reviewed-by: Yi Liu Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/vfio/container.c | 14

[PATCH v2 06/11] vfio-iommufd: Allow iommufd to be used in place of a container fd

2022-11-07 Thread Jason Gunthorpe
. Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/vfio/Kconfig | 1 + drivers/vfio/container.c | 7 +++- drivers/vfio/vfio.h | 2 + drivers/vfio/vfio_main.c | 86 +--- 4 files changed, 80 insertions(+), 16 deletions(-) diff

[PATCH v2 11/11] iommufd: Allow iommufd to supply /dev/vfio/vfio

2022-11-07 Thread Jason Gunthorpe
-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/Kconfig | 12 drivers/iommu/iommufd/main.c | 36 +++ 2 files changed, 48 insertions(+) diff --git a/drivers/iommu/iommufd/Kconfig b/drivers/iommu/iommufd/Kconfig index 399a2edeaef6de

Re: [PATCH 06/10] vfio-iommufd: Allow iommufd to be used in place of a container fd

2022-11-07 Thread Jason Gunthorpe
On Wed, Nov 02, 2022 at 03:28:20PM +0800, Yi Liu wrote: > On 2022/10/26 02:50, Jason Gunthorpe wrote: > > This makes VFIO_GROUP_SET_CONTAINER accept both a vfio container FD and an > > iommufd. > > > > In iommufd mode an IOAS will exist after the SET_CONTAINER, but

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-07 Thread Jason Gunthorpe
On Mon, Nov 07, 2022 at 11:05:08AM -0700, Alex Williamson wrote: > After further consideration... I don't think the option on vfio-main > makes sense, basically for the same reason that the original option > existed on the IOMMU backend rather than vfio-core. The option > describes a means to

Re: [RFC PATCH v2 1/3] drivers/accel: define kconfig and register a new major

2022-11-07 Thread Jason Gunthorpe
On Mon, Nov 07, 2022 at 05:53:55PM +0200, Oded Gabbay wrote: > On Mon, Nov 7, 2022 at 4:10 PM Jason Gunthorpe wrote: > > > > On Mon, Nov 07, 2022 at 04:02:01PM +0200, Oded Gabbay wrote: > > > On Mon, Nov 7, 2022 at 3:10 PM Jason Gunthorpe wrote: > > > > > &

Re: [RFC PATCH v3 0/3] new subsystem for compute accelerator devices

2022-11-07 Thread Jason Gunthorpe
On Sun, Nov 06, 2022 at 11:02:22PM +0200, Oded Gabbay wrote: > Another important change is that I have reverted back to use IDR for minor > handling instead of xarray. This is because I have found that xarray doesn't > handle well the scenario where you allocate a NULL entry and then exchange it >

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-07 Thread Jason Gunthorpe
On Mon, Nov 07, 2022 at 08:18:53AM -0700, Alex Williamson wrote: > On Mon, 7 Nov 2022 09:19:43 -0400 > Jason Gunthorpe wrote: > > > On Mon, Oct 31, 2022 at 04:45:26PM -0600, Alex Williamson wrote: > > > > > > It is one idea, it depends how literal you want to

Re: [RFC PATCH v2 1/3] drivers/accel: define kconfig and register a new major

2022-11-07 Thread Jason Gunthorpe
On Mon, Nov 07, 2022 at 04:02:01PM +0200, Oded Gabbay wrote: > On Mon, Nov 7, 2022 at 3:10 PM Jason Gunthorpe wrote: > > > > On Mon, Nov 07, 2022 at 03:01:08PM +0200, Oded Gabbay wrote: > > > I don't agree with your statement that it should be "a layer over top o

Re: [PATCH 10/10] iommufd: Allow iommufd to supply /dev/vfio/vfio

2022-11-07 Thread Jason Gunthorpe
On Mon, Oct 31, 2022 at 04:53:11PM -0600, Alex Williamson wrote: > On Fri, 28 Oct 2022 15:44:36 -0300 > Jason Gunthorpe wrote: > > > On Wed, Oct 26, 2022 at 03:31:33PM -0600, Alex Williamson wrote: > > > On Tue, 25 Oct 2022 15:50:45 -0300 >

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-11-07 Thread Jason Gunthorpe
On Mon, Oct 31, 2022 at 04:45:26PM -0600, Alex Williamson wrote: > > It is one idea, it depends how literal you want to be on "module > > parameters are ABI". IMHO it is a weak form of ABI and the need of > > this paramter in particular is not that common in modern times, AFAIK. > > > > So

Re: [RFC PATCH v2 1/3] drivers/accel: define kconfig and register a new major

2022-11-07 Thread Jason Gunthorpe
On Mon, Nov 07, 2022 at 03:01:08PM +0200, Oded Gabbay wrote: > I don't agree with your statement that it should be "a layer over top of DRM". > Anything on top of DRM is a device driver. > Accel is not a device driver, it is a new type of drm minor / drm driver. Yeah, I still think this is not

Re: [RFC PATCH v2 1/3] drivers/accel: define kconfig and register a new major

2022-11-07 Thread Jason Gunthorpe
On Thu, Nov 03, 2022 at 10:39:36PM +0200, Oded Gabbay wrote: > On Thu, Nov 3, 2022 at 3:31 PM Oded Gabbay wrote: > > > > On Thu, Nov 3, 2022 at 2:31 AM Greg Kroah-Hartman > > wrote: > > > > > > On Wed, Nov 02, 2022 at 10:34:03PM +0200, Oded Gabbay wrote: > > > > --- /dev/null > > > > +++

Re: [PATCH 07/10] vfio-iommufd: Support iommufd for physical VFIO devices

2022-11-04 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 08:21:20AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, October 26, 2022 2:51 AM > > > > +int vfio_iommufd_bind(struct vfio_device *vdev, struct iommufd_ctx *ictx) > > +{ > > + u32 ioas_id; &g

Re: [PATCH 05/10] vfio: Use IOMMU_CAP_ENFORCE_CACHE_COHERENCY for vfio_file_enforced_coherent()

2022-11-04 Thread Jason Gunthorpe
On Thu, Nov 03, 2022 at 04:38:16AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, November 1, 2022 8:26 PM > > And this: > > > > /* > > * If the device does not have > > IOMMU_CAP_ENFORCE_CACHE_COHERENCY then > >

Re: [PATCH 02/10] vfio: Move vfio_device_assign_container() into vfio_device_first_open()

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 10:37:14PM +0800, Yi Liu wrote: > > @@ -784,12 +791,6 @@ static struct file *vfio_device_open(struct > > vfio_device *device) > > struct file *filep; > > int ret; > > - mutex_lock(>group->group_lock); > > - ret = vfio_device_assign_container(device); > > -

Re: [PATCH 09/10] vfio: Make vfio_container optionally compiled

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 08:41:56AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, October 26, 2022 2:51 AM > > > > if VFIO > > +config VFIO_CONTAINER > > + bool "Support for the VFIO container /dev/vfio/vfio" > >

Re: [PATCH 08/10] vfio-iommufd: Support iommufd for emulated VFIO devices

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 08:37:39AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, October 26, 2022 2:51 AM > > > > Emulated VFIO devices are calling vfio_register_emulated_iommu_dev() and > > consist of all the mdev drivers. > > &g

Re: [PATCH 06/10] vfio-iommufd: Allow iommufd to be used in place of a container fd

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 08:09:52AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, October 26, 2022 2:51 AM > > > > menuconfig VFIO > > tristate "VFIO Non-Privileged userspace driver framework" > > select IOMMU_API &

Re: [PATCH 05/10] vfio: Use IOMMU_CAP_ENFORCE_CACHE_COHERENCY for vfio_file_enforced_coherent()

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 07:52:23AM +, Tian, Kevin wrote: > > IOMMU_CAP_ENFORCE_CACHE_COHERENCY we know what the > > iommu_domain will be > > capable of without having to create it. Use this to compute > > it's worth noting that the prerequisite is that vfio always enforces > cache coherency

Re: [PATCH 02/10] vfio: Move vfio_device_assign_container() into vfio_device_first_open()

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 07:38:47AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, October 26, 2022 2:17 AM > > > > +err_container: > > + vfio_device_unassign_container(device); > > err_module_put: > > device->kvm = NU

Re: [PATCH 01/10] vfio: Move vfio_device driver open/close code to a function

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 07:33:30AM +, Tian, Kevin wrote: > > + /* > > +* Here we pass the KVM pointer with the group under the read lock. > > Now the read lock is replaced by mutex. Let's correct it when moving this > piece of code. Done, thanks Jason

Re: [PATCH 06/10] vfio-iommufd: Allow iommufd to be used in place of a container fd

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 02:19:04AM -0700, Nicolin Chen wrote: > On Tue, Nov 01, 2022 at 08:09:52AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > > Sent: Wednesday, October 26, 2022 2:51 AM > > > > > > menuconfig VFIO > > >

Re: [PATCH 00/10] Connect VFIO to IOMMUFD

2022-11-01 Thread Jason Gunthorpe
On Tue, Nov 01, 2022 at 11:04:38AM +0800, Yi Liu wrote: > On 2022/11/1 07:24, Jason Gunthorpe wrote: > > On Mon, Oct 31, 2022 at 08:25:39PM +0800, Yi Liu wrote: > > > > There is something wrong with the test suite that it isn't covering > > > > the above,

Re: [PATCH 00/10] Connect VFIO to IOMMUFD

2022-10-31 Thread Jason Gunthorpe
On Mon, Oct 31, 2022 at 08:25:39PM +0800, Yi Liu wrote: > > There is something wrong with the test suite that it isn't covering > > the above, I'm going to look into that today. > > sounds to be the cause. I didn't see any significant change in vfio_main.c > that may fail gvt. So should the

Re: [PATCH 00/10] Connect VFIO to IOMMUFD

2022-10-31 Thread Jason Gunthorpe
On Mon, Oct 31, 2022 at 06:38:45PM +0800, Yi Liu wrote: > Hi Jason, > > On 2022/10/26 02:17, Jason Gunthorpe wrote: > > This series provides an alternative container layer for VFIO implemented > > using iommufd. This is optional, if CONFIG_IOMMUFD is not set then it wil

Re: [PATCH 10/10] iommufd: Allow iommufd to supply /dev/vfio/vfio

2022-10-28 Thread Jason Gunthorpe
On Wed, Oct 26, 2022 at 03:31:33PM -0600, Alex Williamson wrote: > On Tue, 25 Oct 2022 15:50:45 -0300 > Jason Gunthorpe wrote: > > > If the VFIO container is compiled out, give a kconfig option for iommufd > > to provide the miscdev node with the same name and permiss

Re: [PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-10-28 Thread Jason Gunthorpe
On Wed, Oct 26, 2022 at 03:24:42PM -0600, Alex Williamson wrote: > On Tue, 25 Oct 2022 15:17:10 -0300 > Jason Gunthorpe wrote: > > > This legacy module knob has become uAPI, when set on the vfio_iommu_type1 > > it disables some security protections in the iommu drivers.

Re: [PATCH] mm/memremap: Introduce pgmap_request_folio() using pgmap offsets

2022-10-28 Thread Jason Gunthorpe
On Thu, Oct 20, 2022 at 02:56:39PM -0700, Dan Williams wrote: > A 'struct dev_pagemap' (pgmap) represents a collection of ZONE_DEVICE > pages. The pgmap is a reference counted object that serves a similar > role as a 'struct request_queue'. Live references are obtained for each > in flight request

[PATCH 09/10] vfio: Make vfio_container optionally compiled

2022-10-25 Thread Jason Gunthorpe
the device/group and container have been created. Signed-off-by: Jason Gunthorpe --- drivers/vfio/Kconfig | 37 drivers/vfio/Makefile | 4 +-- drivers/vfio/vfio.h | 65 +++ 3 files changed, 92 insertions(+), 14 deletions(-) diff

[PATCH 07/10] vfio-iommufd: Support iommufd for physical VFIO devices

2022-10-25 Thread Jason Gunthorpe
by providing bind_iommufd/unbind_iommufd/attach_ioas callbacks to implement as required. The core code provides three default callbacks for physical mode using a real iommu_domain. This is suitable for drivers using vfio_register_group_dev() Signed-off-by: Jason Gunthorpe --- drivers/vfio/Makefile

[PATCH 06/10] vfio-iommufd: Allow iommufd to be used in place of a container fd

2022-10-25 Thread Jason Gunthorpe
lly present on the group. Signed-off-by: Jason Gunthorpe --- drivers/vfio/Kconfig | 1 + drivers/vfio/container.c | 7 ++-- drivers/vfio/vfio.h | 2 ++ drivers/vfio/vfio_main.c | 76 4 files changed, 69 insertions(+), 17 deletions(-) diff --

[PATCH 08/10] vfio-iommufd: Support iommufd for emulated VFIO devices

2022-10-25 Thread Jason Gunthorpe
vfio_register_emulated_iommu_dev() does. Emulated drivers are where it is more likely to see variation in the iommfd support ops. For instance IDXD will probably need to setup both a iommfd_device context linked to a PASID and an iommufd_access context to support all their mdev operations. Signed-off-by: Jason Gunthorpe

[PATCH 10/10] iommufd: Allow iommufd to supply /dev/vfio/vfio

2022-10-25 Thread Jason Gunthorpe
-by: Jason Gunthorpe --- drivers/iommu/iommufd/Kconfig | 12 drivers/iommu/iommufd/main.c | 35 --- 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/iommufd/Kconfig b/drivers/iommu/iommufd/Kconfig index f0a2012234fa09

[PATCH 01/10] vfio: Move vfio_device driver open/close code to a function

2022-10-25 Thread Jason Gunthorpe
This error unwind is getting complicated. Move all the code into two pair'd function. The functions should be called when the open_count == 1 after incrementing/before decrementing. Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio_main.c | 95 ++-- 1 file

[PATCH 02/10] vfio: Move vfio_device_assign_container() into vfio_device_first_open()

2022-10-25 Thread Jason Gunthorpe
be closed. Thus we do not need to check this on evey device FD open, just the first. Reorganize the code so that only the first open and last close manages the container. Signed-off-by: Jason Gunthorpe --- drivers/vfio/container.c | 4 ++-- drivers/vfio/vfio_main.c | 18 -- 2 files

[PATCH 05/10] vfio: Use IOMMU_CAP_ENFORCE_CACHE_COHERENCY for vfio_file_enforced_coherent()

2022-10-25 Thread Jason Gunthorpe
lve the ordering problems. Signed-off-by: Jason Gunthorpe --- drivers/vfio/container.c | 5 +++-- drivers/vfio/vfio.h | 2 -- drivers/vfio/vfio_main.c | 27 ++- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/drivers/vfio/container.c b/drivers/vfio/c

[PATCH 00/10] Connect VFIO to IOMMUFD

2022-10-25 Thread Jason Gunthorpe
://github.com/jgunthorpe/linux/commits/vfio_iommufd It requires the iommufd series: https://lore.kernel.org/r/0-v3-402a7d6459de+24b-iommufd_...@nvidia.com Jason Gunthorpe (10): vfio: Move vfio_device driver open/close code to a function vfio: Move vfio_device_assign_container

[PATCH 03/10] vfio: Rename vfio_device_assign/unassign_container()

2022-10-25 Thread Jason Gunthorpe
These functions don't really assign anything anymore, they just increment some refcounts and do a sanity check. Call them vfio_group_[un]use_container() Signed-off-by: Jason Gunthorpe --- drivers/vfio/container.c | 14 ++ drivers/vfio/vfio.h | 4 ++-- drivers/vfio/vfio_main.c

[PATCH 04/10] vfio: Move storage of allow_unsafe_interrupts to vfio_main.c

2022-10-25 Thread Jason Gunthorpe
This legacy module knob has become uAPI, when set on the vfio_iommu_type1 it disables some security protections in the iommu drivers. Move the storage for this knob to vfio_main.c so that iommufd can access it too. Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.h | 2

Re: [RFC PATCH 0/3] new subsystem for compute accelerator devices

2022-10-25 Thread Jason Gunthorpe
On Tue, Oct 25, 2022 at 10:21:34AM -0400, Alex Deucher wrote: > E.g., the kfd node provides platform level compute > topology information; e.g., the NUMA details for connected GPUs and > CPUs, non-GPU compute node information, cache level topologies, etc. See, this is exactly what I'm talking

Re: [RFC PATCH 0/3] new subsystem for compute accelerator devices

2022-10-25 Thread Jason Gunthorpe
On Tue, Oct 25, 2022 at 12:27:11PM +1000, Dave Airlie wrote: > The userspace for those is normally bespoke like ROCm, which uses > amdkfd, and amdkfd doesn't operate like most device files from what I > know, so I'm not sure we'd want it to operate as an accel device. I intensely dislike this

Re: [PATCH v1 6/7] vfio/ccw: replace vfio_init_device with _alloc_

2022-10-20 Thread Jason Gunthorpe
On Wed, Oct 19, 2022 at 01:57:41PM -0400, Eric Farman wrote: > On Wed, 2022-10-19 at 14:15 -0300, Jason Gunthorpe wrote: > > On Wed, Oct 19, 2022 at 06:21:34PM +0200, Eric Farman wrote: > > > > >  /* > > >   * Initialize a vfio_device so it can be registered to vf

Re: [PATCH v1 0/7] vfio-ccw parent rework

2022-10-19 Thread Jason Gunthorpe
On Wed, Oct 19, 2022 at 06:21:28PM +0200, Eric Farman wrote: > This series attempts to address the oddities/shortcomings of vfio-ccw, such > that vfio-ccw can use the same helpers as everyone else, and the tricks that > were implemented by the other drivers can be removed. It is built on 6.1-rc1,

Re: [PATCH v1 7/7] vfio: Remove vfio_free_device

2022-10-19 Thread Jason Gunthorpe
em from driver release callbacks. > > Signed-off-by: Eric Farman Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH v1 6/7] vfio/ccw: replace vfio_init_device with _alloc_

2022-10-19 Thread Jason Gunthorpe
On Wed, Oct 19, 2022 at 06:21:34PM +0200, Eric Farman wrote: > /* > * Initialize a vfio_device so it can be registered to vfio core. > - * > - * Only vfio-ccw driver should call this interface. > */ > int vfio_init_device(struct vfio_device *device, struct device *dev, >

Re: [PATCH v6 10/21] RDMA/umem: Prepare to dynamic dma-buf locking specification

2022-10-10 Thread Jason Gunthorpe
> > - dma_buf_unmap_attachment(umem_dmabuf->attach, umem_dmabuf->sgt, > > -DMA_BIDIRECTIONAL); > > + dma_buf_unmap_attachment_unlocked(umem_dmabuf->attach, umem_dmabuf->sgt, > > + DMA_BIDIRECTIONAL); > > > > umem_dmabuf->sgt = NULL; > > } > > Jason / Leon, > > Could you please ack this patch? You probably don't need it, for something so simple, but sure Acked-by: Jason Gunthorpe Jason

Re: [PATCH v1 1/5] treewide: use prandom_u32_max() when possible

2022-10-06 Thread Jason Gunthorpe
On Thu, Oct 06, 2022 at 06:45:25AM -0600, Jason A. Donenfeld wrote: > Hi Kees, > > On Wed, Oct 05, 2022 at 09:16:50PM -0700, Kees Cook wrote: > > On Wed, Oct 05, 2022 at 11:48:40PM +0200, Jason A. Donenfeld wrote: > > > Rather than incurring a division or requesting too many random bytes for > >

Re: [PATCH v1 3/5] treewide: use get_random_u32() when possible

2022-10-06 Thread Jason Gunthorpe
On Wed, Oct 05, 2022 at 11:48:42PM +0200, Jason A. Donenfeld wrote: > index 14392c942f49..499a425a3379 100644 > --- a/drivers/infiniband/hw/cxgb4/cm.c > +++ b/drivers/infiniband/hw/cxgb4/cm.c > @@ -734,7 +734,7 @@ static int send_connect(struct c4iw_ep *ep) >

Re: [PATCH v1 3/5] treewide: use get_random_u32() when possible

2022-10-06 Thread Jason Gunthorpe
On Thu, Oct 06, 2022 at 07:05:48AM -0600, Jason A. Donenfeld wrote: > > > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > > b/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > > index fd9d7f2c4d64..a605cf66b83e 100644 > > > --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c > > > +++

Re: [PATCH] drm/i915/gvt: Add missing vfio_unregister_group_dev() call

2022-10-06 Thread Jason Gunthorpe
On Wed, Oct 05, 2022 at 04:03:56PM -0600, Alex Williamson wrote: > We can't have a .remove callback that does nothing, this breaks > removing the device while it's in use. Once we have the > vfio_unregister_group_dev() fix below, we'll block until the device is > unused, at which point

Re: [PATCH] drm/i915/gvt: Add missing vfio_unregister_group_dev() call

2022-10-06 Thread Jason Gunthorpe
On Wed, Oct 05, 2022 at 02:17:17PM -0600, Alex Williamson wrote: > > diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c > > b/drivers/gpu/drm/i915/gvt/kvmgt.c > > index 41bba40feef8f4..9003145adb5a93 100644 > > --- a/drivers/gpu/drm/i915/gvt/kvmgt.c > > +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c > > @@

Re: [PATCH v4 15/15] vfio: Add struct device to vfio_device

2022-09-29 Thread Jason Gunthorpe
On Thu, Sep 29, 2022 at 10:55:19AM -0600, Alex Williamson wrote: > Hi Kevin, > > This introduced the regression discovered here: > > https://lore.kernel.org/all/20220928125650.0a2ea297.alex.william...@redhat.com/ > > Seems we're not releasing the resources when removing an mdev. This is > a

[PATCH] drm/i915/gvt: Add missing vfio_unregister_group_dev() call

2022-09-29 Thread Jason Gunthorpe
nvert to use vfio_register_emulated_iommu_dev") Reported-by: Alex Williamson Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/kvmgt.c | 1 + 1 file changed, 1 insertion(+) Should go through Alex's tree. diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvm

Re: [PATCH 0/4] RDMA/mlx5: Support DMABUF in umems and enable ATS

2022-09-26 Thread Jason Gunthorpe
On Thu, Sep 01, 2022 at 11:20:52AM -0300, Jason Gunthorpe wrote: > This series adds support for DMABUF when creating a devx umem. devx umems > are quite similar to MR's execpt they cannot be revoked, so this uses the > dmabuf pinned memory flow. Several mlx5dv flows require umem and cann

Re: [PATCH 2/7] mm: Free device private pages have zero refcount

2022-09-26 Thread Jason Gunthorpe
On Mon, Sep 26, 2022 at 04:03:06PM +1000, Alistair Popple wrote: > Since 27674ef6c73f ("mm: remove the extra ZONE_DEVICE struct page > refcount") device private pages have no longer had an extra reference > count when the page is in use. However before handing them back to the > owning device

Re: [PATCH v3 15/15] vfio: Add struct device to vfio_device

2022-09-21 Thread Jason Gunthorpe
On Tue, Sep 20, 2022 at 10:55:40PM +, Tian, Kevin wrote: > > From: Alex Williamson > > Sent: Wednesday, September 21, 2022 4:27 AM > > > > On Fri, 9 Sep 2022 18:22:47 +0800 > > Kevin Tian wrote: > > > > > From: Yi Liu > > > > > > and replace kref. With it a 'vfio-dev/vfioX' node is

Re: [PATCH v3 01/15] vfio: Add helpers for unifying vfio_device life cycle

2022-09-09 Thread Jason Gunthorpe
On Fri, Sep 09, 2022 at 08:42:25AM +, Tian, Kevin wrote: > I think it's quite common to have an alloc() helper initialize refcount, e.g. > vfio_group_alloc() both initialize its user refcount and also call > device_initialize() to gets kref initialized. Similar example in >

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-09 Thread Jason Gunthorpe
On Fri, Sep 09, 2022 at 06:24:35AM -0700, Christoph Hellwig wrote: > On Wed, Sep 07, 2022 at 01:12:52PM -0300, Jason Gunthorpe wrote: > > The PCI offset is some embedded thing - I've never seen it in a server > > platform. > > That's not actually true, e.g. some power sy

Re: [PATCH v2 15/15] vfio: Add struct device to vfio_device

2022-09-08 Thread Jason Gunthorpe
On Thu, Sep 08, 2022 at 11:39:07AM +0200, Eric Auger wrote: > >> I am not totally clear about remaining 'struct device *dev;' in > >> vfio_device struct. I see it used in some places. Is it supposed to > >> disappear at some point? > > > > no, Eric. *dev will not disappear, it stores the dev

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-07 Thread Jason Gunthorpe
On Wed, Sep 07, 2022 at 05:31:14PM +0100, Robin Murphy wrote: > The only trouble is that it's not geared for *PCI* P2P when that may or may > not happen entirely upstream of IOMMU translation. This is why PCI users have to call the pci_distance stuff before using dma_map_resource(), it ensures

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-07 Thread Jason Gunthorpe
On Wed, Sep 07, 2022 at 08:32:23AM -0700, Christoph Hellwig wrote: > On Wed, Sep 07, 2022 at 12:23:28PM -0300, Jason Gunthorpe wrote: > > 2) DMABUF abuses dma_map_resource() for P2P and thus doesn't work in > > certain special cases. > > Not just certain special cases

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-07 Thread Jason Gunthorpe
On Wed, Sep 07, 2022 at 07:29:58AM -0700, Christoph Hellwig wrote: > On Wed, Sep 07, 2022 at 09:33:11AM -0300, Jason Gunthorpe wrote: > > Yes, you said that, and I said that when the AMD driver first merged > > it - but it went in anyhow and now people are using it in a bunc

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-07 Thread Jason Gunthorpe
On Wed, Sep 07, 2022 at 05:05:57AM -0700, Christoph Hellwig wrote: > On Tue, Sep 06, 2022 at 08:48:28AM -0300, Jason Gunthorpe wrote: > > Right, this whole thing is the "standard" that dmabuf has adopted > > instead of the struct pages. Once the AMD GPU driver started doi

Re: [PATCH v2 01/15] vfio: Add helpers for unifying vfio_device life cycle

2022-09-07 Thread Jason Gunthorpe
On Wed, Sep 07, 2022 at 04:55:18AM -0700, Christoph Hellwig wrote: > On Wed, Sep 07, 2022 at 12:43:30AM +, Tian, Kevin wrote: > > > From: Christoph Hellwig > > > Sent: Tuesday, September 6, 2022 5:42 PM > > > > > > What is the point? This adds indirect calls, and actually creates > > > more

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-06 Thread Jason Gunthorpe
On Tue, Sep 06, 2022 at 03:34:02PM +0300, Oded Gabbay wrote: > > > > > + /* > > > > > + * Since the memory being mapped is a device memory it could never > > > > > be in > > > > > + * CPU caches. > > > > > + */ > > > > DMA_ATTR_SKIP_CPU_SYNC doesn't even apply to dma_map_resource, not sure >

Re: [PATCH v2 1/4] dma-buf: Add dma_buf_try_get()

2022-09-06 Thread Jason Gunthorpe
On Thu, Sep 01, 2022 at 09:55:08AM +0200, Christian König wrote: > Am 01.09.22 um 01:12 schrieb Jason Gunthorpe: > > Used to increment the refcount of the dma buf's struct file, only if the > > refcount is not zero. Useful to allow the struct file's lifetime to > >

Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

2022-09-06 Thread Jason Gunthorpe
On Tue, Sep 06, 2022 at 12:38:44PM +0200, Christian König wrote: > Am 06.09.22 um 11:51 schrieb Christoph Hellwig: > > > +{ > > > + struct vfio_pci_dma_buf *priv = dmabuf->priv; > > > + int rc; > > > + > > > + rc = pci_p2pdma_distance_many(priv->vdev->pdev, >dev, 1, > > > +

[PATCH 0/4] RDMA/mlx5: Support DMABUF in umems and enable ATS

2022-09-01 Thread Jason Gunthorpe
redirect setting to be bypassed in the switch. Have mlx5 automatically use ATS in places where it makes sense. Jason Gunthorpe (4): net/mlx5: Add IFC bits for mkey ATS RDMA/core: Add UVERBS_ATTR_RAW_FD RDMA/mlx5: Add support for dmabuf to devx umem RDMA/mlx5: Enable ATS support for MRs

[PATCH 1/4] net/mlx5: Add IFC bits for mkey ATS

2022-09-01 Thread Jason Gunthorpe
Allows telling a mkey to use PCI ATS for DMA that flows through it. Signed-off-by: Jason Gunthorpe --- include/linux/mlx5/mlx5_ifc.h | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 4acd5610e96bc0

[PATCH 4/4] RDMA/mlx5: Enable ATS support for MRs and umems

2022-09-01 Thread Jason Gunthorpe
of these cases revolve around optimizing PCI P2P transfers and avoiding bad cases where the bus just doesn't work. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/hw/mlx5/devx.c| 37 drivers/infiniband/hw/mlx5/mlx5_ib.h | 36 +++ drivers

[PATCH 3/4] RDMA/mlx5: Add support for dmabuf to devx umem

2022-09-01 Thread Jason Gunthorpe
else stays the same. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/hw/mlx5/devx.c| 24 +--- include/uapi/rdma/mlx5_user_ioctl_cmds.h | 1 + 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniba

[PATCH 2/4] RDMA/core: Add UVERBS_ATTR_RAW_FD

2022-09-01 Thread Jason Gunthorpe
This uses the same passing protocol as UVERBS_ATTR_FD (eg len = 0 data_s64 = fd), except that the FD is not required to be a uverbs object and the core code does not covert the FD to an object handle automatically. Access to the int fd is provided by uverbs_get_raw_fd(). Signed-off-by: Jason

<    1   2   3   4   5   6   7   8   9   10   >