RE: [PATCH v11 00/13] SMMUv3 Nested Stage Setup (IOMMU part)

2020-04-30 Thread Shameerali Kolothum Thodi
Hi Eric, > -Original Message- > From: Auger Eric [mailto:eric.au...@redhat.com] > Sent: 16 April 2020 08:45 > To: Zhangfei Gao ; eric.auger@gmail.com; > iommu@lists.linux-foundation.org; linux-ker...@vger.kernel.org; > k...@vger.kernel.org; kvm...@lists.cs.columbia.edu; w...@kernel.org

[RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Srivatsa Vaddagiri
The Type-1 hypervisor we are dealing with does not allow for MMIO transport. [1] summarizes some of the problems we have in making virtio work on such hypervisors. This patch proposes a solution for transport problem viz how we can do config space IO on such a hypervisor. Hypervisor specific metho

[RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Srivatsa Vaddagiri
Some hypervisors may not support MMIO transport i.e trap config space access and have it be handled by backend driver. They may allow other ways to interact with backend such as message-queue or doorbell API. This patch allows for hypervisor specific methods for config space IO. Signed-off-by: Sri

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Michael S. Tsirkin
On Thu, Apr 30, 2020 at 03:32:55PM +0530, Srivatsa Vaddagiri wrote: > The Type-1 hypervisor we are dealing with does not allow for MMIO transport. How about PCI then? -- MST ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxf

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Will Deacon
On Thu, Apr 30, 2020 at 03:32:55PM +0530, Srivatsa Vaddagiri wrote: > The Type-1 hypervisor we are dealing with does not allow for MMIO transport. > [1] summarizes some of the problems we have in making virtio work on such > hypervisors. This patch proposes a solution for transport problem viz how

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Will Deacon
On Thu, Apr 30, 2020 at 03:32:56PM +0530, Srivatsa Vaddagiri wrote: > Some hypervisors may not support MMIO transport i.e trap config > space access and have it be handled by backend driver. They may > allow other ways to interact with backend such as message-queue > or doorbell API. This patch all

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Srivatsa Vaddagiri
* Will Deacon [2020-04-30 11:08:22]: > > This patch is meant to seek comments. If its considered to be in right > > direction, will work on making it more complete and send the next version! > > What's stopping you from implementing the trapping support in the > hypervisor? Unlike the other patc

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Srivatsa Vaddagiri
* Will Deacon [2020-04-30 11:14:32]: > > +#ifdef CONFIG_VIRTIO_MMIO_OPS > > > > +static struct virtio_mmio_ops *mmio_ops; > > + > > +#define virtio_readb(a)mmio_ops->mmio_readl((a)) > > +#define virtio_readw(a)mmio_ops->mmio_readl((a)) > > +#define virtio_readl(a)

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Will Deacon
Hi Vatsa, On Thu, Apr 30, 2020 at 03:59:39PM +0530, Srivatsa Vaddagiri wrote: > * Will Deacon [2020-04-30 11:08:22]: > > > > This patch is meant to seek comments. If its considered to be in right > > > direction, will work on making it more complete and send the next version! > > > > What's sto

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Srivatsa Vaddagiri
* Michael S. Tsirkin [2020-04-30 06:07:56]: > On Thu, Apr 30, 2020 at 03:32:55PM +0530, Srivatsa Vaddagiri wrote: > > The Type-1 hypervisor we are dealing with does not allow for MMIO > > transport. > > How about PCI then? Correct me if I am wrong, but basically virtio_pci uses the same low-l

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Will Deacon
On Thu, Apr 30, 2020 at 04:04:46PM +0530, Srivatsa Vaddagiri wrote: > * Will Deacon [2020-04-30 11:14:32]: > > > > +#ifdef CONFIG_VIRTIO_MMIO_OPS > > > > > > +static struct virtio_mmio_ops *mmio_ops; > > > + > > > +#define virtio_readb(a) mmio_ops->mmio_readl((a)) > > > +#define virtio

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Jason Wang
On 2020/4/30 下午6:07, Michael S. Tsirkin wrote: On Thu, Apr 30, 2020 at 03:32:55PM +0530, Srivatsa Vaddagiri wrote: The Type-1 hypervisor we are dealing with does not allow for MMIO transport. How about PCI then? Or maybe you can use virtio-vdpa. Thanks ___

Re: [RFC/PATCH 0/1] virtio_mmio: hypervisor specific interfaces for MMIO

2020-04-30 Thread Srivatsa Vaddagiri
* Will Deacon [2020-04-30 11:39:19]: > Hi Vatsa, > > On Thu, Apr 30, 2020 at 03:59:39PM +0530, Srivatsa Vaddagiri wrote: > > > What's stopping you from implementing the trapping support in the > > > hypervisor? Unlike the other patches you sent out, where the guest memory > > > is not accessible

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Srivatsa Vaddagiri
* Will Deacon [2020-04-30 11:41:50]: > On Thu, Apr 30, 2020 at 04:04:46PM +0530, Srivatsa Vaddagiri wrote: > > If CONFIG_VIRTIO_MMIO_OPS is defined, then I expect this to be > > unconditionally > > set to 'magic_qcom_ops' that uses hypervisor-supported interface for IO (for > > example: message_

[PATCH] drivers/iommu: properly export iommu_group_get_for_dev

2020-04-30 Thread Greg Kroah-Hartman
In commit a7ba5c3d008d ("drivers/iommu: Export core IOMMU API symbols to permit modular drivers") a bunch of iommu symbols were exported, all with _GPL markings except iommu_group_get_for_dev(). That export should also be _GPL like the others. Cc: Will Deacon Cc: Joerg Roedel Cc: John Garry Fi

Re: [PATCH] drivers/iommu: properly export iommu_group_get_for_dev

2020-04-30 Thread Will Deacon
On Thu, Apr 30, 2020 at 02:01:20PM +0200, Greg Kroah-Hartman wrote: > In commit a7ba5c3d008d ("drivers/iommu: Export core IOMMU API symbols to > permit modular drivers") a bunch of iommu symbols were exported, all > with _GPL markings except iommu_group_get_for_dev(). That export should > also be

Re: [PATCH] drivers/iommu: properly export iommu_group_get_for_dev

2020-04-30 Thread Joerg Roedel
On Thu, Apr 30, 2020 at 01:17:53PM +0100, Will Deacon wrote: > Thanks, not sure how I managed to screw this up in the original patch! > > Acked-by: Will Deacon > > Joerg -- can you pick this one up please? Yes, will send it as a fix for 5.7, but note that this function will be unexported in 5.8

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Jan Kiszka
On 30.04.20 13:11, Srivatsa Vaddagiri wrote: * Will Deacon [2020-04-30 11:41:50]: On Thu, Apr 30, 2020 at 04:04:46PM +0530, Srivatsa Vaddagiri wrote: If CONFIG_VIRTIO_MMIO_OPS is defined, then I expect this to be unconditionally set to 'magic_qcom_ops' that uses hypervisor-supported interface

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Srivatsa Vaddagiri
* Jan Kiszka [2020-04-30 14:59:50]: > >I believe ivshmem2_virtio requires hypervisor to support PCI device emulation > >(for life-cycle management of VMs), which our hypervisor may not support. A > >simple shared memory and doorbell or message-queue based transport will work > >for > >us. > > A

Re: [Intel-gfx] [RFC 06/17] drm: i915: fix sg_table nents vs. orig_nents misuse

2020-04-30 Thread Marek Szyprowski
Hi On 28.04.2020 16:27, Tvrtko Ursulin wrote: > > On 28/04/2020 14:19, Marek Szyprowski wrote: >> The Documentation/DMA-API-HOWTO.txt states that dma_map_sg returns the >> numer of the created entries in the DMA address space. However the >> subsequent calls to dma_sync_sg_for_{device,cpu} and dma

[PATCH v6 00/25] iommu: Shared Virtual Addressing for SMMUv3

2020-04-30 Thread Jean-Philippe Brucker
Shared Virtual Addressing (SVA) allows to share process page tables with devices using the IOMMU, PASIDs and I/O page faults. Add SVA support to the Arm SMMUv3 driver. Since v5 [1]: * Added patches 1-3. Patch 1 adds a PASID field to mm_struct as discussed in [1] and [2]. This is also needed for

[PATCH v6 03/25] iommu/sva: Add PASID helpers

2020-04-30 Thread Jean-Philippe Brucker
Let IOMMU drivers allocate a single PASID per mm. Store the mm in the IOASID set to allow refcounting and searching mm by PASID, when handling an I/O page fault. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/Kconfig | 5 +++ drivers/iommu/Makefile| 1 + drivers/iommu/iommu-sva

[PATCH v6 19/25] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update

2020-04-30 Thread Jean-Philippe Brucker
If the SMMU supports it and the kernel was built with HTTU support, enable hardware update of access and dirty flags. This is essential for shared page tables, to reduce the number of access faults on the fault queue. We can enable HTTU even if CPUs don't support it, because the kernel always chec

[PATCH v6 06/25] arm64: mm: Add asid_gen_match() helper

2020-04-30 Thread Jean-Philippe Brucker
Add a macro to check if an ASID is from the current generation, since a subsequent patch will introduce a third user for this test. Signed-off-by: Jean-Philippe Brucker --- arch/arm64/mm/context.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/context.c

[PATCH v6 21/25] dt-bindings: document stall property for IOMMU masters

2020-04-30 Thread Jean-Philippe Brucker
On ARM systems, some platform devices behind an IOMMU may support stall, which is the ability to recover from page faults. Let the firmware tell us when a device supports stall. Reviewed-by: Rob Herring Signed-off-by: Jean-Philippe Brucker --- .../devicetree/bindings/iommu/iommu.txt| 18

[PATCH v6 18/25] iommu/arm-smmu-v3: Hook up ATC invalidation to mm ops

2020-04-30 Thread Jean-Philippe Brucker
The invalidate_range() notifier is called for any change to the address space. Perform the required ATC invalidations. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 56 ++--- 1 file changed, 46 insertions(+), 10 deletions(-) diff --git a/

[PATCH v6 24/25] PCI/ATS: Export PRI functions

2020-04-30 Thread Jean-Philippe Brucker
The SMMUv3 driver uses pci_{enable,disable}_pri() and related functions. Export those functions to allow the driver to be built as a module. Acked-by: Bjorn Helgaas Reviewed-by: Kuppuswamy Sathyanarayanan Signed-off-by: Jean-Philippe Brucker --- drivers/pci/ats.c | 4 1 file changed, 4 i

[PATCH v6 09/25] iommu/arm-smmu-v3: Manage ASIDs with xarray

2020-04-30 Thread Jean-Philippe Brucker
In preparation for sharing some ASIDs with the CPU, use a global xarray to store ASIDs and their context. ASID#0 is now reserved, and the ASID space is global. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 27 ++- 1 file changed, 18 insertions(+),

[PATCH v6 20/25] iommu/arm-smmu-v3: Maintain a SID->device structure

2020-04-30 Thread Jean-Philippe Brucker
When handling faults from the event or PRI queue, we need to find the struct device associated to a SID. Add a rb_tree to keep track of SIDs. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 175 +--- 1 file changed, 145 insertions(+), 30 del

[PATCH v6 11/25] iommu/arm-smmu-v3: Share process page tables

2020-04-30 Thread Jean-Philippe Brucker
With Shared Virtual Addressing (SVA), we need to mirror CPU TTBR, TCR, MAIR and ASIDs in SMMU contexts. Each SMMU has a single ASID space split into two sets, shared and private. Shared ASIDs correspond to those obtained from the arch ASID allocator, and private ASIDs are used for "classic" map/unm

[PATCH v6 02/25] iommu/ioasid: Add ioasid references

2020-04-30 Thread Jean-Philippe Brucker
Let IOASID users take references to existing ioasids with ioasid_get(). ioasid_free() drops a reference and only frees the ioasid when its reference number is zero. It returns whether the ioasid was freed. Signed-off-by: Jean-Philippe Brucker --- include/linux/ioasid.h | 10 -- drivers/i

[PATCH v6 10/25] arm64: cpufeature: Export symbol read_sanitised_ftr_reg()

2020-04-30 Thread Jean-Philippe Brucker
The SMMUv3 driver would like to read the MMFR0 PARANGE field in order to share CPU page tables with devices. Allow the driver to be built as module by exporting the read_sanitized_ftr_reg() cpufeature symbol. Cc: Suzuki K Poulose Signed-off-by: Jean-Philippe Brucker --- arch/arm64/kernel/cpufea

[PATCH v6 14/25] iommu/arm-smmu-v3: Enable broadcast TLB maintenance

2020-04-30 Thread Jean-Philippe Brucker
The SMMUv3 can handle invalidation targeted at TLB entries with shared ASIDs. If the implementation supports broadcast TLB maintenance, enable it and keep track of it in a feature bit. The SMMU will then be affected by inner-shareable TLB invalidations from other agents. A major side-effect of thi

[PATCH v6 25/25] iommu/arm-smmu-v3: Add support for PRI

2020-04-30 Thread Jean-Philippe Brucker
For PCI devices that support it, enable the PRI capability and handle PRI Page Requests with the generic fault handler. It is enabled on demand by iommu_sva_device_init(). Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 286 +--- 1 file chan

[PATCH v6 01/25] mm: Add a PASID field to mm_struct

2020-04-30 Thread Jean-Philippe Brucker
Some devices can tag their DMA requests with a 20-bit Process Address Space ID (PASID), allowing them to access multiple address spaces. In combination with recoverable I/O page faults (for example PCIe PRI), PASID allows the IOMMU to share page tables with the MMU. To make sure that a single PASI

[PATCH v6 17/25] iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind()

2020-04-30 Thread Jean-Philippe Brucker
The sva_bind() function allows devices to access process address spaces using a PASID (aka SSID). (1) bind() allocates or gets an existing MMU notifier tied to the (domain, mm) pair. Each mm gets one PASID. (2) Any change to the address space calls invalidate_range() which sends ATC inval

[PATCH v6 13/25] iommu/arm-smmu-v3: Add support for VHE

2020-04-30 Thread Jean-Philippe Brucker
ARMv8.1 extensions added Virtualization Host Extensions (VHE), which allow to run a host kernel at EL2. When using normal DMA, Device and CPU address spaces are dissociated, and do not need to implement the same capabilities, so VHE hasn't been used in the SMMU until now. With shared address space

[PATCH v6 22/25] iommu/arm-smmu-v3: Add stall support for platform devices

2020-04-30 Thread Jean-Philippe Brucker
The SMMU provides a Stall model for handling page faults in platform devices. It is similar to PCI PRI, but doesn't require devices to have their own translation cache. Instead, faulting transactions are parked and the OS is given a chance to fix the page tables and retry the transaction. Enable s

[PATCH v6 12/25] iommu/arm-smmu-v3: Seize private ASID

2020-04-30 Thread Jean-Philippe Brucker
The SMMU has a single ASID space, the union of shared and private ASID sets. This means that the SMMU driver competes with the arch allocator for ASIDs. Shared ASIDs are those of Linux processes, allocated by the arch, and contribute in broadcast TLB maintenance. Private ASIDs are allocated by the

[PATCH v6 07/25] arm64: mm: Pin down ASIDs for sharing mm with devices

2020-04-30 Thread Jean-Philippe Brucker
To enable address space sharing with the IOMMU, introduce mm_context_get() and mm_context_put(), that pin down a context and ensure that it will keep its ASID after a rollover. Export the symbols to let the modular SMMUv3 driver use them. Pinning is necessary because a device constantly needs a va

[PATCH v6 04/25] iommu: Add a page fault handler

2020-04-30 Thread Jean-Philippe Brucker
Some systems allow devices to handle I/O Page Faults in the core mm. For example systems implementing the PCIe PRI extension or Arm SMMU stall model. Infrastructure for reporting these recoverable page faults was added to the IOMMU core by commit 0c830e6b3282 ("iommu: Introduce device fault report

[PATCH v6 16/25] iommu/arm-smmu-v3: Add SVA device feature

2020-04-30 Thread Jean-Philippe Brucker
Implement the IOMMU device feature callbacks to support the SVA feature. At the moment dev_has_feat() returns false since I/O Page Faults isn't yet implemented. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 125 1 file changed, 125 in

[PATCH v6 05/25] iommu/iopf: Handle mm faults

2020-04-30 Thread Jean-Philippe Brucker
When a recoverable page fault is handled by the fault workqueue, find the associated mm and call handle_mm_fault. Signed-off-by: Jean-Philippe Brucker --- v5->v6: select CONFIG_IOMMU_SVA --- drivers/iommu/Kconfig | 1 + drivers/iommu/io-pgfault.c | 79 +-

[PATCH v6 08/25] iommu/io-pgtable-arm: Move some definitions to a header

2020-04-30 Thread Jean-Philippe Brucker
Extract some of the most generic TCR defines, so they can be reused by the page table sharing code. Signed-off-by: Jean-Philippe Brucker --- v5->v6: Update MAINTAINERS --- drivers/iommu/io-pgtable-arm.h | 30 ++ drivers/iommu/io-pgtable-arm.c | 27 ++--

[PATCH v6 15/25] iommu/arm-smmu-v3: Add SVA feature checking

2020-04-30 Thread Jean-Philippe Brucker
Aggregate all sanity-checks for sharing CPU page tables with the SMMU under a single ARM_SMMU_FEAT_SVA bit. For PCIe SVA, users also need to check FEAT_ATS and FEAT_PRI. For platform SVA, they will most likely have to check FEAT_STALLS. Cc: Suzuki K Poulose Signed-off-by: Jean-Philippe Brucker -

[PATCH v6 23/25] PCI/ATS: Add PRI stubs

2020-04-30 Thread Jean-Philippe Brucker
The SMMUv3 driver, which can be built without CONFIG_PCI, will soon gain support for PRI. Partially revert commit c6e9aefbf9db ("PCI/ATS: Remove unused PRI and PASID stubs") to re-introduce the PRI stubs, and avoid adding more #ifdefs to the SMMU driver. Acked-by: Bjorn Helgaas Reviewed-by: Kupp

Re: [PATCH v6 10/25] arm64: cpufeature: Export symbol read_sanitised_ftr_reg()

2020-04-30 Thread Suzuki K Poulose
On 04/30/2020 03:34 PM, Jean-Philippe Brucker wrote: The SMMUv3 driver would like to read the MMFR0 PARANGE field in order to share CPU page tables with devices. Allow the driver to be built as module by exporting the read_sanitized_ftr_reg() cpufeature symbol. Cc: Suzuki K Poulose Signed-off-b

Re: [PATCH 5/5] virtio: Add bounce DMA ops

2020-04-30 Thread Konrad Rzeszutek Wilk
On Wed, Apr 29, 2020 at 06:20:48AM -0400, Michael S. Tsirkin wrote: > On Wed, Apr 29, 2020 at 03:39:53PM +0530, Srivatsa Vaddagiri wrote: > > That would still not work I think where swiotlb is used for pass-thr devices > > (when private memory is fine) as well as virtio devices (when shared memory

Re: [PATCH v6 11/25] iommu/arm-smmu-v3: Share process page tables

2020-04-30 Thread Suzuki K Poulose
On 04/30/2020 03:34 PM, Jean-Philippe Brucker wrote: With Shared Virtual Addressing (SVA), we need to mirror CPU TTBR, TCR, MAIR and ASIDs in SMMU contexts. Each SMMU has a single ASID space split into two sets, shared and private. Shared ASIDs correspond to those obtained from the arch ASID allo

Re: [PATCH v6 02/25] iommu/ioasid: Add ioasid references

2020-04-30 Thread Jacob Pan
On Thu, 30 Apr 2020 16:34:01 +0200 Jean-Philippe Brucker wrote: > Let IOASID users take references to existing ioasids with > ioasid_get(). ioasid_free() drops a reference and only frees the > ioasid when its reference number is zero. It returns whether the > ioasid was freed. > Looks good to me

Re: [RFC/PATCH 1/1] virtio: Introduce MMIO ops

2020-04-30 Thread Michael S. Tsirkin
On Thu, Apr 30, 2020 at 07:03:21PM +0530, Srivatsa Vaddagiri wrote: > * Jan Kiszka [2020-04-30 14:59:50]: > > > >I believe ivshmem2_virtio requires hypervisor to support PCI device > > >emulation > > >(for life-cycle management of VMs), which our hypervisor may not support. PCI is mostly just 2

Re: [PATCH v6 02/25] iommu/ioasid: Add ioasid references

2020-04-30 Thread Jacob Pan
On Thu, 30 Apr 2020 11:39:31 -0700 Jacob Pan wrote: > > -void ioasid_free(ioasid_t ioasid) > > +bool ioasid_free(ioasid_t ioasid) > > { Sorry I missed this in the last reply. I think free needs to be unconditional since there is not a good way to fail it. Also can we have more symmetric APIs,

Re: [PATCH v6 17/25] iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind()

2020-04-30 Thread Jacob Pan
Hi Jean, A couple question on how SMMU handles CD.v and translation disable. On Thu, 30 Apr 2020 16:34:16 +0200 Jean-Philippe Brucker wrote: > The sva_bind() function allows devices to access process address > spaces using a PASID (aka SSID). > > (1) bind() allocates or gets an existing MMU no

Re: [PATCH v6 00/25] iommu: Shared Virtual Addressing for SMMUv3

2020-04-30 Thread Jacob Pan
On Thu, 30 Apr 2020 16:33:59 +0200 Jean-Philippe Brucker wrote: > Shared Virtual Addressing (SVA) allows to share process page tables > with devices using the IOMMU, PASIDs and I/O page faults. Add SVA > support to the Arm SMMUv3 driver. > > Since v5 [1]: > > * Added patches 1-3. Patch 1 adds a

Re: [PATCH 21/29] mm: remove the pgprot argument to __vmalloc

2020-04-30 Thread John Dorminy
>> On Tue, Apr 14, 2020 at 03:13:40PM +0200, Christoph Hellwig wrote: >> > The pgprot argument to __vmalloc is always PROT_KERNEL now, so remove >> > it. Greetings; I recently noticed this change via the linux-next tree. It may not be possible to edit at this late date, but the change descriptio

Re: [PATCH 21/29] mm: remove the pgprot argument to __vmalloc

2020-04-30 Thread John Dorminy
Greetings; I recently noticed this change via the linux-next tree. It may not be possible to edit at this late date, but the change description refers to PROT_KERNEL, which is a symbol which does not appear to exist; perhaps PAGE_KERNEL was meant? The mismatch caused me and a couple other folks s