Re: [PATCH v2 00/22] Add new DMA mapping operation for P2PDMA

2021-05-14 Thread Logan Gunthorpe
On 2021-05-14 8:00 a.m., Christoph Hellwig wrote: > On Thu, May 13, 2021 at 04:31:41PM -0600, Logan Gunthorpe wrote: >> 17 files changed, 570 insertions(+), 290 deletions(-) > > I'm a little worried about all this extra code for no new functionality > at all. Yes, this ser

Re: [PATCH v2 15/22] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-14 Thread Logan Gunthorpe
On 2021-05-14 7:57 a.m., Christoph Hellwig wrote: >> +for_each_sg(sgl, sg, nents, i) { >> +if (sg_is_dma_pci_p2pdma(sg)) { >> +sg_dma_unmark_pci_p2pdma(sg); >> +} else { > > Double space here. We also don't really need the curly braces to start

Re: [PATCH v2 06/22] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-14 Thread Logan Gunthorpe
On 2021-05-14 7:49 a.m., Christoph Hellwig wrote: > On Thu, May 13, 2021 at 04:31:47PM -0600, Logan Gunthorpe wrote: >> Attempt to find the mapping type for P2PDMA pages on the first >> DMA map attempt if it has not been done ahead of time. >> >> Previously, th

Re: [PATCH v2 14/22] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations

2021-05-14 Thread Logan Gunthorpe
On 2021-05-14 7:53 a.m., Christoph Hellwig wrote: > I think helpers for the dma mapping implementation should probably > go into dma-map-ops.h so that the header for the public API doesn't get > polluted with them. Ok. Will do for v3. Logan ___

Re: [PATCH v2 08/22] dma-mapping: Allow map_sg() ops to return negative error codes

2021-05-14 Thread Logan Gunthorpe
On 2021-05-14 7:51 a.m., Christoph Hellwig wrote: >> +int __dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int >> nents, >> enum dma_data_direction dir, unsigned long attrs); > > I don't think it makes sense to expose this __dma_map_sg_attrs helper. > Just keep it

[PATCH v2 05/22] PCI/P2PDMA: Print a warning if the host bridge is not in the whitelist

2021-05-13 Thread Logan Gunthorpe
If the host bridge is not in the whitelist print a warning in the calc_map_type_and_dist_warn() path detailing the vendor and device IDs that would need to be added to the whitelist. Suggested-by: Don Dutile Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 17 +++-- 1 file

[PATCH v2 08/22] dma-mapping: Allow map_sg() ops to return negative error codes

2021-05-13 Thread Logan Gunthorpe
multiple negative error codes. Legacy map_sg implementations can continue to return zero until they are all converted. Signed-off-by: Logan Gunthorpe --- include/linux/dma-map-ops.h | 8 ++-- include/linux/dma-mapping.h | 41 - kernel/dma/mapping.c| 13

[PATCH v2 07/22] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagemap and device

2021-05-13 Thread Logan Gunthorpe
All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a struct device (of the client doing the DMA transfer). Thus move the conversion to struct pci_devs for the provider and client into this function. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 29

[PATCH v2 06/22] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-13 Thread Logan Gunthorpe
. With this change it's no longer invalid to call pci_p2pdma_map_sg() before the mapping type is calculated so drop the WARN_ON when that is the case. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers

[PATCH v2 12/22] lib/scatterlist: Add flag for indicating P2PDMA segments in an SGL

2021-05-13 Thread Logan Gunthorpe
-off-by: Logan Gunthorpe Reviewed-by: John Hubbard --- drivers/pci/Kconfig | 2 +- include/linux/scatterlist.h | 50 ++--- 2 files changed, 47 insertions(+), 5 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 0c473d75e625

[PATCH v2 11/22] dma-iommu: Return error code from iommu_dma_map_sg()

2021-05-13 Thread Logan Gunthorpe
Pass through appropriate error codes from iommu_dma_map_sg() now that the error code will be passed through dma_map_sgtable(). Signed-off-by: Logan Gunthorpe --- drivers/iommu/dma-iommu.c | 20 +--- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/dma

[PATCH v2 09/22] dma-direct: Return appropriate error code from dma_direct_map_sg()

2021-05-13 Thread Logan Gunthorpe
Now that the map_sg() op expects error codes instead of return zero on error, convert dma_direct_map_sg() to return an error code. The only error to return presently is EINVAL if a page could not be mapped. Signed-off-by: Logan Gunthorpe --- kernel/dma/direct.c | 2 +- 1 file changed, 1

[PATCH v2 10/22] iommu: Return full error code from iommu_map_sg[_atomic]()

2021-05-13 Thread Logan Gunthorpe
Convert to ssize_t return code so the return code from __iommu_map() can be returned all the way down through dma_iommu_map_sg(). Signed-off-by: Logan Gunthorpe --- drivers/iommu/iommu.c | 15 +++ include/linux/iommu.h | 22 +++--- 2 files changed, 18 insertions

[PATCH v2 13/22] PCI/P2PDMA: Make pci_p2pdma_map_type() non-static

2021-05-13 Thread Logan Gunthorpe
pci_p2pdma_map_type() will be needed by the dma-iommu map_sg implementation because it will need to determine the mapping type ahead of actually doing the mapping to create the actual iommu mapping. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 24

[PATCH v2 22/22] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg()

2021-05-13 Thread Logan Gunthorpe
This interface is superseded by support in dma_map_sg() which now supports heterogeneous scatterlists. There are no longer any users, so remove it. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 65 -- include/linux/pci-p2pdma.h | 27

[PATCH v2 14/22] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations

2021-05-13 Thread Logan Gunthorpe
is useful for IOMMU dma_map_sg() implementations where the sg segment containing the page differs from the sg segment containing the DMA address. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 59 ++ include/linux/pci-p2pdma.h | 21

[PATCH v2 17/22] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-05-13 Thread Logan Gunthorpe
addresses. With this change, the flags variable in the dma_map_ops is set to DMA_F_PCI_P2PDMA_SUPPORTED to indicate support for P2PDMA pages. Signed-off-by: Logan Gunthorpe --- drivers/iommu/dma-iommu.c | 66 +++ 1 file changed, 60 insertions(+), 6 deletions

[PATCH v2 16/22] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-05-13 Thread Logan Gunthorpe
Add a flags member to the dma_map_ops structure with one flag to indicate support for PCI P2PDMA. Also, add a helper to check if a device supports PCI P2PDMA. Signed-off-by: Logan Gunthorpe --- include/linux/dma-map-ops.h | 10 ++ include/linux/dma-mapping.h | 5 + kernel/dma

[PATCH v2 15/22] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-13 Thread Logan Gunthorpe
the mapping operation should fail (and it will return -EREMOTEIO). SGL segments that contain PCI bus addresses are marked with sg_dma_mark_pci_p2pdma() and are ignored when unmapped. Signed-off-by: Logan Gunthorpe --- kernel/dma/direct.c | 44 ++-- 1

[PATCH v2 19/22] nvme-pci: Convert to using dma_map_sgtable()

2021-05-13 Thread Logan Gunthorpe
P2PDMA transfer is requested. When this happens, return BLK_STS_TARGET so the request isn't retried. Signed-off-by: Logan Gunthorpe --- drivers/nvme/host/pci.c | 69 + 1 file changed, 29 insertions(+), 40 deletions(-) diff --git a/drivers/nvme/host/pci.c

[PATCH v2 04/22] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-13 Thread Logan Gunthorpe
bridge's device to go away while references are held on child devices, so the first element should not be able to change and, thus, this should be safe. Introduce a static function called pci_host_bridge_dev() to obtain the host bridge's root device. Signed-off-by: Logan Gunthorpe --- drivers

[PATCH v2 21/22] RDMA/rw: use dma_map_sgtable()

2021-05-13 Thread Logan Gunthorpe
were unmapped with the incorrect number of nents. Signed-off-by: Logan Gunthorpe --- drivers/infiniband/core/rw.c | 75 +++- include/rdma/ib_verbs.h | 19 + 2 files changed, 51 insertions(+), 43 deletions(-) diff --git a/drivers/infiniband/core

[PATCH v2 20/22] RDMA/core: Introduce ib_dma_pci_p2p_dma_supported()

2021-05-13 Thread Logan Gunthorpe
for ib_uses_virt_dma(). Adding the dma_pci_p2pdma_supported() check allows switching away from pci_p2pdma_[un]map_sg(). Signed-off-by: Logan Gunthorpe --- drivers/nvme/target/rdma.c | 2 +- include/rdma/ib_verbs.h| 11 +++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git

[PATCH v2 00/22] Add new DMA mapping operation for P2PDMA

2021-05-13 Thread Logan Gunthorpe
d the code in dma_pci_p2pdma_supported() for clarity. (per Ira and Christoph) * Finished off using the new dma_map_sg_p2pdma() call in rdma_rw and removed the old pci_p2pdma_[un]map_sg(). (per Jason) -- Logan Gunthorpe (22): PCI/P2PDMA: Rename upstream_bridge_distance() and rework documentation PCI/P2PDMA: Us

[PATCH v2 01/22] PCI/P2PDMA: Rename upstream_bridge_distance() and rework documentation

2021-05-13 Thread Logan Gunthorpe
the function to calc_map_type_and_dist() and rework the documentation some to better describe the two pieces of information the function returns. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 63 ++-- 1 file changed, 32 insertions(+), 31 deletions

[PATCH v2 18/22] nvme-pci: Check DMA ops when indicating support for PCI P2PDMA

2021-05-13 Thread Logan Gunthorpe
Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops flags can be checked for PCI P2PDMA support. Signed-off-by: Logan Gunthorpe --- drivers/nvme/host/core.c | 3 ++- drivers/nvme/host/nvme.h | 2 +- drivers/nvme

[PATCH v2 02/22] PCI/P2PDMA: Use a buffer on the stack for collecting the acs list

2021-05-13 Thread Logan Gunthorpe
(and broken) return with -ENOMEM which contradicts the return type and the caller was not prepared for. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 6f90e9812f6e

[PATCH v2 03/22] PCI/P2PDMA: Cleanup type for return value of calc_map_type_and_dist()

2021-05-13 Thread Logan Gunthorpe
Instead of using an int for the return value of this function use the correct enum pci_p2pdma_map_type. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index

Re: [PATCH 11/16] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-05-11 Thread Logan Gunthorpe
On 2021-05-11 10:06 a.m., Don Dutile wrote: > On 4/8/21 1:01 PM, Logan Gunthorpe wrote: >> When a PCI P2PDMA page is seen, set the IOVA length of the segment >> to zero so that it is not mapped into the IOVA. Then, in finalise_sg(), >> apply the appropriate bus address to

Re: [PATCH 10/16] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-05-11 Thread Logan Gunthorpe
On 2021-05-11 10:06 a.m., Don Dutile wrote: > On 4/8/21 1:01 PM, Logan Gunthorpe wrote: >> Add a flags member to the dma_map_ops structure with one flag to >> indicate support for PCI P2PDMA. >> >> Also, add a helper to check if a device supports PCI P2PDMA. >>

Re: [PATCH 07/16] PCI/P2PDMA: Make pci_p2pdma_map_type() non-static

2021-05-11 Thread Logan Gunthorpe
On 2021-05-11 10:06 a.m., Don Dutile wrote: > On 4/8/21 1:01 PM, Logan Gunthorpe wrote: >> pci_p2pdma_map_type() will be needed by the dma-iommu map_sg >> implementation because it will need to determine the mapping type >> ahead of actually doing the mapping to create the

Re: [PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-11 Thread Logan Gunthorpe
On 2021-05-11 10:05 a.m., Don Dutile wrote: > ... add a flag (set for p2pdma use)  to the function to print out what the > root->devfn is, and what > the device is so the needed quirk &/or modification can added to handle when > this assumption fails; > or make it a prdebug that can be flipped

Re: [PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-11 Thread Logan Gunthorpe
On 2021-05-11 10:05 a.m., Don Dutile wrote: > On 4/8/21 1:01 PM, Logan Gunthorpe wrote: >> In order to use upstream_bridge_distance_warn() from a dma_map function, >> it must not sleep. However, pci_get_slot() takes the pci_bus_sem so it >> might sleep. >> >> I

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-11 Thread Logan Gunthorpe
On 2021-05-11 10:05 a.m., Don Dutile wrote: > On 5/1/21 11:58 PM, John Hubbard wrote: >> On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >>> In order to call upstream_bridge_distance_warn() from a dma_map function, >>> it must not sleep. The only reason it does slee

Re: [PATCH 11/16] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-05-06 Thread Logan Gunthorpe
Sorry, I think I missed responding to this one so here are the answers: On 2021-05-02 7:14 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> When a PCI P2PDMA page is seen, set the IOVA length of the segment >> to zero so that it is not mappe

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:31 p.m., John Hubbard wrote: > On 5/3/21 9:30 AM, Logan Gunthorpe wrote: >> >> >> On 2021-05-02 2:41 p.m., John Hubbard wrote: >>> On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >>>> All callers of pci_p2pdma_map_type() have a

Re: [PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:35 p.m., Christoph Hellwig wrote: > On Mon, May 03, 2021 at 10:17:59AM -0600, Logan Gunthorpe wrote: >> I agree that some of this has evolved in a way that some of the names >> are a bit odd now. Could definitely use a cleanup, but that's not really >> part

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:28 p.m., Christoph Hellwig wrote: > On Tue, Apr 27, 2021 at 08:01:13PM -0300, Jason Gunthorpe wrote: >> At a high level I'm OK with it. dma_map_sg_attrs() is the extra >> extended version of dma_map_sg(), it already has a different >> signature, a different return code is not

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:17 p.m., John Hubbard wrote: > On 5/3/21 8:57 AM, Logan Gunthorpe wrote: >> >> >> On 2021-05-01 9:58 p.m., John Hubbard wrote: >>> Another odd thing: this used to check for memory failure and just give >>> up, and now it doesn't.

Re: [PATCH 13/16] nvme-pci: Convert to using dma_map_sg_p2pdma for p2pdma pages

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 7:34 p.m., John Hubbard wrote: >> if (iod->npages == 0) >> dma_pool_free(dev->prp_small_pool, nvme_pci_iod_list(req)[0], >>iod->first_dma); >> @@ -868,14 +857,13 @@ static blk_status_t nvme_map_data(struct nvme_dev >> *dev, struct

Re: [PATCH 12/16] nvme-pci: Check DMA ops when indicating support for PCI P2PDMA

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 7:29 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to >> replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops >> flags can be checked for PCI P2PDMA supp

Re: [PATCH 08/16] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 6:50 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() >> implementations. It takes an scatterlist segment that must point to a >> pci_p2pdma struct page and will

Re: [PATCH 10/16] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 6:32 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Add a flags member to the dma_map_ops structure with one flag to >> indicate support for PCI P2PDMA. >> >> Also, add a helper to check if a device supports PCI P2PDMA.

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 5:32 p.m., John Hubbard wrote: > On 5/2/21 4:28 PM, John Hubbard wrote: >> On 4/8/21 10:01 AM, Logan Gunthorpe wrote: > ... >>> @@ -387,19 +388,37 @@ void dma_direct_unmap_sg(struct device *dev, struct >>> scatterlist *sgl, >> >&

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread Logan Gunthorpe
Oops missed a comment: On 2021-05-02 5:28 p.m., John Hubbard wrote: >> int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int >> nents, >> enum dma_data_direction dir, unsigned long attrs) >> { >> -int i; >> +struct pci_p2pdma_map_state p2pdma_state = {};

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 5:28 p.m., John Hubbard wrote: >> @@ -387,19 +388,37 @@ void dma_direct_unmap_sg(struct device *dev, struct >> scatterlist *sgl, > > This routine now deserves a little bit of commenting, now that it is > doing less obvious things. How about something like this: > > /* > *

Re: [PATCH 07/16] PCI/P2PDMA: Make pci_p2pdma_map_type() non-static

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 4:44 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> pci_p2pdma_map_type() will be needed by the dma-iommu map_sg >> implementation because it will need to determine the mapping type >> ahead of actually doing the mapping to cr

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 3:23 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> dma_map_sg() either returns a positive number indicating the number >> of entries mapped or zero indicating that resources were not available >> to create the mapping.

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 2:41 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a >> struct device (of the client doing the DMA transfer). Thus move the >> conversion to struct pci_devs for th

Re: [PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 1:58 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Attempt to find the mapping type for P2PDMA pages on the first >> DMA map attempt if it has not been done ahead of time. >> >> Previously, the mapping type was expected to be

Re: [PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-03 Thread Logan Gunthorpe
On 2021-05-01 11:35 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> In order to use upstream_bridge_distance_warn() from a dma_map function, >> it must not sleep. However, pci_get_slot() takes the pci_bus_sem so it >> might sleep. >> >

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-01 9:58 p.m., John Hubbard wrote: > Another odd thing: this used to check for memory failure and just give > up, and now it doesn't. Yes, I realize that it all still works at the > moment, but this is quirky and we shouldn't stop here. > > Instead, a cleaner approach would be to

Re: [PATCH 14/16] nvme-rdma: Ensure dma support when using p2pdma

2021-04-27 Thread Logan Gunthorpe
On 2021-04-27 1:47 p.m., Jason Gunthorpe wrote: > On Thu, Apr 08, 2021 at 11:01:21AM -0600, Logan Gunthorpe wrote: >> Ensure the dma operations support p2pdma before using the RDMA >> device for P2PDMA. This allows switching the RDMA driver from >> pci_p2pdma_map_sg()

Re: [PATCH 11/16] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-04-27 Thread Logan Gunthorpe
On 2021-04-27 1:43 p.m., Jason Gunthorpe wrote: > On Thu, Apr 08, 2021 at 11:01:18AM -0600, Logan Gunthorpe wrote: >> When a PCI P2PDMA page is seen, set the IOVA length of the segment >> to zero so that it is not mapped into the IOVA. Then, in finalise_sg(), >> appl

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-04-27 Thread Logan Gunthorpe
On 2021-04-27 1:40 p.m., Jason Gunthorpe wrote: > On Tue, Apr 27, 2021 at 04:33:51PM -0300, Jason Gunthorpe wrote: >> On Thu, Apr 08, 2021 at 11:01:16AM -0600, Logan Gunthorpe wrote: >>> Add PCI P2PDMA support for dma_direct_map_sg() so that it can map >>> PCI P2

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-04-27 Thread Logan Gunthorpe
On 2021-04-27 1:31 p.m., Jason Gunthorpe wrote: > On Thu, Apr 08, 2021 at 11:01:12AM -0600, Logan Gunthorpe wrote: >> +/* >> + * dma_maps_sg_attrs returns 0 on error and > 0 on success. >> + * It should never return a value < 0. >> + */ > > Also it

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-04-27 Thread Logan Gunthorpe
On 2021-04-27 1:22 p.m., Jason Gunthorpe wrote: > On Thu, Apr 08, 2021 at 11:01:12AM -0600, Logan Gunthorpe wrote: >> dma_map_sg() either returns a positive number indicating the number >> of entries mapped or zero indicating that resources were not available >> to create t

[PATCH 08/16] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations

2021-04-08 Thread Logan Gunthorpe
is useful for IOMMU dma_map_sg() implementations where the sg segment containing the page differs from the sg segment containing the DMA address. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 65 ++ include/linux/pci-p2pdma.h | 21

[PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-04-08 Thread Logan Gunthorpe
to inform the lower level implementations that P2PDMA pages are allowed and to warn if a caller introduces them into the regular dma_map_sg() interface. Signed-off-by: Logan Gunthorpe --- include/linux/dma-mapping.h | 15 +++ kernel/dma/mapping.c| 52

[PATCH 10/16] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-04-08 Thread Logan Gunthorpe
Add a flags member to the dma_map_ops structure with one flag to indicate support for PCI P2PDMA. Also, add a helper to check if a device supports PCI P2PDMA. Signed-off-by: Logan Gunthorpe --- include/linux/dma-map-ops.h | 3 +++ include/linux/dma-mapping.h | 5 + kernel/dma/mapping.c

[PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-04-08 Thread Logan Gunthorpe
() and are ignored when unmapped. Signed-off-by: Logan Gunthorpe --- kernel/dma/direct.c | 25 ++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 002268262c9a..108dfb4ecbd5 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma

[PATCH 07/16] PCI/P2PDMA: Make pci_p2pdma_map_type() non-static

2021-04-08 Thread Logan Gunthorpe
pci_p2pdma_map_type() will be needed by the dma-iommu map_sg implementation because it will need to determine the mapping type ahead of actually doing the mapping to create the actual iommu mapping. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 34

[PATCH 16/16] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg()

2021-04-08 Thread Logan Gunthorpe
This interface is superseded by the new dma_map_sg_p2pdma() interface which supports heterogeneous scatterlists. There are no longer any users, so remove it. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 67 -- include/linux/pci-p2pdma.h

[PATCH 13/16] nvme-pci: Convert to using dma_map_sg_p2pdma for p2pdma pages

2021-04-08 Thread Logan Gunthorpe
-by: Logan Gunthorpe --- drivers/nvme/host/pci.c | 28 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 14f092973792..a1ed07ff38b7 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -577,17

[PATCH 06/16] lib/scatterlist: Add flag for indicating P2PDMA segments in an SGL

2021-04-08 Thread Logan Gunthorpe
-off-by: Logan Gunthorpe --- drivers/pci/Kconfig | 2 +- include/linux/scatterlist.h | 49 ++--- 2 files changed, 46 insertions(+), 5 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 0c473d75e625..90b4bddb3300 100644 --- a/drivers

[PATCH 12/16] nvme-pci: Check DMA ops when indicating support for PCI P2PDMA

2021-04-08 Thread Logan Gunthorpe
Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops flags can be checked for PCI P2PDMA support. Signed-off-by: Logan Gunthorpe --- drivers/nvme/host/core.c | 3 ++- drivers/nvme/host/nvme.h | 2 +- drivers/nvme

[PATCH 11/16] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-04-08 Thread Logan Gunthorpe
-by: Logan Gunthorpe --- drivers/iommu/dma-iommu.c | 66 ++- 1 file changed, 58 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index af765c813cc8..ef49635f9819 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers

[PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-04-08 Thread Logan Gunthorpe
. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 473a08940fbc..2574a062a255 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -825,11 +825,18

[PATCH 14/16] nvme-rdma: Ensure dma support when using p2pdma

2021-04-08 Thread Logan Gunthorpe
Ensure the dma operations support p2pdma before using the RDMA device for P2PDMA. This allows switching the RDMA driver from pci_p2pdma_map_sg() to dma_map_sg_p2pdma(). Signed-off-by: Logan Gunthorpe --- drivers/nvme/target/rdma.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff

[PATCH 00/16] Add new DMA mapping operation for P2PDMA

2021-04-08 Thread Logan Gunthorpe
) * Finished off using the new dma_map_sg_p2pdma() call in rdma_rw and removed the old pci_p2pdma_[un]map_sg(). (per Jason) -- Logan Gunthorpe (16): PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn() PCI/P2PDMA: Avoid pci_get_slot() which sleeps PCI/P2PDMA: Attempt to set

[PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-04-08 Thread Logan Gunthorpe
All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a struct device (of the client doing the DMA transfer). Thus move the conversion to struct pci_devs for the provider and client into this function. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 29

[PATCH 15/16] RDMA/rw: use dma_map_sg_p2pdma()

2021-04-08 Thread Logan Gunthorpe
Drop the use of pci_p2pdma_map_sg() in favour of dma_map_sg_p2pdma(). The new interface allows mapping scatterlists that mix both regular and P2PDMA pages and will verify that the dma device can communicate with the device the pages are on. Signed-off-by: Logan Gunthorpe --- drivers/infiniband

[PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-04-08 Thread Logan Gunthorpe
ice list. It should be impossible for the host bridge's device to go away while references are held on child devices, so the first element should not be able to change and, thus, this should be safe. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 14 -- 1 file changed,

[PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-04-08 Thread Logan Gunthorpe
to be allocated. Signed-off-by: Logan Gunthorpe Acked-by: Bjorn Helgaas --- drivers/pci/p2pdma.c | 21 +++-- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 196382630363..bd89437faf06 100644 --- a/drivers/pci

Re: [RFC PATCH v2 09/11] block: Add BLK_STS_P2PDMA

2021-03-16 Thread Logan Gunthorpe
On 2021-03-16 2:00 a.m., Christoph Hellwig wrote: > On Thu, Mar 11, 2021 at 04:31:39PM -0700, Logan Gunthorpe wrote: >> Create a specific error code for when P2PDMA pages are passed to a block >> devices that cannot map them (due to no IOMMU support or ACS protections). >>

Re: [RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-03-16 Thread Logan Gunthorpe
On 2021-03-16 1:58 a.m., Christoph Hellwig wrote: > On Fri, Mar 12, 2021 at 11:27:46AM -0700, Logan Gunthorpe wrote: >> So then we reject the patches that make that change. Seems like an odd >> argument to say that we can't do something that won't cause problems >> beca

Re: [RFC PATCH v2 07/11] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-03-15 Thread Logan Gunthorpe
On 2021-03-12 7:36 p.m., Ira Weiny wrote: > On Thu, Mar 11, 2021 at 04:31:37PM -0700, Logan Gunthorpe wrote: > >> +int dma_pci_p2pdma_supported(struct device *dev) >^^^ > bool? Sure. > >> +{ >> +const struct dma_map_ops *ops = get_dma_ops(dev)

Re: [RFC PATCH v2 04/11] PCI/P2PDMA: Introduce pci_p2pdma_should_map_bus() and pci_p2pdma_bus_offset()

2021-03-15 Thread Logan Gunthorpe
On 2021-03-12 7:32 p.m., Ira Weiny wrote: > On Thu, Mar 11, 2021 at 04:31:34PM -0700, Logan Gunthorpe wrote: >> Introduce pci_p2pdma_should_map_bus() which is meant to be called by > ^ > pci_p2pdma_dma_map_type() ??? > >

Re: [RFC PATCH v2 04/11] PCI/P2PDMA: Introduce pci_p2pdma_should_map_bus() and pci_p2pdma_bus_offset()

2021-03-15 Thread Logan Gunthorpe
On 2021-03-12 6:38 p.m., Ira Weiny wrote: > On Thu, Mar 11, 2021 at 04:31:34PM -0700, Logan Gunthorpe wrote: >> Introduce pci_p2pdma_should_map_bus() which is meant to be called by >> DMA map functions to determine how to map a given p2pdma page. >> >> pci_p2pdma

Re: [RFC PATCH v2 02/11] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 1:57 p.m., Bjorn Helgaas wrote: > On Thu, Mar 11, 2021 at 04:31:32PM -0700, Logan Gunthorpe wrote: >> In order to use upstream_bridge_distance_warn() from a dma_map function, >> it must not sleep. However, pci_get_slot() takes the pci_bus_sem so it >> might

Re: [RFC PATCH v2 01/11] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 1:39 p.m., Bjorn Helgaas wrote: On Thu, Mar 11, 2021 at 04:31:31PM -0700, Logan Gunthorpe wrote: In order to call this function from a dma_map function, it must not sleep. The only reason it does sleep so to allocate the seqbuf to print which devices are within the ACS path

Re: [RFC PATCH v2 08/11] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 12:47 p.m., Robin Murphy wrote:    {    struct scatterlist *s, *cur = sg;    unsigned long seg_mask = dma_get_seg_boundary(dev); @@ -864,6 +865,20 @@ static int __finalise_sg(struct device *dev, struct scatterlist *sg, int nents,    sg_dma_address(s) =

Re: [RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 11:11 a.m., Robin Murphy wrote: > On 2021-03-12 16:24, Logan Gunthorpe wrote: >> >> >> On 2021-03-12 8:52 a.m., Robin Murphy wrote: >>>> + >>>>    sg->dma_address = dma_direct_map_page(dev, sg_page(sg), >>&g

Re: [RFC PATCH v2 00/11] Add support to dma_map_sg for P2PDMA

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 10:46 a.m., Robin Murphy wrote: > On 2021-03-12 16:18, Logan Gunthorpe wrote: >> >> >> On 2021-03-12 8:51 a.m., Robin Murphy wrote: >>> On 2021-03-11 23:31, Logan Gunthorpe wrote: >>>> Hi, >>>> >>>> This is a rew

Re: [RFC PATCH v2 08/11] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 8:52 a.m., Robin Murphy wrote: > On 2021-03-11 23:31, Logan Gunthorpe wrote: >> When a PCI P2PDMA page is seen, set the IOVA length of the segment >> to zero so that it is not mapped into the IOVA. Then, in finalise_sg(), >> apply the appropriate bus a

Re: [RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 8:52 a.m., Robin Murphy wrote: >> + >>   sg->dma_address = dma_direct_map_page(dev, sg_page(sg), >>   sg->offset, sg->length, dir, attrs); >>   if (sg->dma_address == DMA_MAPPING_ERROR) >> @@ -411,7 +440,7 @@ int dma_direct_map_sg(struct device *dev,

Re: [RFC PATCH v2 00/11] Add support to dma_map_sg for P2PDMA

2021-03-12 Thread Logan Gunthorpe
On 2021-03-12 8:51 a.m., Robin Murphy wrote: > On 2021-03-11 23:31, Logan Gunthorpe wrote: >> Hi, >> >> This is a rework of the first half of my RFC for doing P2PDMA in >> userspace >> with O_DIRECT[1]. >> >> The largest issue with that serie

Re: [RFC PATCH v2 11/11] nvme-pci: Convert to using dma_map_sg for p2pdma pages

2021-03-11 Thread Logan Gunthorpe
On 2021-03-11 4:59 p.m., Jason Gunthorpe wrote: > On Thu, Mar 11, 2021 at 04:31:41PM -0700, Logan Gunthorpe wrote: >> Convert to using dma_[un]map_sg() for PCI p2pdma pages. >> >> This should be equivalent, though support will be somewhat less >> (only dma-direct

[RFC PATCH v2 03/11] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-03-11 Thread Logan Gunthorpe
. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 2135fe69bb07..7f6836732bce 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -819,11 +819,18

[RFC PATCH v2 11/11] nvme-pci: Convert to using dma_map_sg for p2pdma pages

2021-03-11 Thread Logan Gunthorpe
Convert to using dma_[un]map_sg() for PCI p2pdma pages. This should be equivalent, though support will be somewhat less (only dma-direct and dma-iommu are currently supported). Signed-off-by: Logan Gunthorpe --- drivers/nvme/host/pci.c | 27 +++ 1 file changed, 7

[RFC PATCH v2 01/11] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-03-11 Thread Logan Gunthorpe
. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 21 +++-- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 196382630363..bd89437faf06 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -267,7

[RFC PATCH v2 00/11] Add support to dma_map_sg for P2PDMA

2021-03-11 Thread Logan Gunthorpe
ee.com/ [2] https://lore.kernel.org/linux-block/capcyv4ifgcrdotut8qr7pmfhmecghqgvre9g0rorgczcgve...@mail.gmail.com/ -- Logan Gunthorpe (11): PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn() PCI/P2PDMA: Avoid pci_get_slot() which sleeps PCI/P2PDMA: Attempt to set map_type

[RFC PATCH v2 09/11] block: Add BLK_STS_P2PDMA

2021-03-11 Thread Logan Gunthorpe
Create a specific error code for when P2PDMA pages are passed to a block devices that cannot map them (due to no IOMMU support or ACS protections). This makes request errors in these cases more informative of as to what caused the error. Signed-off-by: Logan Gunthorpe --- block/blk-core.c

[RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-03-11 Thread Logan Gunthorpe
() and are ignored when unmapped. Signed-off-by: Logan Gunthorpe --- kernel/dma/direct.c | 35 --- kernel/dma/mapping.c | 13 ++--- 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 002268262c9a..f326d32062dd

[RFC PATCH v2 08/11] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg

2021-03-11 Thread Logan Gunthorpe
-by: Logan Gunthorpe --- drivers/iommu/dma-iommu.c | 63 --- 1 file changed, 53 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index af765c813cc8..c0821e9051a9 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers

[RFC PATCH v2 07/11] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-03-11 Thread Logan Gunthorpe
Add a flags member to the dma_map_ops structure with one flag to indicate support for PCI P2PDMA. Also, add a helper to check if a device supports PCI P2PDMA. Signed-off-by: Logan Gunthorpe --- include/linux/dma-map-ops.h | 3 +++ include/linux/dma-mapping.h | 5 + kernel/dma/mapping.c

[RFC PATCH v2 05/11] lib/scatterlist: Add flag for indicating P2PDMA segments in an SGL

2021-03-11 Thread Logan Gunthorpe
requires adding an additional dependency on CONFIG_64BIT to CONFIG_PCI_P2PDMA. This should be acceptable as the majority of P2PDMA use cases are restricted to newer root complexes and roughly require the extra address space for memory BARs used in the transactions. Signed-off-by: Logan Gunthorpe

[RFC PATCH v2 10/11] nvme-pci: Check DMA ops when indicating support for PCI P2PDMA

2021-03-11 Thread Logan Gunthorpe
Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops flags can be checked for PCI P2PDMA support. Signed-off-by: Logan Gunthorpe --- drivers/nvme/host/core.c | 3 ++- drivers/nvme/host/nvme.h | 2 +- drivers/nvme

[RFC PATCH v2 02/11] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-03-11 Thread Logan Gunthorpe
ist. It should be impossible for the host bridges device to go away while references are held on child devices, so the first element should not change and this should be safe. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --

[RFC PATCH v2 04/11] PCI/P2PDMA: Introduce pci_p2pdma_should_map_bus() and pci_p2pdma_bus_offset()

2021-03-11 Thread Logan Gunthorpe
Introduce pci_p2pdma_should_map_bus() which is meant to be called by DMA map functions to determine how to map a given p2pdma page. pci_p2pdma_bus_offset() is also added to allow callers to get the bus offset if they need to map the bus address. Signed-off-by: Logan Gunthorpe --- drivers/pci

  1   2   3   >