[PATCH] x86/events/amd/iommu: Fix invalid Perf result due to IOMMU PMC power-gating

2021-05-03 Thread Suravee Suthikulpanit
On certain AMD platforms, when the IOMMU performance counter source (csource) field is zero, power-gating for the counter is enabled, which prevents write access and returns zero for read access. This can cause invalid perf result especially when event multiplexing is needed (i.e. more number of e

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-03 Thread David Gibson
On Mon, May 03, 2021 at 01:05:30PM -0300, Jason Gunthorpe wrote: > On Thu, Apr 29, 2021 at 01:20:22PM +1000, David Gibson wrote: > > > There is a certain appeal to having some > > > 'PPC_TCE_CREATE_SPECIAL_IOASID' entry point that has a wack of extra > > > information like windows that can be optio

Re: [PATCH 13/16] nvme-pci: Convert to using dma_map_sg_p2pdma for p2pdma pages

2021-05-03 Thread John Hubbard
On 5/3/21 10:19 AM, Logan Gunthorpe wrote: ... + nr_mapped = dma_map_sg_p2pdma_attrs(dev->dev, iod->sg, iod->nents, +rq_dma_dir(req), DMA_ATTR_NO_WARN); + if (nr_mapped < 0) { + if (nr_mapped != -ENOMEM) + ret = B

Re: [PATCH 12/16] nvme-pci: Check DMA ops when indicating support for PCI P2PDMA

2021-05-03 Thread John Hubbard
On 5/3/21 10:17 AM, Logan Gunthorpe wrote: ... blk_queue_flag_set(QUEUE_FLAG_NONROT, ns->queue); - if (ctrl->ops->flags & NVME_F_PCI_P2PDMA) + if (ctrl->ops->supports_pci_p2pdma && + ctrl->ops->supports_pci_p2pdma(ctrl)) This is a little excessive, as I suspected.

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread John Hubbard
On 5/3/21 9:55 AM, Logan Gunthorpe wrote: ... The same thing can be achieved with fewer lines and a bit more clarity. Can we please do it like this instead: for_each_sg(sgl, sg, nents, i) { if (sg_is_pci_p2pdma(sg)) sg_unmark_pci_p2pdma(sg);

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread John Hubbard
On 5/3/21 10:04 AM, Logan Gunthorpe wrote: Oops missed a comment: On 2021-05-02 5:28 p.m., John Hubbard wrote: int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents, enum dma_data_direction dir, unsigned long attrs) { - int i; + struct pc

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread John Hubbard
On 5/3/21 3:57 PM, Jason Gunthorpe wrote: On Mon, May 03, 2021 at 02:54:26PM -0700, John Hubbard wrote: I guess my main concern here is that there are these pci*() functions that somehow want to pass around struct device. Well, this is the main issue - helpers being used inside IOMMU code sho

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread Jason Gunthorpe
On Mon, May 03, 2021 at 02:54:26PM -0700, John Hubbard wrote: > I guess my main concern here is that there are these pci*() functions > that somehow want to pass around struct device. Well, this is the main issue - helpers being used inside IOMMU code should not be called pci* functions. This is

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread John Hubbard
On 5/3/21 11:56 AM, Logan Gunthorpe wrote: ... IMHO, it is better to have all of the pci_*() functions dealing with pci_dev instead of dev, but it is also true that this is a larger change, so I won't press the point too hard right now. As a general rule, I'd agree with you. However, it's not g

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:31 p.m., John Hubbard wrote: > On 5/3/21 9:30 AM, Logan Gunthorpe wrote: >> >> >> On 2021-05-02 2:41 p.m., John Hubbard wrote: >>> On 4/8/21 10:01 AM, Logan Gunthorpe wrote: All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a struct device (of the client

Re: [PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:35 p.m., Christoph Hellwig wrote: > On Mon, May 03, 2021 at 10:17:59AM -0600, Logan Gunthorpe wrote: >> I agree that some of this has evolved in a way that some of the names >> are a bit odd now. Could definitely use a cleanup, but that's not really >> part of this series. When

Re: [PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-03 Thread Christoph Hellwig
On Mon, May 03, 2021 at 10:17:59AM -0600, Logan Gunthorpe wrote: > I agree that some of this has evolved in a way that some of the names > are a bit odd now. Could definitely use a cleanup, but that's not really > part of this series. When I have some time I can look at doing a cleanup > series to

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread John Hubbard
On 5/3/21 9:30 AM, Logan Gunthorpe wrote: On 2021-05-02 2:41 p.m., John Hubbard wrote: On 4/8/21 10:01 AM, Logan Gunthorpe wrote: All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a struct device (of the client doing the DMA transfer). Thus move the conversion to struct pci_dev

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:28 p.m., Christoph Hellwig wrote: > On Tue, Apr 27, 2021 at 08:01:13PM -0300, Jason Gunthorpe wrote: >> At a high level I'm OK with it. dma_map_sg_attrs() is the extra >> extended version of dma_map_sg(), it already has a different >> signature, a different return code is not ou

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-05-03 Thread Christoph Hellwig
On Tue, Apr 27, 2021 at 08:01:13PM -0300, Jason Gunthorpe wrote: > At a high level I'm OK with it. dma_map_sg_attrs() is the extra > extended version of dma_map_sg(), it already has a different > signature, a different return code is not out of the question. > > dma_map_sg() is just the simple eas

Re: [PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-03 Thread Christoph Hellwig
On Mon, May 03, 2021 at 10:08:34AM -0600, Logan Gunthorpe wrote: > Per above, I think the reference count is unnecessary. But I could wrap > it in a static function for clarity. (There's no reason to export this > function). A well documented helper function would really help to improve the code f

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread Christoph Hellwig
On Mon, May 03, 2021 at 11:17:31AM -0700, John Hubbard wrote: > That's the thing: memory failure should be exceedingly rare for this. > Therefore, just fail out entirely (which I don't expect we'll likely > ever see), instead of doing all this weird stuff to try to continue > on if you cannot alloc

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread John Hubbard
On 5/3/21 11:20 AM, Logan Gunthorpe wrote: ... That's the thing: memory failure should be exceedingly rare for this. Therefore, just fail out entirely (which I don't expect we'll likely ever see), instead of doing all this weird stuff to try to continue on if you cannot allocate a single page. If

Re: [PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-03 Thread John Hubbard
On 5/3/21 9:17 AM, Logan Gunthorpe wrote: Returning a "bridge distance" from a "get map type" routine is jarring, and I think it is because of a pre-existing problem: the above function is severely misnamed. Let's try renaming it (and the other one) to approximately: upstream_bridge_map_ty

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-03 12:17 p.m., John Hubbard wrote: > On 5/3/21 8:57 AM, Logan Gunthorpe wrote: >> >> >> On 2021-05-01 9:58 p.m., John Hubbard wrote: >>> Another odd thing: this used to check for memory failure and just give >>> up, and now it doesn't. Yes, I realize that it all still works at the >>>

Re: [PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-03 Thread John Hubbard
On 5/3/21 9:08 AM, Logan Gunthorpe wrote: ... By the way, pre-existing code comment: pci_p2pdma_whitelist[] seems really short. From a naive point of view, I'd expect that there must be a lot more CPUs/chipsets that can do pci p2p, what do you think? I wonder if we have to be so super strict, any

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread John Hubbard
On 5/3/21 8:57 AM, Logan Gunthorpe wrote: On 2021-05-01 9:58 p.m., John Hubbard wrote: Another odd thing: this used to check for memory failure and just give up, and now it doesn't. Yes, I realize that it all still works at the moment, but this is quirky and we shouldn't stop here. Instead, a

Re: [PATCH 13/16] nvme-pci: Convert to using dma_map_sg_p2pdma for p2pdma pages

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 7:34 p.m., John Hubbard wrote: >> if (iod->npages == 0) >> dma_pool_free(dev->prp_small_pool, nvme_pci_iod_list(req)[0], >>iod->first_dma); >> @@ -868,14 +857,13 @@ static blk_status_t nvme_map_data(struct nvme_dev >> *dev, struct requ

Re: [PATCH 12/16] nvme-pci: Check DMA ops when indicating support for PCI P2PDMA

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 7:29 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to >> replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops >> flags can be checked for PCI P2PDMA support. >> >> Signed-off-by:

Re: [PATCH 08/16] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 6:50 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() >> implementations. It takes an scatterlist segment that must point to a >> pci_p2pdma struct page and will map it if the mapping requires

Re: [PATCH 10/16] dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 6:32 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Add a flags member to the dma_map_ops structure with one flag to >> indicate support for PCI P2PDMA. >> >> Also, add a helper to check if a device supports PCI P2PDMA. >> >> Signed-off-by: Logan Gunthorp

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 5:32 p.m., John Hubbard wrote: > On 5/2/21 4:28 PM, John Hubbard wrote: >> On 4/8/21 10:01 AM, Logan Gunthorpe wrote: > ... >>> @@ -387,19 +388,37 @@ void dma_direct_unmap_sg(struct device *dev, struct >>> scatterlist *sgl, >> >> This routine now deserves a little bit of commentin

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread Logan Gunthorpe
Oops missed a comment: On 2021-05-02 5:28 p.m., John Hubbard wrote: >> int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int >> nents, >> enum dma_data_direction dir, unsigned long attrs) >> { >> -int i; >> +struct pci_p2pdma_map_state p2pdma_state = {};

Re: [PATCH 09/16] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 5:28 p.m., John Hubbard wrote: >> @@ -387,19 +388,37 @@ void dma_direct_unmap_sg(struct device *dev, struct >> scatterlist *sgl, > > This routine now deserves a little bit of commenting, now that it is > doing less obvious things. How about something like this: > > /* > * Unma

Re: [PATCH 07/16] PCI/P2PDMA: Make pci_p2pdma_map_type() non-static

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 4:44 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> pci_p2pdma_map_type() will be needed by the dma-iommu map_sg >> implementation because it will need to determine the mapping type >> ahead of actually doing the mapping to create the actual iommu mapping

Re: [PATCH 05/16] dma-mapping: Introduce dma_map_sg_p2pdma()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 3:23 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> dma_map_sg() either returns a positive number indicating the number >> of entries mapped or zero indicating that resources were not available >> to create the mapping. When zero is returned, it is always

Re: [PATCH 04/16] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagmap and device

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 2:41 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a >> struct device (of the client doing the DMA transfer). Thus move the >> conversion to struct pci_devs for the provider and client into

Re: [PATCH 03/16] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-05-03 Thread Logan Gunthorpe
On 2021-05-02 1:58 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> Attempt to find the mapping type for P2PDMA pages on the first >> DMA map attempt if it has not been done ahead of time. >> >> Previously, the mapping type was expected to be calculated ahead of >> time,

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-03 Thread Jason Gunthorpe
On Thu, Apr 29, 2021 at 01:04:05PM +1000, David Gibson wrote: > Again, I don't know enough about VDPA to make sense of that. Are we > essentially talking non-PCI virtual devices here? In which case you > could define the VDPA "bus" to always have one-device groups. It is much worse than that. W

Re: [PATCH 02/16] PCI/P2PDMA: Avoid pci_get_slot() which sleeps

2021-05-03 Thread Logan Gunthorpe
On 2021-05-01 11:35 p.m., John Hubbard wrote: > On 4/8/21 10:01 AM, Logan Gunthorpe wrote: >> In order to use upstream_bridge_distance_warn() from a dma_map function, >> it must not sleep. However, pci_get_slot() takes the pci_bus_sem so it >> might sleep. >> >> In order to avoid this, try to ge

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-03 Thread Jason Gunthorpe
On Thu, Apr 29, 2021 at 01:20:22PM +1000, David Gibson wrote: > > There is a certain appeal to having some > > 'PPC_TCE_CREATE_SPECIAL_IOASID' entry point that has a wack of extra > > information like windows that can be optionally called by the viommu > > driver and it remains well defined and des

Re: [PATCH 01/16] PCI/P2PDMA: Pass gfp_mask flags to upstream_bridge_distance_warn()

2021-05-03 Thread Logan Gunthorpe
On 2021-05-01 9:58 p.m., John Hubbard wrote: > Another odd thing: this used to check for memory failure and just give > up, and now it doesn't. Yes, I realize that it all still works at the > moment, but this is quirky and we shouldn't stop here. > > Instead, a cleaner approach would be to push

Re: [PATCH v5 14/16] dma-direct: Allocate memory from restricted DMA pool if available

2021-05-03 Thread Claire Chang
On Fri, Apr 23, 2021 at 9:46 PM Robin Murphy wrote: > > On 2021-04-22 09:15, Claire Chang wrote: > > The restricted DMA pool is preferred if available. > > > > The restricted DMA pools provide a basic level of protection against the > > DMA overwriting buffer contents at unexpected times. However,