Re: [PATCH 07/14] infiniband: utilize new device_add_cdev helper function

2017-02-21 Thread Jason Gunthorpe
On Tue, Feb 21, 2017 at 04:54:05PM -0700, Logan Gunthorpe wrote: > Is that true? Once device_register or device_add is called then you need > to use put_device. General rule is once kref_init has been called then you should use kref_put and not kfree. device_initialize ultimately calls

Re: [PATCH] device-dax: fix cdev leak

2017-02-23 Thread Jason Gunthorpe
gan Gunthorpe <log...@deltatee.com> > Reported-by: Jason Gunthorpe <jguntho...@obsidianresearch.com> > Signed-off-by: Dan Williams <dan.j.willi...@intel.com> > drivers/dax/dax.c |1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/dax/dax.c

Re: [PATCH 07/14] infiniband: utilize new device_add_cdev helper function

2017-02-21 Thread Jason Gunthorpe
device *device) > set_bit(devnum, dev_map); > } > > + device_initialize(_dev->dev); Ah, this needs more help. Once device_initialize is called put_device should be the error-unwind, can you use something more like this? >From ac7a35ea98066c9a9e3f3e54557c0c

Re: [PATCH v2 06/16] tpm-chip: utilize new cdev_device_add helper function

2017-02-27 Thread Jason Gunthorpe
; - dev_name(>dev), MAJOR(chip->dev.devt), > - MINOR(chip->dev.devt), rc); > - > - cdev_del(>cdev); > - return rc; > - } This will collide with the patch I sent: https://patchwork.k

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 02:42:12PM -0800, Dan Williams wrote: > > The crucial part for this discussion is the ability to fence and block > > DMA for a specific range. This is the hardware capability that lets > > page migration happen: fence DMA, migrate page, update page > > table in HCA, unblock

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-25 Thread Jason Gunthorpe
On Fri, Nov 25, 2016 at 02:22:17PM +0100, Christian König wrote: > >Like you say below we have to handle short lived in the usual way, and > >that covers basically every device except IB MRs, including the > >command queue on a NVMe drive. > > Well a problem which wasn't mentioned so far is that

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-25 Thread Jason Gunthorpe
On Fri, Nov 25, 2016 at 02:49:50PM -0500, Serguei Sagalovitch wrote: > GPU could perfectly access all VRAM. It is only issue for p2p without > special interconnect and CPU access. Strictly speaking as long as we > have "bus address" we could have RDMA but I agreed that for > RDMA we

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-25 Thread Jason Gunthorpe
On Fri, Nov 25, 2016 at 09:40:10PM +0100, Christian König wrote: > We call this "userptr" and it's just a combination of get_user_pages() on > command submission and making sure the returned list of pages stays valid > using a MMU notifier. Doesn't that still pin the page? > The "big" problem

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-25 Thread Jason Gunthorpe
On Fri, Nov 25, 2016 at 12:16:30PM -0500, Serguei Sagalovitch wrote: > b) Allocation may not have CPU address at all - only GPU one. But you don't expect RDMA to work in the case, right? GPU people need to stop doing this windowed memory stuff :) Jason

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 10:13:03AM -0700, Logan Gunthorpe wrote: > an MR would be very tricky. The MR may be relied upon by another host > and the kernel would have to inform user-space the MR was invalid then > user-space would have to tell the remote application. As Bart says, it would be best

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 10:40:47AM -0800, Dan Williams wrote: > I don't think that was designed for the case where the backing memory > is a special/static physical address range rather than anonymous > "System RAM", right? The hardware doesn't care where the memory is. ODP is just a generic

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-28 Thread Jason Gunthorpe
On Sun, Nov 27, 2016 at 04:02:16PM +0200, Haggai Eran wrote: > > Like in ODP, MMU notifiers/HMM are used to monitor for translation > > changes. If a change comes in the GPU driver checks if an executing > > command is touching those pages and blocks the MMU notifier until the > > command

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-28 Thread Jason Gunthorpe
On Mon, Nov 28, 2016 at 06:19:40PM +, Haggai Eran wrote: > > > GPU memory. We create a non-ODP MR pointing to VRAM but rely on > > > user-space and the GPU not to migrate it. If they do, the MR gets > > > destroyed immediately. > > That sounds horrible. How can that possibly work? What if the

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-28 Thread Jason Gunthorpe
On Mon, Nov 28, 2016 at 04:55:23PM -0500, Serguei Sagalovitch wrote: > >We haven't touch this in a long time and perhaps it changed, but there > >definitely was a call back in the PeerDirect API to allow the GPU to > >invalidate the mapping. That's what we don't want. > I assume that you are

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 02:11:29PM -0700, Logan Gunthorpe wrote: > > As I said, there is no possible special handling. Standard IB hardware > > does not support changing the DMA address once a MR is created. Forget > > about doing that. > > Yeah, that's essentially the point I was trying to make.

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 02:58:38PM -0500, Serguei Sagalovitch wrote: >We do not want to have "highly" dynamic translation due to >performance cost. We need to support "overcommit" but would >like to minimize impact. To support RDMA MRs for GPU/VRAM/PCIe >device memory (which is

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-24 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 06:25:21PM -0700, Logan Gunthorpe wrote: > > > On 23/11/16 02:55 PM, Jason Gunthorpe wrote: > >>> Only ODP hardware allows changing the DMA address on the fly, and it > >>> works at the page table level. We do not need special handling for

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2016 at 02:14:40PM -0500, Serguei Sagalovitch wrote: > > On 2016-11-23 02:05 PM, Jason Gunthorpe wrote: > >As Bart says, it would be best to be combined with something like > >Mellanox's ODP MRs, which allows a page to be evicted and then trigger > >

Re: Enabling peer to peer device transactions for PCIe devices

2016-12-06 Thread Jason Gunthorpe
On Tue, Dec 06, 2016 at 09:51:15AM -0700, Logan Gunthorpe wrote: > Hey, > > On 06/12/16 09:38 AM, Jason Gunthorpe wrote: > >>> I'm not opposed to mapping /dev/nvmeX. However, the lookup is trivial > >>> to accomplish in sysfs through /sys/dev/char to find the s

Re: Enabling peer to peer device transactions for PCIe devices

2017-01-12 Thread Jason Gunthorpe
On Thu, Jan 12, 2017 at 10:11:29AM -0500, Jerome Glisse wrote: > On Wed, Jan 11, 2017 at 10:54:39PM -0600, Stephen Bates wrote: > > > What we want is for RDMA, O_DIRECT, etc to just work with special VMAs > > > (ie. at least those backed with ZONE_DEVICE memory). Then > > > GPU/NVME/DAX/whatever

Re: Enabling peer to peer device transactions for PCIe devices

2016-11-30 Thread Jason Gunthorpe
On Wed, Nov 30, 2016 at 12:45:58PM +0200, Haggai Eran wrote: > > That just forces applications to handle horrible unexpected > > failures. If this sort of thing is needed for correctness then OOM > > kill the offending process, don't corrupt its operation. > Yes, that sounds fine. Can we simply

Re: Enabling peer to peer device transactions for PCIe devices

2016-12-05 Thread Jason Gunthorpe
On Mon, Dec 05, 2016 at 10:48:58AM -0800, Dan Williams wrote: > On Mon, Dec 5, 2016 at 10:39 AM, Logan Gunthorpe wrote: > > On 05/12/16 11:08 AM, Dan Williams wrote: > >> > >> I've already recommended that iopmem not be a block device and instead > >> be a device-dax

Re: Enabling peer to peer device transactions for PCIe devices

2016-12-05 Thread Jason Gunthorpe
On Mon, Dec 05, 2016 at 12:27:20PM -0700, Logan Gunthorpe wrote: > > > On 05/12/16 12:14 PM, Jason Gunthorpe wrote: > >But CMB sounds much more like the GPU case where there is a > >specialized allocator handing out the BAR to consumers, so I'm not > >sure a general

Re: Enabling peer to peer device transactions for PCIe devices

2016-12-05 Thread Jason Gunthorpe
On Mon, Dec 05, 2016 at 09:40:38AM -0800, Dan Williams wrote: > > If it is kernel only with physical addresess we don't need a uAPI for > > it, so I'm not sure #1 is at all related to iopmem. > > > > Most people who want #1 probably can just mmap > > /sys/../pci/../resourceX to get a user handle

Re: Enabling peer to peer device transactions for PCIe devices

2016-12-05 Thread Jason Gunthorpe
On Sun, Dec 04, 2016 at 07:23:00AM -0600, Stephen Bates wrote: > Hi All > > This has been a great thread (thanks to Alex for kicking it off) and I > wanted to jump in and maybe try and put some summary around the > discussion. I also wanted to propose we include this as a topic for LFS/MM >

Re: Enabling peer to peer device transactions for PCIe devices

2017-01-05 Thread Jason Gunthorpe
On Thu, Jan 05, 2017 at 01:39:29PM -0500, Jerome Glisse wrote: > 1) peer-to-peer because of userspace specific API like NVidia GPU > direct (AMD is pushing its own similar API i just can't remember > marketing name). This does not happen through a vma, this happens > through

Re: Enabling peer to peer device transactions for PCIe devices

2017-01-05 Thread Jason Gunthorpe
On Thu, Jan 05, 2017 at 02:54:24PM -0500, Jerome Glisse wrote: > Mellanox and NVidia support peer to peer with what they market a > GPUDirect. It only works without IOMMU. It is probably not upstream : > > https://www.mail-archive.com/linux-rdma@vger.kernel.org/msg21402.html > > I thought it

Re: Enabling peer to peer device transactions for PCIe devices

2017-01-05 Thread Jason Gunthorpe
On Thu, Jan 05, 2017 at 03:19:36PM -0500, Jerome Glisse wrote: > > Always having a VMA changes the discussion - the question is how to > > create a VMA that reprensents IO device memory, and how do DMA > > consumers extract the correct information from that VMA to pass to the > > kernel DMA API

Re: Enabling peer to peer device transactions for PCIe devices

2017-01-06 Thread Jason Gunthorpe
On Fri, Jan 06, 2017 at 12:37:22PM -0500, Jerome Glisse wrote: > On Fri, Jan 06, 2017 at 11:56:30AM -0500, Serguei Sagalovitch wrote: > > On 2017-01-05 08:58 PM, Jerome Glisse wrote: > > > On Thu, Jan 05, 2017 at 05:30:34PM -0700, Jason Gunthorpe wrote: > > > > On T

Re: Enabling peer to peer device transactions for PCIe devices

2017-01-05 Thread Jason Gunthorpe
On Thu, Jan 05, 2017 at 06:23:52PM -0500, Jerome Glisse wrote: > > I still don't understand what you driving at - you've said in both > > cases a user VMA exists. > > In the former case no, there is no VMA directly but if you want one than > a device can provide one. But such VMA is useless as

Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem

2017-04-04 Thread Jason Gunthorpe
On Tue, Apr 04, 2017 at 01:59:26PM +0300, Sagi Grimberg wrote: > Note that the nvme completion queues are still on the host memory, so > this means we have lost the ordering between data and completions as > they go to different pcie targets. Hmm, in this simple up/down case with a switch, I

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-19 Thread Jason Gunthorpe
On Wed, Apr 19, 2017 at 12:01:39PM -0600, Logan Gunthorpe wrote: > I'm just spit balling here but if HMM wanted to use unaddressable memory > as a DMA target, it could set that function to create a window ine gpu > memory, then call the pci_p2p_same_segment and return the result as the > dma

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-19 Thread Jason Gunthorpe
On Wed, Apr 19, 2017 at 01:02:49PM -0600, Logan Gunthorpe wrote: > > > On 19/04/17 12:32 PM, Jason Gunthorpe wrote: > > On Wed, Apr 19, 2017 at 12:01:39PM -0600, Logan Gunthorpe wrote: > > Not entirely, it would have to call through the whole process > > including

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Mon, Apr 17, 2017 at 08:23:16AM +1000, Benjamin Herrenschmidt wrote: > Thanks :-) There's a reason why I'm insisting on this. We have constant > requests for this today. We have hacks in the GPU drivers to do it for > GPUs behind a switch, but those are just that, ad-hoc hacks in the >

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-19 Thread Jason Gunthorpe
On Wed, Apr 19, 2017 at 01:41:49PM -0600, Logan Gunthorpe wrote: > > But.. it could point to a GPU and the GPU struct device could have a > > proxy dma_ops like Dan pointed out. > > Seems a bit awkward to me that in order for the intended use case, you > have to proxy the dma_ops. I'd probably

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 12:48:35PM -0700, Dan Williams wrote: > > Yes, I noticed this problem too and that makes sense. It just means > > every dma_ops will probably need to be modified to either support p2p > > pages or fail on them. Though, the only real difficulty there is that it > > will be

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 01:35:32PM -0600, Logan Gunthorpe wrote: > > Ultimately every dma_ops will need special code to support P2P with > > the special hardware that ops is controlling, so it makes some sense > > to start by pushing the check down there in the first place. This > > advice is

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 03:31:58PM -0600, Logan Gunthorpe wrote: > 1) It means that sg_has_p2p has to walk the entire sg and check every > page. Then map_sg_p2p/map_sg has to walk it again and repeat the check > then do some operation per page. If anyone is concerned about the > dma_map

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 12:30:59PM -0600, Logan Gunthorpe wrote: > > - The dma ops provider must be able to tell if source memory is bar > >mapped and recover the pci device backing the mapping. > > Do you mean to say that every dma-ops provider needs to be taught about > p2p backed pages?

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 02:11:33PM -0700, Dan Williams wrote: > > I think this opens an even bigger can of worms.. > > No, I don't think it does. You'd only shim when the target page is > backed by a device, not host memory, and you can figure this out by a > is_zone_device_page()-style lookup.

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 03:28:17PM -0700, Dan Williams wrote: > Unlike the pci bus address offset case which I think is fundamental to > support since shipping archs do this toda But we can support this by modifying those arch's unique dma_ops directly. Eg as I explained, my

Re: [RFC 0/8] Copy Offload with Peer-to-Peer PCI Memory

2017-04-18 Thread Jason Gunthorpe
On Tue, Apr 18, 2017 at 03:51:27PM -0700, Dan Williams wrote: > > This really seems like much less trouble than trying to wrapper all > > the arch's dma ops, and doesn't have the wonky restrictions. > > I don't think the root bus iommu drivers have any business knowing or > caring about dma

Re: [PATCH v2 15/21] xen-blkfront: Make use of the new sg_map helper function

2017-04-27 Thread Jason Gunthorpe
On Thu, Apr 27, 2017 at 02:19:24PM -0600, Logan Gunthorpe wrote: > > > On 26/04/17 01:37 AM, Roger Pau Monné wrote: > > On Tue, Apr 25, 2017 at 12:21:02PM -0600, Logan Gunthorpe wrote: > >> Straightforward conversion to the new helper, except due to the lack > >> of error path, we have to use

Re: [PATCH v2 15/21] xen-blkfront: Make use of the new sg_map helper function

2017-04-27 Thread Jason Gunthorpe
On Thu, Apr 27, 2017 at 05:03:45PM -0600, Logan Gunthorpe wrote: > > > On 27/04/17 04:11 PM, Jason Gunthorpe wrote: > > On Thu, Apr 27, 2017 at 03:53:37PM -0600, Logan Gunthorpe wrote: > > Well, that is in the current form, with more users it would make sense > > to o

Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu()

2017-10-10 Thread Jason Gunthorpe
On Mon, Oct 09, 2017 at 12:28:29PM -0700, Dan Williams wrote: > > I don't think this has ever come up in the context of an all-device MR > > invalidate requirement. Drivers already have code to invalidate > > specifc MRs, but to find all MRs that touch certain pages and then > > invalidate them

Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu()

2017-10-10 Thread Jason Gunthorpe
On Tue, Oct 10, 2017 at 10:39:27AM -0700, Dan Williams wrote: > On Tue, Oct 10, 2017 at 10:25 AM, Jason Gunthorpe > >> Have a look at the patch [1], I don't touch the ODP path. > > > > But, does ODP work OK already? I'm not clear on that.. > > It had better. If the

Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu()

2017-10-09 Thread Jason Gunthorpe
On Fri, Oct 06, 2017 at 03:35:54PM -0700, Dan Williams wrote: > otherwise be quiesced. The need for this knowledge is driven by a need > to make RDMA transfers to DAX mappings safe. If the DAX file's block map > changes we need to be to reliably stop accesses to blocks that have been > freed or

Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu()

2017-10-09 Thread Jason Gunthorpe
On Mon, Oct 09, 2017 at 12:05:30PM -0700, Dan Williams wrote: > On Mon, Oct 9, 2017 at 11:58 AM, Jason Gunthorpe > <jguntho...@obsidianresearch.com> wrote: > > On Fri, Oct 06, 2017 at 03:35:54PM -0700, Dan Williams wrote: > >> otherwise be quiesced. The need for this kn

Re: [PATCH v9 0/6] MAP_DIRECT for DAX userspace flush

2017-10-13 Thread Jason Gunthorpe
On Fri, Oct 13, 2017 at 08:14:55AM -0700, Dan Williams wrote: > scheme specific to RDMA which seems like a waste to me when we can > generically signal an event on the fd for any event that effects any > of the vma's on the file. The FL_LAYOUT lease impacts the entire file, > so as far as I can

Re: [PATCH v9 0/6] MAP_DIRECT for DAX userspace flush

2017-10-13 Thread Jason Gunthorpe
On Fri, Oct 13, 2017 at 11:22:21AM -0700, Dan Williams wrote: > > So, who should be responsible for MR coherency? Today we say the MPI > > is responsible. But we can't really expect the MPI > > to hook SIGIO and somehow try to reverse engineer what MRs are > > impacted from a FD that may not even

Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu()

2017-10-13 Thread Jason Gunthorpe
On Fri, Oct 13, 2017 at 08:50:47AM +0200, Christoph Hellwig wrote: > > However, chatting this over with a few more people I have an alternate > > solution that effectively behaves the same as how non-ODP hardware > > handles this case of hole punch / truncation today. So, today if this > >

Re: [PATCH v9 0/6] MAP_DIRECT for DAX userspace flush

2017-10-13 Thread Jason Gunthorpe
On Fri, Oct 13, 2017 at 10:01:04AM -0700, Dan Williams wrote: > On Fri, Oct 13, 2017 at 9:38 AM, Jason Gunthorpe > <jguntho...@obsidianresearch.com> wrote: > > On Fri, Oct 13, 2017 at 08:14:55AM -0700, Dan Williams wrote: > > > >> scheme specific to RDMA which see

Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu()

2017-10-12 Thread Jason Gunthorpe
On Tue, Oct 10, 2017 at 01:17:26PM -0700, Dan Williams wrote: > Also keep in mind that what triggers the lease break is another > application trying to write or punch holes in a file that is mapped > for RDMA. So, if the hardware can't handle the iommu mapping getting > invalidated asynchronously

Re: [PATCH v9 0/6] MAP_DIRECT for DAX userspace flush

2017-10-19 Thread Jason Gunthorpe
On Mon, Oct 16, 2017 at 03:02:52PM +0300, Sagi Grimberg wrote: > But why should the kernel ever need to mangle the CQ? if a lease break > would deregister the MR the device is expected to generate remote > protection errors on its own. The point is to avoid protection errors - hittles change over

Re: [PATCH 02/12] pci-p2p: Add sysfs group to display p2pmem stats

2018-01-04 Thread Jason Gunthorpe
On Thu, Jan 04, 2018 at 03:50:40PM -0600, Bjorn Helgaas wrote: > > This is similar to /sys/bus/pci/drivers_autoprobe, but > > affects only the VFs associated with a specific PF. > > + > > +What: /sys/bus/pci/devices/.../p2pmem/available > > I wonder if

Re: [LSF/MM TOPIC] Filesystem-DAX, page-pinning, and RDMA

2018-02-01 Thread Jason Gunthorpe
On Mon, Jan 29, 2018 at 06:33:25PM -0500, Jerome Glisse wrote: > Between i would also like to participate, in my view the burden should > be on GUP users, so if hardware is not ODP capable then you should at > least be able to kill the mapping/GUP and force the hardware to redo a > GUP if it get

Re: [PATCH v5 10/13] nvme-pci: Add support for P2P memory in requests

2018-09-04 Thread Jason Gunthorpe
On Thu, Aug 30, 2018 at 12:53:49PM -0600, Logan Gunthorpe wrote: > For P2P requests, we must use the pci_p2pmem_map_sg() function > instead of the dma_map_sg functions. > > With that, we can then indicate PCI_P2P support in the request queue. > For this, we create an NVME_F_PCI_P2P flag which

Re: [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory

2018-03-26 Thread Jason Gunthorpe
On Mon, Mar 26, 2018 at 12:11:38PM +0100, Jonathan Cameron wrote: > On Tue, 13 Mar 2018 10:43:55 -0600 > Logan Gunthorpe wrote: > > > On 12/03/18 09:28 PM, Sinan Kaya wrote: > > > On 3/12/2018 3:35 PM, Logan Gunthorpe wrote: > > > Regarding the switch business, It is amazing

Re: [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory

2018-03-26 Thread Jason Gunthorpe
On Mon, Mar 26, 2018 at 11:30:38AM -0600, Logan Gunthorpe wrote: > > > On 26/03/18 10:41 AM, Jason Gunthorpe wrote: > > On Mon, Mar 26, 2018 at 12:11:38PM +0100, Jonathan Cameron wrote: > >> On Tue, 13 Mar 2018 10:43:55 -0600 > >> Logan Gunthorpe <log...@delt

Re: [PATCH v2 07/10] nvme-pci: Use PCI p2pmem subsystem to manage the CMB

2018-03-05 Thread Jason Gunthorpe
On Mon, Mar 05, 2018 at 01:42:12PM -0700, Keith Busch wrote: > On Mon, Mar 05, 2018 at 01:10:53PM -0700, Jason Gunthorpe wrote: > > So when reading the above mlx code, we see the first wmb() being used > > to ensure that CPU stores to cachable memory are visible to the DM

Re: [PATCH 18/22] mm: mark DEVICE_PUBLIC as broken

2019-06-25 Thread Jason Gunthorpe
On Tue, Jun 25, 2019 at 09:29:15AM +0200, Christoph Hellwig wrote: > On Thu, Jun 20, 2019 at 09:26:48PM +0200, Michal Hocko wrote: > > On Thu 13-06-19 11:43:21, Christoph Hellwig wrote: > > > The code hasn't been used since it was added to the tree, and doesn't > > > appear to actually be usable.

Re: dev_pagemap related cleanups

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:43:03AM +0200, Christoph Hellwig wrote: > Hi Dan, Jérôme and Jason, > > below is a series that cleans up the dev_pagemap interface so that > it is more easily usable, which removes the need to wrap it in hmm > and thus allowing to kill a lot of code Do you want some of

Re: [PATCH 06/22] mm: factor out a devm_request_free_mem_region helper

2019-06-13 Thread Jason Gunthorpe
quest_mem_region(dev, addr, size, dev_name(dev)); > + if (!res) > + return ERR_PTR(-ENOMEM); > + res->desc = IORES_DESC_DEVICE_PRIVATE_MEMORY; I wonder if IORES_DESC_DEVICE_PRIVATE_MEMORY should be a function argument? Not really any substantive remark, so Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH 08/22] memremap: pass a struct dev_pagemap to ->kill

2019-06-13 Thread Jason Gunthorpe
tic void dev_dax_percpu_kill(struct percpu_ref *data) > +static void dev_dax_percpu_kill(struct dev_pagemap *pgmap) > { Looks like it was always like this, but I also can't see a reason to use the percpu as a handle for a dev_pagemap callback. Reviewed-by: Jason Gunthorpe Jason __

Re: [PATCH 20/22] mm: sort out the DEVICE_PRIVATE Kconfig mess

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:43:23AM +0200, Christoph Hellwig wrote: > The ZONE_DEVICE support doesn't depend on anything HMM related, just on > various bits of arch support as indicated by the architecture. Also > don't select the option from nouveau as it isn't present in many setups, > and

Re: [PATCH 09/22] memremap: lift the devmap_enable manipulation into devm_memremap_pages

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:43:12AM +0200, Christoph Hellwig wrote: > Just check if there is a ->page_free operation set and take care of the > static key enable, as well as the put using device managed resources. > diff --git a/mm/hmm.c b/mm/hmm.c > index c76a1b5defda..6dc769feb2e1 100644 > +++

Re: [PATCH 18/22] mm: mark DEVICE_PUBLIC as broken

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 12:53:02PM -0700, Ralph Campbell wrote: > > On 6/13/19 12:44 PM, Jason Gunthorpe wrote: > > On Thu, Jun 13, 2019 at 11:43:21AM +0200, Christoph Hellwig wrote: > > > The code hasn't been used since it was added to the tree, and doesn't > > >

Re: [PATCH 22/22] mm: don't select MIGRATE_VMA_HELPER from HMM_MIRROR

2019-06-13 Thread Jason Gunthorpe
ivers/gpu/drm/nouveau/Kconfig | 1 + > mm/Kconfig | 1 - > 2 files changed, 1 insertion(+), 1 deletion(-) Yes, the thing that calls migrate_vma() should be the thing that has the kconfig stuff. Reviewed-by: Jason Gunthorpe Jason _

Re: [PATCH 18/22] mm: mark DEVICE_PUBLIC as broken

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:43:21AM +0200, Christoph Hellwig wrote: > The code hasn't been used since it was added to the tree, and doesn't > appear to actually be usable. Mark it as BROKEN until either a user > comes along or we finally give up on it. > > Signed-off-by: Christoph Hellwig >

Re: [PATCH 17/22] mm: remove hmm_devmem_add

2019-06-13 Thread Jason Gunthorpe
.c | 115 -- > 3 files changed, 270 deletions(-) I looked for in-flight patches that might be using these APIs and found nothing. To be sent patches can use the new API with no loss in functionality... Reviewed-

Re: [PATCH 21/22] mm: remove the HMM config option

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:43:24AM +0200, Christoph Hellwig wrote: > All the mm/hmm.c code is better keyed off HMM_MIRROR. Also let nouveau > depend on it instead of the mix of a dummy dependency symbol plus the > actually selected one. Drop various odd dependencies, as the code is > pretty

Re: dev_pagemap related cleanups

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:27:39AM -0700, Dan Williams wrote: > On Thu, Jun 13, 2019 at 2:43 AM Christoph Hellwig wrote: > > > > Hi Dan, Jérôme and Jason, > > > > below is a series that cleans up the dev_pagemap interface so that > > it is more easily usable, which removes the need to wrap it in

Re: [PATCH 07/22] memremap: move dev_pagemap callbacks into a separate structure

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:43:10AM +0200, Christoph Hellwig wrote: > The dev_pagemap is a growing too many callbacks. Move them into a > separate ops structure so that they are not duplicated for multiple > instances, and an attacker can't easily overwrite them. Reviewed-by: Jason

Re: [PATCH 14/22] nouveau: use alloc_page_vma directly

2019-06-13 Thread Jason Gunthorpe
d, 2 insertions(+), 1 deletion(-) Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH 02/22] mm: remove the struct hmm_device infrastructure

2019-06-13 Thread Jason Gunthorpe
rd 'hmm_deviceX' name, without also defining a core kernel ABI for that char dev.. If this comes back it needs a proper explanation and review, with a user. Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https:/

Re: [PATCH 03/22] mm: remove hmm_devmem_add_resource

2019-06-13 Thread Jason Gunthorpe
iver to call devm_memremap_pages(), maybe even with all this boiler plate, in future. If we eventually get many users that need some simplified registration then we should add a devm_memremap_pages_simplified() interface and factor out that code when we can see the pattern. Reviewed-by: Jason Gunth

Re: [PATCH 01/22] mm: remove the unused ARCH_HAS_HMM_DEVICE Kconfig option

2019-06-13 Thread Jason Gunthorpe
ed the crazy process described in the commit message that (deliberately) introduced this unused kconfig. I also tried to find something in-flight for 5.3 that would need this and found nothing Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailin

Re: [PATCH 04/22] mm: don't clear ->mapping in hmm_devmem_free

2019-06-13 Thread Jason Gunthorpe
to start using mapping then the driver should be responsible to set it back to NULL before being done with the page. Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH 18/22] mm: mark DEVICE_PUBLIC as broken

2019-06-19 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 06:23:04PM -0700, John Hubbard wrote: > On 6/13/19 5:43 PM, Ira Weiny wrote: > > On Thu, Jun 13, 2019 at 07:58:29PM +0000, Jason Gunthorpe wrote: > >> On Thu, Jun 13, 2019 at 12:53:02PM -0700, Ralph Campbell wrote: > >>> > ... > >&g

Re: [PATCH 20/25] mm: remove hmm_vma_alloc_locked_page

2019-06-27 Thread Jason Gunthorpe
3 --- > mm/hmm.c| 14 -- > 2 files changed, 17 deletions(-) Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH 03/25] mm: remove hmm_devmem_add_resource

2019-06-27 Thread Jason Gunthorpe
map_pages > directly. > > Signed-off-by: Christoph Hellwig > Reviewed-by: Jason Gunthorpe > Reviewed-by: John Hubbard > Acked-by: Michal Hocko > --- > include/linux/hmm.h | 3 --- > mm/hmm.c| 50 - > 2 file

Re: [PATCH 12/25] memremap: add a migrate_to_ram method to struct dev_pagemap_ops

2019-06-27 Thread Jason Gunthorpe
-- > mm/memory.c | 9 ++--- > 6 files changed, 17 insertions(+), 67 deletions(-) Reviewed-by: Jason Gunthorpe I'ver heard there are some other use models for fault() here beyond migrate to ram, but we can rename it if we ever see them. > +static vm_fault_

Re: [PATCH 24/25] mm: remove the HMM config option

2019-06-27 Thread Jason Gunthorpe
le | 2 +- > mm/hmm.c| 2 -- > 6 files changed, 8 insertions(+), 33 deletions(-) Makes more sense to me too Reviewed-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org h

Re: dev_pagemap related cleanups v4

2019-07-02 Thread Jason Gunthorpe
On Tue, Jul 02, 2019 at 04:17:48PM -0700, Dan Williams wrote: > On Tue, Jul 2, 2019 at 11:42 AM Jason Gunthorpe wrote: > > > > On Mon, Jul 01, 2019 at 10:25:17AM +0200, Christoph Hellwig wrote: > > > And I've demonstrated that I can't send patch series.. While this

Re: [PATCH 16/25] device-dax: use the dev_pagemap internal refcount

2019-06-28 Thread Jason Gunthorpe
On Fri, Jun 28, 2019 at 10:10:12AM -0700, Dan Williams wrote: > On Fri, Jun 28, 2019 at 10:08 AM Dan Williams > wrote: > > > > On Fri, Jun 28, 2019 at 10:02 AM Jason Gunthorpe wrote: > > > > > > On Fri, Jun 28, 2019 at 09:27:44AM -0700, Dan Williams wrote:

Re: [PATCH 16/25] device-dax: use the dev_pagemap internal refcount

2019-06-28 Thread Jason Gunthorpe
On Fri, Jun 28, 2019 at 09:27:44AM -0700, Dan Williams wrote: > On Fri, Jun 28, 2019 at 8:39 AM Jason Gunthorpe wrote: > > > > On Wed, Jun 26, 2019 at 02:27:15PM +0200, Christoph Hellwig wrote: > > > The functionality is identical to the one currently open c

Re: add a not device managed memremap_pages v2

2019-08-16 Thread Jason Gunthorpe
On Fri, Aug 16, 2019 at 08:54:30AM +0200, Christoph Hellwig wrote: > Hi Dan and Jason, > > Bharata has been working on secure page management for kvmppc guests, > and one I thing I noticed is that he had to fake up a struct device > just so that it could be passed to the devm_memremap_pages >

Re: add a not device managed memremap_pages v2

2019-08-16 Thread Jason Gunthorpe
On Fri, Aug 16, 2019 at 02:36:07PM +0200, Christoph Hellwig wrote: > > > Changes since v1: > > > - don't overload devm_request_free_mem_region > > > - export the memremap_pages and munmap_pages as kvmppc can be a module > > > > What tree do we want this to go through? Dan are you running a

Re: [PATCH 2/5] resource: add a not device managed request_free_mem_region variant

2019-08-11 Thread Jason Gunthorpe
On Sun, Aug 11, 2019 at 10:12:44AM +0200, Christoph Hellwig wrote: > Just add a simple macro that passes a NULL dev argument to > dev_request_free_mem_region, and call request_mem_region in the > function for that particular case. > > Signed-off-by: Christoph Hellwig > include/linux/ioport.h |

Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

2019-08-11 Thread Jason Gunthorpe
On Sun, Aug 11, 2019 at 10:12:47AM +0200, Christoph Hellwig wrote: > The kvmppc ultravisor code wants a device private memory pool that is > system wide and not attached to a device. Instead of faking up one > provide a low-level memremap_pages for it. Note that this function is > not exported,

Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap

2019-08-21 Thread Jason Gunthorpe
On Wed, Aug 21, 2019 at 01:24:20PM -0300, Jason Gunthorpe wrote: > On Tue, Aug 20, 2019 at 07:58:22PM -0700, Dan Williams wrote: > > On Tue, Aug 20, 2019 at 6:27 AM Jason Gunthorpe wrote: > > > > > > On Mon, Aug 19, 2019 at 06:44:02PM -0700, Dan Williams wrote: > &

Re: add a not device managed memremap_pages v3

2019-08-20 Thread Jason Gunthorpe
On Sun, Aug 18, 2019 at 11:05:53AM +0200, Christoph Hellwig wrote: > Hi Dan and Jason, > > Bharata has been working on secure page management for kvmppc guests, > and one I thing I noticed is that he had to fake up a struct device > just so that it could be passed to the devm_memremap_pages >

Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap

2019-08-20 Thread Jason Gunthorpe
On Mon, Aug 19, 2019 at 06:44:02PM -0700, Dan Williams wrote: > On Sun, Aug 18, 2019 at 2:12 AM Christoph Hellwig wrote: > > > > The dev field in struct dev_pagemap is only used to print dev_name in > > two places, which are at best nice to have. Just remove the field > > and thus the name in

Re: [PATCH v3] libnvdimm: Enable unit test infrastructure compile checks

2019-09-02 Thread Jason Gunthorpe
s not result in a functional unit test environment, it is only a > helper for 0day to catch unit test build regressions. > > Note that there are a few x86isms in the implementation, so this does > not bother compile testing this architectures other than 64-bit x86. > > Cc: Jérôme

Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap

2019-08-21 Thread Jason Gunthorpe
On Tue, Aug 20, 2019 at 07:58:22PM -0700, Dan Williams wrote: > On Tue, Aug 20, 2019 at 6:27 AM Jason Gunthorpe wrote: > > > > On Mon, Aug 19, 2019 at 06:44:02PM -0700, Dan Williams wrote: > > > On Sun, Aug 18, 2019 at 2:12 AM Christoph Hellwig wrote: > > >

Re: [PATCH 16/25] device-dax: use the dev_pagemap internal refcount

2019-06-28 Thread Jason Gunthorpe
On Wed, Jun 26, 2019 at 02:27:15PM +0200, Christoph Hellwig wrote: > The functionality is identical to the one currently open coded in > device-dax. > > Signed-off-by: Christoph Hellwig > Reviewed-by: Ira Weiny > --- > drivers/dax/dax-private.h | 4 > drivers/dax/device.c | 43

Re: dev_pagemap related cleanups

2019-06-13 Thread Jason Gunthorpe
On Thu, Jun 13, 2019 at 11:21:01PM +0200, Christoph Hellwig wrote: > On Thu, Jun 13, 2019 at 08:40:46PM +0000, Jason Gunthorpe wrote: > > > Perhaps we should pull those out and resend them through hmm.git? > > > > It could be done - but how bad is the conflict resolu

Re: [PATCH v3 00/23] device-dax: Support sub-dividing soft-reserved ranges

2020-08-04 Thread Jason Gunthorpe
On Fri, Jul 31, 2020 at 08:24:58PM -0700, Dan Williams wrote: > - Fix test_hmm and other compilation fixups (Ralph) The hmm parts look OK Acked-by: Jason Gunthorpe Jason ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe s

Re: [PATCH v2 11/12] PM, libnvdimm: Add 'mem-quiet' state and callback for firmware activation

2020-07-09 Thread Jason Gunthorpe
On Thu, Jul 09, 2020 at 04:00:51PM +0100, Christoph Hellwig wrote: > On Mon, Jul 06, 2020 at 06:59:32PM -0700, Dan Williams wrote: > > The runtime firmware activation capability of Intel NVDIMM devices > > requires memory transactions to be disabled for 100s of microseconds. > > This timeout is

Re: [PATCH v2 11/12] PM, libnvdimm: Add 'mem-quiet' state and callback for firmware activation

2020-07-09 Thread Jason Gunthorpe
On Thu, Jul 09, 2020 at 09:10:06AM -0700, Dan Williams wrote: > On Thu, Jul 9, 2020 at 8:39 AM Jason Gunthorpe wrote: > > > > On Thu, Jul 09, 2020 at 04:00:51PM +0100, Christoph Hellwig wrote: > > > On Mon, Jul 06, 2020 at 06:59:32PM -0700, Dan Williams wrote: >

  1   2   >