On 09/05/18 07:40 AM, Christian König wrote:
> The key takeaway is that when any device has ATS enabled you can't
> disable ACS without breaking it (even if you unplug and replug it).
I don't follow how you came to this conclusion...
The ACS bits we'd be turning off are the ones that force TLP
On 10/05/18 08:16 AM, Stephen Bates wrote:
> Hi Christian
>
>> Why would a switch not identify that as a peer address? We use the PASID
>>together with ATS to identify the address space which a transaction
>>should use.
>
> I think you are conflating two types of TLPs here. If the de
On 10/05/18 11:11 AM, Stephen Bates wrote:
>> Not to me. In the p2pdma code we specifically program DMA engines with
>> the PCI bus address.
>
> Ah yes of course. Brain fart on my part. We are not programming the P2PDMA
> initiator with an IOVA but with the PCI bus address...
>
>> So regardl
On 10/05/18 12:41 PM, Stephen Bates wrote:
> Hi Jerome
>
>>Note on GPU we do would not rely on ATS for peer to peer. Some part
>>of the GPU (DMA engines) do not necessarily support ATS. Yet those
>>are the part likely to be use in peer to peer.
>
> OK this is good to know. I agree
On 5/11/2018 2:52 AM, Christian König wrote:
This only works when the IOVA and the PCI bus addresses never overlap.
I'm not sure how the IOVA allocation works but I don't think we
guarantee that on Linux.
I find this hard to believe. There's always the possibility that some
part of the system
On 5/11/2018 4:24 PM, Stephen Bates wrote:
All
Alex (or anyone else) can you point to where IOVA addresses are generated?
A case of RTFM perhaps (though a pointer to the code would still be
appreciated).
https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt
Some exceptions to IOVA
--
Thanks for the review Randy! I'll make the changes for the next time we
post the series.
On 22/05/18 03:24 PM, Randy Dunlap wrote:
>> +The first task an orchestrator driver must do is compile a list of
>> +all client drivers that will be involved in a given transaction. For
>> +example, the NVMe T
ot be supported by memremap()
and therefore will not be support PCI P2P and have no support for CMB.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/pci.c | 80 +++--
1 file changed, 45 insertions(+), 35 deletions(-)
diff --git a/drivers/nvme/host/pci.c b/drivers
: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
Reviewed-by: Christoph Hellwig
---
drivers/nvme/host/core.c | 4
drivers/nvme/host/nvme.h | 1 +
drivers/nvme/host/pci.c | 17 +
3 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers
a new capability bit to advertise whether this is possible for
future hardware.
This commit includes significant rework and feedback from Christoph
Hellwig.
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas # PCI pieces
---
drivers/pci/Kconfig
attributes which take a boolean or a PCI device.
Any boolean as accepted by strtobool() turn P2P on or off (such as 'y', 'n',
'1', '0', etc). Specifying a full PCI device name/BDF will select the
specific device.
Signed-off-by: Logan Gunthorpe
Acked-by:
NVMe
SSDs (Intel, Seagate, Samsung) and p2pdma devices (Eideticom,
Microsemi, Chelsio and Everspin) using switches from both Microsemi
and Broadcomm.
--
Logan Gunthorpe (13):
PCI/P2PDMA: Support peer-to-peer memory
PCI/P2PDMA: Add sysfs group to display p2pmem stats
PCI/P2PDMA: Add PCI p2pme
Introduce a quirk to use CMB-like memory on older devices that have
an exposed BAR but do not advertise support for using CMBLOC and
CMBSIZE.
We'd like to use some of these older cards to test P2P memory.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
---
drivers/nvme/host/n
Add a new directory in the driver API guide for PCI specific
documentation.
This is in preparation for adding a new PCI P2P DMA driver writers
guide which will go in this directory.
Signed-off-by: Logan Gunthorpe
Cc: Jonathan Corbet
Cc: Mauro Carvalho Chehab
Cc: Greg Kroah-Hartman
Cc: Vinod
converted to restructured text
at this time.
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas
Cc: Jonathan Corbet
---
Documentation/driver-api/pci/index.rst | 1 +
Documentation/driver-api/pci/p2pdma.rst | 170
2 files changed, 171 insertions(+)
create mode
he initial code]
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/target/configfs.c| 36
drivers/nvme/target/core.c| 138 +-
drivers/nvme/target/io-cmd-bdev.c | 3 +
drivers/nvme/target/nvmet.h | 13 +++
dr
Add a sysfs group to display statistics about P2P memory that is
registered in each PCI device.
Attributes in the group display the total amount of P2P memory, the
amount available and whether it is published or not.
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas
---
Documentation/ABI
QUEUE_FLAG_PCI_P2P is introduced meaning a driver's request queue
supports targeting P2P memory. This will be used by P2P providers and
orchestrators (in subsequent patches) to ensure block devices can
support P2P memory before submitting P2P backed pages to submit_bio().
Signed-off-by:
is P2P the entire SGL should be P2P.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Christoph Hellwig
Reviewed-by: Sagi Grimberg
---
drivers/infiniband/core/rw.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/rw.c b/drivers/infiniband/core
emory.
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas
---
drivers/pci/p2pdma.c | 43 ++
include/linux/memremap.h | 1 +
include/linux/pci-p2pdma.h | 7 +++
3 files changed, 51 insertions(+)
diff --git a/drivers/pci/p2pdma.c b/driver
et earlier and cleared on any error.
It also seems to be unnecessary to accumulate the length as the map_sgl
functions should only ever be called once per request.
Signed-off-by: Logan Gunthorpe
Acked-by: Sagi Grimberg
Cc: Christoph Hellwig
---
drivers/nvme/target/core.c | 18 +
On 2018-09-27 11:12 AM, Keith Busch wrote:
> Reviewed-by: Keith Busch
Thanks for the reviews Keith!
Logan
MA to io memory, memory that is unmappable, or memory that
doesn't have struct page backings.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/pfn_t.h | 37 -
include/linux/scatterlist.h | 31 ++
r other's
opinions.
This series is based on v4.12-rc2 and a git tree is available here:
https://github.com/sbates130272/linux-p2pmem.git io_pfn_t
Thanks for your time,
Logan
[1] https://lkml.org/lkml/2017/4/25/738
Logan Gunthorpe (16):
dmaengine: ste_dma40, imx-dma: Cleanup scatterlist
Add a default ioremap function which was not provided in all
circumstances. (Only when CONFIG_PCI and CONFIG_TILEGX was set).
I have designs to use them in scatterlist.c where they'd likely never
be called with this architecture, but it is needed to compile.
Signed-off-by: Logan Gunt
ut it does need
to compile.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
arch/um/include/asm/io.h | 17 +
1 file changed, 17 insertions(+)
create mode 100644 arch/um/include/asm/io.h
diff --git a/arch/um/include/asm/io.h b/arch/um/include/asm/io.h
new file m
This is a layering violation so we replace it with calls to
sg_page. This is a prep patch for replacing page_link and this
is one of the very few uses outside of scatterlist.h.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
drivers/staging/ccree/ssi_buffer_mgr.c | 17
Going straight from pfn_t to physical address is cheaper and
avoids the potential BUG_ON in bvec_page for unmappable memory.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/bio.h | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/include
affect is that, on 32 bit systems, the sgl entry will be
32 bits larger seeing pfn_t is always 64 bits and the unsigned long it
replaced was 32 bits. However, it should still fit the same
SG_CHUNK_SIZE entries into a single page so this probably isn't a huge
issue.
Signed-off-by: Logan Gunt
Introduce two accessor functions for bv_page: bvec_page to return the
page and bvec_set_page. A follow on patch will mechanically convert all the
individual uses within the kernel.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/bvec.h | 10 ++
1 file
d-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/dma-mapping.h | 9 +++--
include/linux/scatterlist.h | 16
2 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..5e
We introduce bio_add_pfn which is the same as bio_add_page but expects a
pfn_t instead of a page pointer. bio_add_page is then converted to be
an inline call to bio_add_pfn.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
block/bio.c | 16
include
This is a layering violation so we replace the uses with calls to
sg_page(). This is a prep patch for replacing page_link and this
is one of the very few uses outside of scatterlist.h.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
samples/kfifo/dma-example.c | 8
1
actually a struct.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
block/blk-integrity.c | 4 ++--
block/blk-merge.c | 6 +++---
include/linux/bvec.h | 13 +
3 files changed, 14 insertions(+), 9 deletions(-)
diff --git a/block/blk-integrity.c b/block/blk
len;
expression offset;
@@
-sg_set_page(sg, bvec_page(&bv), len, offset);
+sg_set_pfn(sg, bv.bv_pfn, len, offset);
@@
expression sg;
expression bv;
expression len;
expression offset;
@@
-sg_set_page(sg, bvec_page(bv), len, offset);
+sg_set_pfn(sg, bv->bv_pfn, len, offset);
Signed-off-by
18593/
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
drivers/nvme/host/pci.c | 3 ++-
drivers/nvme/target/Kconfig | 12
drivers/nvme/target/io-cmd.c | 2 +-
drivers/nvme/target/rdma.c | 29 +
4 files changed, 40 insertions(
Two dma engine drivers directly accesses page_link assuming knowledge
that should be contained only in scatterlist.h.
We replace these with calls to sg_chain and sg_assign_page.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
drivers/dma/imx-dma.c | 7 ++-
drivers/dma
support IO memory by simply calling
the appropriate memcpy when required.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/scatterlist.h | 3 +++
lib/scatterlist.c | 35 +--
2 files changed, 32 insertions(+), 6 deletions(-)
diff
each major number in the correct
order, regardless of where they are stored in the hash table.
In order to do this, we introduce BLKDEV_MAJOR_MAX as an artificial
limit (chosen to be 512). It will then print all devices in major
order number from 0 to the maximum.
Signed-off-by: Logan Gunthorpe
Hey,
I found a regression in v5.0-rc1 this morning. My system panics on boot.
I've attached a log of the panic.
I bisected to find the problematic commit is:
Fixes: 9d037ad707ed ("block: remove req->timeout_list")
But it makes no sense to me why this commit would cause a problem like
this. I've
On 2019-01-08 6:19 a.m., Christoph Hellwig wrote:
> On Mon, Jan 07, 2019 at 12:41:06PM -0700, Logan Gunthorpe wrote:
>> Hey,
>>
>> I found a regression in v5.0-rc1 this morning. My system panics on boot.
>> I've attached a log of the panic.
>>
>>
On 2019-01-08 6:49 a.m., Jeff Moyer wrote:
>> The traceback seems to indicate the problem is on the bip_get_seed()
>> line in t10_pi_complete(). Which suggests that bio_integrity() is
>> returning NULL.
>
> Does your hardware actually support protection information?
As far as I know, I do not.
t_set_guard().
Fixes: 3d2d75254915 ("[SCSI] isci: T10 DIF support")
Link: http://lkml.kernel.org/r/da851333-eadd-163a-8c78-e1f4ec5ec...@deltatee.com
Signed-off-by: Logan Gunthorpe
Cc: Intel SCU Linux support
Cc: Artur Paszkiewicz
Cc: "James E.J. Bottomley"
Cc: "Martin
On 2019-01-07 12:41 p.m., Logan Gunthorpe wrote:
> I found a regression in v5.0-rc1 this morning. My system panics on boot.
> I've attached a log of the panic.
I just sent a patch which, I believe, resolves the regression:
http://lkml.kernel.org/r/20190108205043.3122-1-log...@d
Hi Christoph,
Thanks so much for the detailed review of the code! Even though by the
sounds of things we will be moving to device dax and most of this is
moot. Still, it's great to get some feedback and learn a few things.
I've given some responses below.
On 28/10/16 12:45 AM, Christoph Hellwig
Hi guys,
We were looking at testing the new IO polling improvements and we built
a kernel from the 'for-4.10/dio' (64ead7d) branch in linux-block.
However this branch seems to cause hung tasks when booted. Most
noticeably, dhclient seems to always hang as it tries to read from it's
leases file, an
Hi Jens,
Thanks for the quick reply. I just built 82a78cd and I'm seeing the same
problem as reported.
Logan
On 08/11/16 11:21 AM, Jens Axboe wrote:
> On 11/08/2016 11:16 AM, Logan Gunthorpe wrote:
>> Hi guys,
>>
>> We were looking at testing the new IO polling impr
Hey,
I haven't check 82a78cd, but when I tried reverting the commit in
yesterdays version there were conflicts, as a subsequent patch removed
the defines that the specific patch operated on.
Logan
On 08/11/16 12:01 PM, Jens Axboe wrote:
> On 11/08/2016 11:59 AM, Logan Gunthorpe wrot
Hey,
On 08/11/16 12:19 PM, Jens Axboe wrote:
> Can you try and boot for-4.10/block instead?
Yup. I'm seeing the same issue with that branch too. (b57d74a)
Thanks,
Logan
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.or
Hey,
I just tested with the latest for-4.10/block branch and it looks like it
fixed our problem.
Thanks!
Logan
On 08/11/16 07:55 PM, Damien Le Moal wrote:
>
> Jens,
>
> On 11/9/16 11:45, Jens Axboe wrote:
>> I just committed the work-around. But yes, let's have a logical revert
>> and require
I_P2P
flag set.
Signed-off-by: Logan Gunthorpe
---
block/blk-core.c | 3 +++
include/linux/blk_types.h | 18 +-
include/linux/blkdev.h| 2 ++
3 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index b888175
Attributes display the total amount of P2P memory, the ammount available
and whether it is published or not.
Signed-off-by: Logan Gunthorpe
---
Documentation/ABI/testing/sysfs-bus-pci | 25
drivers/pci/p2p.c | 51 +
2 files
From: Christoph Hellwig
Refactor the call to nvme_map_cmb, and change the conditions for probing
for the CMB. First remove the version check as NVMe TPs always apply
to earlier versions of the spec as well. Second check for the whole CMBSZ
register for support of the CMB feature instead of just
ar RC supports P2P transfers is non-trivial. Additionally, the
benefits of P2P transfers that go through the RC is limited to only
reducing DRAM usage.
This commit includes significant rework and feedback from Christoph
Hellwig.
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
driver
Introduce a quirk to use CMB-like memory on older devices that have
an exposed BAR but do not advertise support for using CMBLOC and
CMBSIZE.
We'd like to use some of these older cards to test P2P memory.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/nvme.h | 7 +++
drivers
In order to use PCI P2P memory pci_p2pmem_[un]map_sg() functions must be
called to map the correct DMA address. To do this, we add a flags
variable and the RDMA_RW_CTX_FLAG_PCI_P2P flag. When the flag is
specified use the appropriate map function.
Signed-off-by: Logan Gunthorpe
---
drivers
-by: Logan Gunthorpe
---
drivers/nvme/host/core.c | 4
drivers/nvme/host/nvme.h | 1 +
drivers/nvme/host/pci.c | 18 ++
3 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1e46e60b8f10..8a7caaa5ee49
ory will fail.
This series is based off of Christoph's v3 series to revamp
dev_pagemap. A git repo of the patches is available here[2].
Logan
Christoph Hellwig (2):
nvme-pci: clean up CMB initialization
nvme-pci: clean up SMBSZ bit definitions
Logan Gunthorpe (10):
pci-p2p: Support peer
devices involved in transactions with the p2pmem.
A count of the number of requests to disable the flags is maintained.
When the count transitions from 1 to 0, the old flags are restored.
Signed-off-by: Logan Gunthorpe
---
drivers/pci/p2p.c | 144
Register the CMB buffer as p2pmem and use the appropriate allocation
functions to create and destroy the IO SQ.
If the CMB supports WDS and RDS, publish it for use as p2p memory
by other devices.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/pci.c | 74
The DMA address used when mapping PCI P2P memory must be the PCI bus
address. Thus, introduce pci_p2pmem_[un]map_sg() to map the correct
addresses when using P2P memory.
For this, we assume that an SGL passed to these functions contain all
p2p memory or no p2p memory.
Signed-off-by: Logan
From: Christoph Hellwig
Define the bit positions instead of macros using the magic values,
and move the expanded helpers to calculate the size and size unit into
the implementation C file.
Signed-off-by: Christoph Hellwig
---
drivers/nvme/host/pci.c | 23 +--
include/linux/
f-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/target/configfs.c | 29 +
drivers/nvme/target/core.c | 95 +-
drivers/nvme/target/io-cmd.c | 3 ++
drivers/nvme/target/nvmet.h| 10 +
drivers/nvme/target/
On 04/01/18 12:01 PM, Logan Gunthorpe wrote:
From: Christoph Hellwig
Define the bit positions instead of macros using the magic values,
and move the expanded helpers to calculate the size and size unit into
the implementation C file.
Signed-off-by: Christoph Hellwig
Oops, I forgot
On 04/01/18 12:01 PM, Logan Gunthorpe wrote:
From: Christoph Hellwig
Refactor the call to nvme_map_cmb, and change the conditions for probing
for the CMB. First remove the version check as NVMe TPs always apply
to earlier versions of the spec as well. Second check for the whole CMBSZ
On 04/01/18 12:22 PM, Jason Gunthorpe wrote:
This seems really clunky since we are going to want to do this same
logic all over the place.
I'd be much happier if dma_map_sg can tell the memory is P2P or not
from the scatterlist or dir arguments and not require the callers to
have this.
We tr
Thanks for the speedy review!
On 04/01/18 02:40 PM, Bjorn Helgaas wrote:
Run "git log --oneline drivers/pci" and follow the convention. I
think it would make sense to add a new tag like "PCI/P2P", although
"P2P" has historically also been used in the "PCI-to-PCI bridge"
context, so maybe there'
On 04/01/18 02:50 PM, Bjorn Helgaas wrote:
On Thu, Jan 04, 2018 at 12:01:27PM -0700, Logan Gunthorpe wrote:
Attributes display the total amount of P2P memory, the ammount available
and whether it is published or not.
s/ammount/amount/ (also below)
Will fix.
I wonder if "p2pdma"
On 04/01/18 03:13 PM, Jason Gunthorpe wrote:
On Thu, Jan 04, 2018 at 12:52:24PM -0700, Logan Gunthorpe wrote:
We tried things like this in an earlier iteration[1] which assumed the SG
was homogenous (all P2P or all regular memory). This required serious
ugliness to try and ensure SGs were in
On 04/01/18 03:35 PM, Alex Williamson wrote:
Yep, flipping these ACS bits invalidates any IOMMU groups that depend
on the isolation of that downstream port and I suspect also any peers
within the same PCI slot of that port and their downstream devices. The
entire sub-hierarchy grouping needs t
On 04/01/18 02:59 PM, Bjorn Helgaas wrote:
On Thu, Jan 04, 2018 at 12:01:26PM -0700, Logan Gunthorpe wrote:
Some PCI devices may have memory mapped in a BAR space that's
intended for use in Peer-to-Peer transactions. In order to enable
such transactions the memory must be registered
On 04/01/18 05:00 PM, Logan Gunthorpe wrote:
>
>
> On 04/01/18 03:35 PM, Alex Williamson wrote:
>> Yep, flipping these ACS bits invalidates any IOMMU groups that depend
>> on the isolation of that downstream port and I suspect also any peers
>> within the same PCI
On 04/01/18 08:33 PM, Alex Williamson wrote:
That's exactly what IOMMU groups represent, the smallest set of devices
which have DMA isolation from other devices. By poking this hole, the
IOMMU group is invalid. We cannot turn off ACS only for a specific
device, in order to enable p2p it needs
On 05/01/18 08:30 AM, Marta Rybczynska wrote:
@@ -429,10 +429,7 @@ static void __nvme_submit_cmd(struct nvme_queue *nvmeq,
{
u16 tail = nvmeq->sq_tail;
- if (nvmeq->sq_cmds_io)
- memcpy_toio(&nvmeq->sq_cmds_io[tail], cmd, sizeof(*cmd));
- else
-
On 05/01/18 11:11 AM, Keith Busch wrote:
On Thu, Jan 04, 2018 at 12:01:34PM -0700, Logan Gunthorpe wrote:
Register the CMB buffer as p2pmem and use the appropriate allocation
functions to create and destroy the IO SQ.
If the CMB supports WDS and RDS, publish it for use as p2p memory
by other
On 05/01/18 12:01 PM, Keith Busch wrote:
On Fri, Jan 05, 2018 at 11:19:28AM -0700, Logan Gunthorpe wrote:
Although it is not explicitly stated anywhere, pci_alloc_p2pmem() should
always be at least 4k aligned. This is because the gen_pool that implements
it is created with PAGE_SHIFT for its
On 08/01/18 11:09 AM, Jason Gunthorpe wrote:
It could, if we had a DMA op for p2p then the drivers that provide
their own ops can implement it appropriately or not at all.
I was thinking of doing something like this. I'll probably rough out a
patch and send it along today or tomorrow.
If
On 08/01/18 11:34 AM, Christoph Hellwig wrote:
But P2P is _not_ a factor of the dma_ops implementation at all,
it is something that happens behind the dma_map implementation.
Think about what the dma mapping routines do:
(a) translate from host address to bus addresses
and
(b) flush cac
On 08/01/18 11:57 AM, Christoph Hellwig wrote:
It does, sort of - but in a different way then the normal DMA map
ops. And only to work around the fact that we need to map our
P2P space into struct pages. Without that we could just pass the
bus address around, but the Linux stack and VM isn't
ot;cd->major != i"),
as it will never be true.
Signed-off-by: Srivatsa S. Bhat
Reviewed-by: Logan Gunthorpe
Logan
(511) ..."
(and also fix off-by-one bugs in the error prints).
While at it, also update the comment describing register_blkdev().
Signed-off-by: Srivatsa S. Bhat
Reviewed-by: Logan Gunthorpe
Register the CMB buffer as p2pmem and use the appropriate allocation
functions to create and destroy the IO SQ.
If the CMB supports WDS and RDS, publish it for use as p2p memory
by other devices.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/pci.c | 75
-by: Logan Gunthorpe
---
drivers/nvme/host/core.c | 4
drivers/nvme/host/nvme.h | 1 +
drivers/nvme/host/pci.c | 19 +++
3 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 0fe7ea35c221..4dd564f175fa
Introduce a quirk to use CMB-like memory on older devices that have
an exposed BAR but do not advertise support for using CMBLOC and
CMBSIZE.
We'd like to use some of these older cards to test P2P memory.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/nvme.h | 7 +++
drivers
Attributes display the total amount of P2P memory, the amount available
and whether it is published or not.
Signed-off-by: Logan Gunthorpe
---
Documentation/ABI/testing/sysfs-bus-pci | 25 +
drivers/pci/p2pdma.c| 50 +
2 files
In order to use PCI P2P memory pci_p2pmem_[un]map_sg() functions must be
called to map the correct DMA address. To do this, we add a flags
variable and the RDMA_RW_CTX_FLAG_PCI_P2P flag. When the flag is
specified use the appropriate map function.
Signed-off-by: Logan Gunthorpe
---
drivers
The DMA address used when mapping PCI P2P memory must be the PCI bus
address. Thus, introduce pci_p2pmem_[un]map_sg() to map the correct
addresses when using P2P memory.
For this, we assume that an SGL passed to these functions contain all
p2p memory or no p2p memory.
Signed-off-by: Logan
effectively means that if CONFIG_PCI_P2PDMA is selected then
all devices behind any switch will be in the same IOMMU group.
Signed-off-by: Logan Gunthorpe
---
drivers/pci/Kconfig| 4
drivers/pci/p2pdma.c | 44
drivers/pci/pci.c | 4
adding new namespaces that are not supported
by that memory will fail.
Logan Gunthorpe (10):
PCI/P2PDMA: Support peer to peer memory
PCI/P2PDMA: Add sysfs group to display p2pmem stats
PCI/P2PDMA: Add PCI p2pmem dma mappings to adjust the bus offset
PCI/P2PDMA: Clear ACS P2P flags for
f-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/target/configfs.c | 29 +
drivers/nvme/target/core.c | 95 +-
drivers/nvme/target/io-cmd.c | 3 ++
drivers/nvme/target/nvmet.h| 10 +
drivers/nvme/target/
ar RC supports P2P transfers is non-trivial. Additionally, the
benefits of P2P transfers that go through the RC is limited to only
reducing DRAM usage.
This commit includes significant rework and feedback from Christoph
Hellwig.
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
driver
I_P2P
flag set.
Signed-off-by: Logan Gunthorpe
---
block/blk-core.c | 3 +++
include/linux/blk_types.h | 18 +-
include/linux/blkdev.h| 3 +++
3 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index 2d1a7bb
Hey Sagi,
Thanks for the review!
On 01/03/18 03:32 AM, Sagi Grimberg wrote:
int rdma_rw_ctx_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8
port_num,
struct scatterlist *sg, u32 sg_cnt, u32 sg_offset,
- u64 remote_addr, u32 rkey, enum dma_data_direction dir)
+ u64
On 01/03/18 04:03 AM, Sagi Grimberg wrote:
Can you describe what would be the plan to have it when these devices
do come along? I'd say that p2p_dev needs to become a nvmet_ns reference
and not from nvmet_ctrl. Then, when cmb capable devices come along, the
ns can prefer to use its own cmb inst
On 28/02/18 08:56 PM, Benjamin Herrenschmidt wrote:
On Thu, 2018-03-01 at 14:54 +1100, Benjamin Herrenschmidt wrote:
The problem is that acccording to him (I didn't double check the latest
patches) you effectively hotplug the PCIe memory into the system when
creating struct pages.
This cannot
Hi Bjorn,
Thanks for the review. I'll correct all the nits for the next version.
On 01/03/18 10:37 AM, Bjorn Helgaas wrote:
On Wed, Feb 28, 2018 at 04:39:57PM -0700, Logan Gunthorpe wrote:
Some PCI devices may have memory mapped in a BAR space that's
intended for use in Pe
Wouldn't it all be simpler if the p2p_dev resolution would be private
to the namespace?
So is adding some all the namespaces in a subsystem must comply to
using p2p? Seems a little bit harsh if its not absolutely needed. Would
be nice to export a subsystems between two ports (on two HCAs, acros
On 01/03/18 11:02 AM, Bjorn Helgaas wrote:
void pci_enable_acs(struct pci_dev *dev)
{
+ if (pci_p2pdma_disable_acs(dev))
+ return;
This doesn't read naturally to me. I do see that when
CONFIG_PCI_P2PDMA is not set, pci_p2pdma_disable_acs() does nothing
and returns 0,
On 01/03/18 11:42 AM, Jason Gunthorpe wrote:
On Thu, Mar 01, 2018 at 08:35:55PM +0200, Sagi Grimberg wrote:
This is also why I don't entirely understand why this series has a
generic allocator for p2p mem, it makes little sense to me.
Why wouldn't the nmve driver just claim the entire CMB of
On 01/03/18 12:21 PM, Dan Williams wrote:
Note: I think the above means it won't work behind a switch on x86
either, will it ?
The devm_memremap_pages() infrastructure allows placing the memmap in
"System-RAM" even if the hotplugged range is in PCI space. So, even if
it is an issue on some co
1 - 100 of 371 matches
Mail list logo
