On Fri, Jul 31, 2015 at 08:50:24AM -0400, Doug Ledford wrote:
So... are we ready to go with V7 upstream?
Yes. I've already pulled it in.
You are taking the netdev stuff without an ack from netdev??
I've been too busy too look at v7, but a quick check of the 'move the
cache into core code
On Fri, Jul 31, 2015 at 03:32:07PM +, Marciniszyn, Mike wrote:
+HFI1
+
+ The hfi1 driver also creates these additional files:
+
+ hw_rev - hardware revision
=20
I'm checking on this to see if it is indeed a duplicate.
=20
Our hardware architect
On Fri, Jul 31, 2015 at 01:41:39PM -0400, Doug Ledford wrote:
Please be more specific here. What are you objecting to? Are you
objecting to a flush_workqueue from a release() context? Because I
don't see anything in the kref documentation or the kref
implementation that prevents or
On Sat, Aug 01, 2015 at 12:24:23AM +0300, Or Gerlitz wrote:
addressed in incremental patch, as Doug suggested. Jason, it's wrong
to send developers again and again to fix things which were
perfect in Vn-1 but also not being covered by reviewers on Vn-1, at
some point the reviewer can't load
On Fri, Jul 31, 2015 at 03:20:40PM -0700, Bart Van Assche wrote:
On 07/30/2015 04:22 PM, Jason Gunthorpe wrote:
All patches are compile tested. I've done basic testing up to and including
the IPoIB patch, the rest required specialized setups I don't have access to,
but are fairly
On Fri, Jul 31, 2015 at 04:04:35PM -0700, Bart Van Assche wrote:
With only patches 1/12 and 8/12 applied my test passes.
Thanks Bart!
If you like, I can try and work on #9 (no promises :(), but I think
I'll need some way to test it here.
Do you by chance have a straightforward recipe to
On Thu, Jul 30, 2015 at 11:46:36PM +0300, Yuval Shaia wrote:
On Thu, Jul 30, 2015 at 11:15:38AM -0600, Jason Gunthorpe wrote:
On Thu, Jul 30, 2015 at 11:51:12AM -0400, Doug Ledford wrote:
In its current state, I have my doubts about this patch. However, it
seems to me that this should
On Thu, Jul 30, 2015 at 03:18:59PM -0400, Mike Marciniszyn wrote:
+static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from)
+{
+ struct hfi1_user_sdma_pkt_q *pq;
+ struct hfi1_user_sdma_comp_q *cq;
+ int ret = 0, done = 0, reqs = 0;
+ unsigned long dim =
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
net/rds/ib.c | 8
net/rds/ib.h | 2 --
net/rds/ib_cm.c | 4 +---
net/rds/ib_recv.c | 6 +++---
net/rds/ib_send.c
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
Tested-by: Dominique Martinet dominique.marti...@cea.fr
---
net/9p/trans_rdma.c | 26 ++
1 file changed, 2
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
Reviewed-by: Sagi Grimberg sa...@mellanox.com
---
drivers/infiniband/ulp/srpt/ib_srpt.c | 15 ---
drivers/infiniband/ulp/srpt
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/core/mad.c | 26 +++---
drivers/infiniband/core/mad_priv.h | 1 -
include/rdma/ib_mad.h
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/ulp/ipoib/ipoib.h | 1 -
drivers/infiniband/ulp/ipoib/ipoib_cm.c| 2 +-
drivers/infiniband/ulp/ipoib
does not create an insecure all physical MR.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/ulp/srp/ib_srp.c | 31 +--
drivers/infiniband/ulp/srp/ib_srp.h | 2 +-
2 files changed, 22 insertions(+), 11 deletions(-)
diff --git
I don't have access to,
but are fairly straightforward.
Jason Gunthorpe (12):
IB/core: Guarantee that a local_dma_lkey is available
IB/mad: Remove ib_get_dma_mr calls
IB/ipoib: Remove ib_get_dma_mr calls
IB/mlx4: Remove ib_get_dma_mr calls
IB/mlx5: Remove ib_get_dma_mr calls
IB/iser
Replace all leys with pd-local_dma_lkey. This driver does not support
iWarp, so this is safe.
The insecure use of ib_get_dma_mr is thus isolated to an rkey, and this
looks trivially fixed by forcing the use of registration in a future
patch.
Signed-off-by: Jason Gunthorpe jguntho
Replace all leys with pd-local_dma_lkey. This driver does not support
iWarp, so this is safe.
The insecure use of ib_get_dma_mr is thus isolated to an rkey, and will
have to be fixed separately.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
Reviewed-by: Sagi Grimberg sa
On Thu, Jul 30, 2015 at 10:28:42PM +, Marciniszyn, Mike wrote:
+ board_id - manufacturing board id
There is no PCIe equivalent.
+ serial - board serial number
No PCIe equivalent.
+ boardversion - board version
These all have PCI-E versions. Most should live in the
On Thu, Jul 30, 2015 at 12:46:52PM -0400, Doug Ledford wrote:
I've pulled this series in for 4.3. There were some additional items in
some of Jason's comments that ought to be looked into, but I think this
patch set has reached the point where it's no worse than existing in
terms of locking,
On Thu, Jul 30, 2015 at 11:51:12AM -0400, Doug Ledford wrote:
In its current state, I have my doubts about this patch. However, it
seems to me that this should be relatively easy to fix in such a way
that you get 90%+ of the performance benefit, and can turn it on by
default, and we don't
On Thu, Jul 30, 2015 at 10:13:09AM +0300, Sagi Grimberg wrote:
Basically, swiotlb realigns everything that passes through it.
So this won't ever happen if the ULP will DMA map the SG and check
for gaps right?
Once mapped the physical address isn't going to change - but at some
point we must
On Thu, Jul 30, 2015 at 12:59:30PM -0400, Doug Ledford wrote:
On 07/30/2015 12:50 PM, Jason Gunthorpe wrote:
On Thu, Jul 30, 2015 at 12:46:52PM -0400, Doug Ledford wrote:
I've pulled this series in for 4.3. There were some additional items in
some of Jason's comments that ought
On Thu, Jul 30, 2015 at 10:47:00PM +, Marciniszyn, Mike wrote:
I'm checking to make sure.
Was this an lspci -vvv output for the example you showed?
Yes.
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More
receives a memory
region type (whcih can be IB_MR_TYPE_MEM_REG for normal memory
registration, IB_MR_TYPE_SIGNATURE for a data-integrity capable
memory region and future arbitrary SG support capable memory
region).
Looks good to me now..
Reviewed-by: Jason Gunthorpe jguntho
On Thu, Jul 30, 2015 at 10:05:53AM -0500, Steve Wise wrote:
+int iser_assign_reg_ops(struct iser_device *device)
+{
+struct ib_device_attr *dev_attr = device-dev_attr;
+
+/* Assign function handles - based on FMR support */
+if (device-ib_device-alloc_fmr
On Thu, Jul 30, 2015 at 11:06:25AM +0300, Sagi Grimberg wrote:
/**
+ * struct iser_device - Memory registration operations
+ * per-device registration schemes
+ *
+ * @alloc_reg_res: Allocate registration resources
+ * @free_reg_res: Free registration resources
+ *
On Wed, Jul 29, 2015 at 04:47:59PM -0400, Chuck Lever wrote:
Apparently this is true for some providers, and not for others, and
I misunderstood that when I put this together last year.
Really? In kernel providers? Interesting, those are probably wrong...
The idea that you can completely
On Tue, Jul 28, 2015 at 04:58:29PM -0400, J.L. Burr wrote:
Is there some way now (in upstream kernels) to create a MR with an
arbitrary (and large) physical address range? That would be great! I
didn't see a way to do that when I started on this journey (about 4
years ago).
The FRWR API
On Sun, Jul 26, 2015 at 01:23:12PM +0300, Sagi Grimberg wrote:
Question though, wouldn't it be better to do a single RDMA_READ to say
4 registered keys rather than RDMA_READ_WITH_INV for each?
RDMA READ is limted to 1 sg in iWarp.
RDMA_READ_WITH_INV and 1 sg is really the only correct way to
On Mon, Jul 27, 2015 at 11:57:46AM -0400, Chuck Lever wrote:
IMO ib_unmap_fmr is a very different animal from LOCAL_INV WR.
Sure, but how many of these properties does NFS actually care about,
now that it is running the API properly?
ib_unmap_fmr is synchronous, provides no ordering guarantees
GID.
Furthermore, such configuration may be desireable to support ipvlan-like
configurations for RDMA CM with IPoIB. To resolve the device in these
cases the code will also take the IP address as an additional input.
I already sent this, but again:
Reviewed-by: Jason Gunthorpe jguntho
pass the client data directly to
remove() callbacks.
This looks fine to me..
Reviewed-By: Jason Gunthorpe jguntho...@obsidianresearch.com
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at http
On Sun, Jul 26, 2015 at 12:37:55PM +0300, Sagi Grimberg wrote:
I guess I can pull DMA mapping in there, but we will need an opposite
routine ib_umap_mr_sg() since it'll be weird if the ULP will do dma
unmap without doing the map...
Yes, and ideally it would help ULPs to order these operations
On Sun, Jul 26, 2015 at 12:45:10PM +0300, Sagi Grimberg wrote:
On 7/23/2015 9:51 PM, Jason Gunthorpe wrote:
On Thu, Jul 23, 2015 at 07:47:14PM +0300, Sagi Grimberg wrote:
So we force ULPs to think about what they are doing properly, and we
get a chance to actually force lkey to be local use
On Mon, Jul 27, 2015 at 03:11:04PM -0500, Steve Wise wrote:
Well technically an MR with REMOTE_WRITE also has LOCAL_WRITE set. So you
are proposing the core disallow a ULP from using the lkey for this type of
MR? Say in a RECV sge?
Yes, absolutely.
It is wrong anyhow, RECV isn't special, if
On Mon, Jul 27, 2015 at 06:09:55PM -0500, Steve Wise wrote:
Resending because I forgot to cc linux-rdma :(
All looks sane to me:
Reviewed-by: Jason Gunthorpe jguntho...@obsidianresearch.com
Thanks,
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body
On Fri, Jul 24, 2015 at 10:36:07AM -0400, Chuck Lever wrote:
Unfinished, but operational:
http://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;h=refs/heads/nfs-rdma-future
Nice..
Can you spend some time and reflect on how some of this could be
lowered into the core code? The FMR and FRWR
On Fri, Jul 24, 2015 at 03:59:06PM -0400, Chuck Lever wrote:
And RPC-over-RDMA version 1 does not have any way to signal that
the server has invalidated the MRs. Such signaling would be a
pre-requisite to allow the Linux NFS/RDMA client to interoperate
with non-Linux NFS/RDMA servers that do
On Fri, Jul 24, 2015 at 04:26:00PM -0400, Chuck Lever wrote:
Basically RPC work flow stopped because an RPC reply never
arrived.
Oh, that is what I expect to see.. Remebmer the cq upcall is edge
triggered, so if you leave stuff in the cq then you don't get another
upcall until another CQE is
On Fri, Jul 24, 2015 at 05:11:04PM -0500, Steve Wise wrote:
By the way, just to be clear: If you use a FRWR, you by definition
only have one SGE entry as the result of the registration. So
regardless of what a device/protocol can do with the destination SGE
of an RDMA READ operation, if you
I am unclear what happens sever side if the server starts issuing
SEND_WITH_INVALIDATE to a client that doesn't expect it. The net
result is a MR would be invalidated twice. I don't know if this is OK
or not.
It is ok to invalidate an already-invalid MR.
Nice, ah but I forgot about the
On Fri, Jul 24, 2015 at 01:46:05PM -0400, Chuck Lever wrote:
I'm not surprised since invalidate is sync. I belive you need to
incorporate SEND WITH INVALIDATE to substantially recover this
overhead.
I tried to find another kernel ULP using SEND WITH INVALIDATE, but
I didn’t see one. I
On Fri, Jul 24, 2015 at 01:40:17PM -0500, Steve Wise wrote:
Huh. How does this relate to the max_page_list_len argument:
struct ib_mr *ib_alloc_fast_reg_mr(struct ib_pd *pd, int max_page_list_len)
Shouldn't max_fast_reg_page_list_len be checked during the above?
Ie does this
On Fri, Jul 24, 2015 at 11:18:21AM -0500, Steve Wise wrote:
Currently the sg tablesize, which dictates fast register page list
depth to use, does not take into account the limits of the rdma device.
So adjust it once we discover the device fastreg max depth limit. Also
adjust the max_sectors
On Fri, Jul 24, 2015 at 11:19:05AM -0500, Steve Wise wrote:
Add new register and unregister functions to be used with devices that
support FRMRs, provide a local dma lkey, yet do not support DIF/PI.
isert_reg_frmr() only needs to use FRMRs for RDMA READ since RDMA WRITE
can be handled
On Thu, Jul 23, 2015 at 05:41:38PM -0400, Doug Ledford wrote:
I assume this prevents the driver from working at all on certain arches
(like ppc with 64k page size)?
Nothing uses page_size_cap correctly, so it has no impact.
Sagi, that is a good point, your generic code for the cleanup series
On Thu, Jul 23, 2015 at 11:30:08PM +, Hefty, Sean wrote:
I don't see where usnic, ipath, qib, or opa technically need an lkey
at all.
The lkey is possibly useful if someone wants to do single op transfers
larger than the S/G limit of the SQE. I haven't noticed any ULPs doing
that..
qib
On Thu, Jul 23, 2015 at 07:26:32PM +, Wan, Kaike wrote:
The nl sequence increased sequentially. What's the difference?
Hmm, looks like in my version the DGID was getting mangled and this
causes ipoib to enter a tight query loop..
Not your problem.
Actually, looks like nothing removes nl
On Thu, Jul 23, 2015 at 02:25:32AM -0700, Christoph Hellwig wrote:
On Wed, Jul 22, 2015 at 11:30:48AM -0600, Jason Gunthorpe wrote:
On Wed, Jul 22, 2015 at 09:55:40AM +0300, Sagi Grimberg wrote:
+ size += max_t(int, MLX5_UMR_ALIGN - ARCH_KMALLOC_MINALIGN, 0);
+ mr-klms = kzalloc(size
On Thu, Jul 23, 2015 at 01:19:16PM +0300, Sagi Grimberg wrote:
Again, related to my prior comments, please have two of these:
ib_map_mr_sg_rkey()
ib_map_mr_sg_lkey()
So we force ULPs to think about what they are doing properly, and we
get a chance to actually force lkey to be local use
On Thu, Jul 23, 2015 at 02:19:55AM -0700, Christoph Hellwig wrote:
Although I wonder if we really need to differentiate between rkey and
leky in this ib_map_mr_sg function, or if we should do it when
allocating the mr, i.e. in ib_alloc_mr.
The allocation is agnostic to the usage, the map is
On Tue, Jul 21, 2015 at 11:42:29AM +0300, Matan Barak wrote:
On Fri, Jul 17, 2015 at 10:02 PM, Jason Gunthorpe
jguntho...@obsidianresearch.com wrote:
On Wed, Jun 24, 2015 at 03:59:21PM +0300, Matan Barak wrote:
+
+ /* in rdma_cap_roce_gid_table, this funciton should be protected
On Thu, Jul 23, 2015 at 01:07:56PM +0300, Sagi Grimberg wrote:
On 7/22/2015 10:05 PM, Jason Gunthorpe wrote:
The reason I named max_entries is because might might not be pages but
real SG elements. It stands for maximum registration entries.
Do you have a better name?
I wouldn't try
On Thu, Jul 23, 2015 at 06:47:26AM -0700, Bart Van Assche wrote:
On 07/22/15 16:34, Jason Gunthorpe wrote:
The remaining users of ib_get_dma_mr are all unsafe:
[ ... ]
drivers/infiniband/ulp/srp/ib_srp.c:
srp_dev-mr = ib_get_dma_mr(srp_dev-pd
On Thu, Jul 23, 2015 at 01:47:02PM +0300, Sagi Grimberg wrote:
+/* Return a pd for in-kernel use that has a local_dma_lkey which provides
+ local access to all physical memory. */
Why not kdoc style? we need to move the ib_verbs.h kdocs here anyway.
Might be a good chance to do that for
On Thu, Jul 23, 2015 at 01:15:16PM +0300, Sagi Grimberg wrote:
On 7/22/2015 8:44 PM, Jason Gunthorpe wrote:
On Wed, Jul 22, 2015 at 09:50:12AM -0700, Christoph Hellwig wrote:
+/**
+ * ib_map_mr_sg() - Populates MR with a dma mapped SG list
+ * @mr:memory region
+ * @sg
On Thu, Jul 23, 2015 at 11:42:11AM -0700, Bart Van Assche wrote:
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c
b/drivers/infiniband/ulp/srp/ib_srp.c
index fb9fed0fac28..a1e3818d0791 100644
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -69,7 +69,7 @@ static unsigned int cmd_sg_entries;
On Thu, Jul 23, 2015 at 07:47:14PM +0300, Sagi Grimberg wrote:
So we force ULPs to think about what they are doing properly, and we
get a chance to actually force lkey to be local use only for IB.
The lkey/rkey decision is passed in the fastreg post_send().
That is too late to check the
On Thu, Jul 23, 2015 at 07:59:48PM +0300, Sagi Grimberg wrote:
I don't mean to be negative about your ideas, I just don't think that
doing all the work in the drivers is going to get us to a better place.
No worries, I'm hoping someone can put the peices together and figure
out how to code
On Thu, Jul 23, 2015 at 01:15:16PM +0300, Sagi Grimberg wrote:
I was hoping we'd move the DMA flush and translate into here and make
it mandatory. Is there any reason not to do that?
The reason I didn't added it in was so the ULPs can make sure they meet
the restrictions of ib_map_mr_sg().
On Thu, Jul 23, 2015 at 01:27:23PM +0300, Sagi Grimberg wrote:
ib_post_fastreg_wr would be a function that needs 3 register passed
arguments and does a simple copy to the driver's actual sendq
That will require to take the SQ lock and write a doorbell for each
registration and post you want
+/**
+ * ib_alloc_mr() - Allocates a memory region
+ * @pd:protection domain associated with the region
+ * @mr_type: memory region type
+ * @max_entries: maximum registration entries available
+ * @flags: create flags
+ */
Can you update this comment to
On Wed, Jul 22, 2015 at 07:59:16PM +0300, Sagi Grimberg wrote:
Do we want to pull ib_get_dma_mr() here with type IB_MR_TYPE_DMA?
I want to get rid of ib_get_dma_mr...
My plan was to get rid of it as my last series shows for all lkey
usages and then rename it to:
On Wed, Jul 22, 2015 at 10:10:23AM -0700, Christoph Hellwig wrote:
The one thing I'm curious about is how we can support SRP with it's
multiple MR support without too much boilerplate code. One option
would be that pass an array of MRs to the map routines, and while
most callers would just
On Wed, Jul 22, 2015 at 09:55:40AM +0300, Sagi Grimberg wrote:
+ size += max_t(int, MLX5_UMR_ALIGN - ARCH_KMALLOC_MINALIGN, 0);
+ mr-klms = kzalloc(size, GFP_KERNEL);
+ if (!mr-klms)
+ return -ENOMEM;
+
+ mr-pl_map = dma_map_single(device-dma_device, mr-klms,
+
On Wed, Jul 22, 2015 at 01:50:01PM -0500, Steve Wise wrote:
43 patches overflows my stack ;) I agree with Jason's suggestion.
Saig, you may as well just send the ib_alloc_mr rework as a series and
get it done with, I'd pass off on the core parts of v2.
Jason
--
To unsubscribe from this list:
On Wed, Jul 22, 2015 at 07:58:23PM +0300, Sagi Grimberg wrote:
On 7/22/2015 7:44 PM, Christoph Hellwig wrote:
On Wed, Jul 22, 2015 at 10:34:05AM -0600, Jason Gunthorpe wrote:
+/**
+ * ib_alloc_mr() - Allocates a memory region
+ * @pd:protection domain associated with the region
/ipoib: Use dedicated workqueues per interface)
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_verbs.c
b/drivers/infiniband/ulp/ipoib
On Wed, Jul 22, 2015 at 08:33:16PM +0300, Sagi Grimberg wrote:
memset(fr_wr, 0, sizeof(fr_wr));
+ ib_set_fastreg_wr(mr, mr-lkey, ISER_FASTREG_LI_WRID,
+ false, fr_wr);
Shouldn't ib_set_fastreg_wr take care of this memset? Also it seems
instead of the singalled
On Thu, Jul 09, 2015 at 01:34:25PM -0400, kaike@intel.com wrote:
+ LS_NLA_TYPE_STATUS = 0,
This is never used, drop it.
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
the body of a message to majord...@vger.kernel.org
More majordomo info at
-by: Jason Gunthorpe jguntho...@obsidianresearch.com
Reviewed-by: Sagi Grimberg sa...@dev.mellanox.co.il
Acked-by: Christoph Hellwig h...@infradead.org
---
drivers/infiniband/core/verbs.c | 40 +++-
include/rdma/ib_verbs.h | 2 ++
2 files changed, 37
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/hw/mlx4/mad.c | 23 ---
drivers/infiniband/hw/mlx4/mlx4_ib.h | 1 -
2 files changed, 4
Replace all leys with pd-local_dma_lkey. This driver does not support
iWarp, so this is safe.
The insecure use of ib_get_dma_mr is thus isolated to an rkey, and this
looks trivially fixed by forcing the use of registration in a future
patch.
Signed-off-by: Jason Gunthorpe jguntho
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/ulp/ipoib/ipoib.h | 1 -
drivers/infiniband/ulp/ipoib/ipoib_cm.c| 2 +-
drivers/infiniband/ulp/ipoib
Replace all leys with pd-local_dma_lkey. This driver does not support
iWarp, so this is safe.
The insecure use of ib_get_dma_mr is thus isolated to an rkey, and will
have to be fixed separately.
Signed-off-by: Jason Gunthorpe jguntho...@obsidianresearch.com
---
drivers/infiniband/ulp/srp
On Tue, Jul 21, 2015 at 02:40:12PM +0300, Sagi Grimberg wrote:
Should be all the page sizes that are supported by the
device.
Reported-by: Jason Gunthorpe jguntho...@obsidianresearch.com
Signed-off-by: Sagi Grimberg sa...@mellanox.com
drivers/infiniband/hw/mlx5/main.c |3 ++-
1 files
On Tue, Jul 21, 2015 at 11:14:59AM -0400, Jerome Glisse wrote:
This is a rebase error. But the #error is there for a purpose the HMM
would not work mid way so if anyone if bisecting up through that i would
rather error out at compilation.
Add the kconfig option last not first?
Also, can you
On Tue, Jul 21, 2015 at 02:40:18PM +0300, Sagi Grimberg wrote:
I'd imagine that the ULP knows when it registers huge-pages.
OK, I can scan the scatterlist and check it.
At some point in the stack.. Not necessarily in the IB ULP though..
It is something that can be reviewed when the API is
On Sun, Jul 19, 2015 at 08:33:24AM +0300, Sagi Grimberg wrote:
I was thinking that the user won't explicitly say which key it registers
and it will be decided from the registration itself.
Meaning, the registration code will do:
Please don't..
if (access | (IB_ACCESS_REMOTE_READ |
On Mon, Jul 20, 2015 at 07:27:52PM +0300, Sagi Grimberg wrote:
I'm thinking now that this should have an input argument
of block_size. Maybe in the future ULPs would want to register
huge pages, it will be a shame to map it into PAGE_SIZE chunks...
Why wouldn't it just transparently support
On Sun, Jul 19, 2015 at 08:45:26AM +0300, Sagi Grimberg wrote:
/**
* ib_mr_set_sg() - populate memory region buffers
* array from a SG list
* @mr: memory region
* @sg: sg list
* @sg_nents:number of elements in the sg
*
* Can fail if the HW
On Mon, Jul 20, 2015 at 01:34:16PM -0700, Tom Talpey wrote:
On 7/20/2015 12:03 PM, Chuck Lever wrote:
All HCA providers have an ib_get_dma_mr() verb. Thus
rpcrdma_ia_open() will either grab the device's local_dma_key if one
is available, or it will call ib_get_dma_mr() which is a 100%
On Mon, Jul 20, 2015 at 04:37:15PM -0500, Steve Wise wrote:
From: Steve Wise [mailto:sw...@opengridcomputing.com]
Sent: Monday, July 20, 2015 4:34 PM
To: 'Jason Gunthorpe'; 'Tom Talpey'
Cc: 'Chuck Lever'; 'linux-rdma@vger.kernel.org'; 'linux-...@vger.kernel.org'
Subject: RE: [PATCH
On Mon, Jul 20, 2015 at 03:04:21PM -0700, Tom Talpey wrote:
B) why bother to check? Are machines with 4GB interesting, and worth
supporting a special optimization?
mainline drivers shouldn't silently malfunction.
Jason
--
To unsubscribe from this list: send the line unsubscribe linux-rdma in
On Mon, Jul 20, 2015 at 08:07:00PM +0300, Sagi Grimberg wrote:
On 7/20/2015 8:00 PM, Jason Gunthorpe wrote:
On Mon, Jul 20, 2015 at 07:27:52PM +0300, Sagi Grimberg wrote:
I'm thinking now that this should have an input argument
of block_size. Maybe in the future ULPs would want to register
On Mon, Jul 20, 2015 at 08:09:57PM +0300, Sagi Grimberg wrote:
On 7/20/2015 8:08 PM, Chuck Lever wrote:
On Jul 20, 2015, at 12:54 PM, Sagi Grimberg sa...@mellanox.com wrote:
The mlx5 driver exposes device capability IB_DEVICE_LOCAL_DMA_LKEY
but does not set the the device local_dma_lkey.
On Mon, Jul 20, 2015 at 05:43:34PM -0500, Steve Wise wrote:
Yah, that seems much better.. With the patch set I am working on this
will mean all ULPs will fail to create a kernel PD on cxgb3 if the
above triggers. If our error handling works that should just make it
unuable from kernel
On Mon, Jul 20, 2015 at 03:03:11PM -0400, Chuck Lever wrote:
+ iov-length = size;
+ iov-lkey = ia-ri_have_dma_lkey ?
+ ia-ri_dma_lkey : ia-ri_bind_mem-lkey;
+ rb-rg_size = size;
+ rb-rg_owner = NULL;
return rb;
There is something odd looking
On Mon, Jul 20, 2015 at 06:31:11PM -0400, Chuck Lever wrote:
On Jul 20, 2015, at 6:26 PM, Jason Gunthorpe
jguntho...@obsidianresearch.com wrote:
On Mon, Jul 20, 2015 at 03:03:11PM -0400, Chuck Lever wrote:
+ iov-length = size;
+ iov-lkey = ia-ri_have_dma_lkey
On Mon, Jul 20, 2015 at 05:41:27PM -0500, Steve Wise wrote:
B) why bother to check? Are machines with 4GB interesting, and worth
supporting a special optimization?
No, but cxgb3 is still interesting to user applications, and perhaps NFSRDMA
using FRMRs.
Doesn't look like the NFS client
On Fri, Jul 17, 2015 at 11:03:45AM -0400, Chuck Lever wrote:
On Jul 16, 2015, at 4:49 PM, Jason Gunthorpe
jguntho...@obsidianresearch.com wrote:
On Thu, Jul 16, 2015 at 04:07:04PM -0400, Chuck Lever wrote:
The MRs are registered only for remote read. I don’t think
catastrophic harm
On Fri, Jul 17, 2015 at 03:26:04PM -0400, Chuck Lever wrote:
I'd say the above is broadly typical for what I'd consider correct use
of a RDMA QP.. The three flow control loops of #0 should be fairly obvious
and explicit in the code.
Jason, thanks for your comments and your time.
No
On Thu, Jul 16, 2015 at 10:45:46AM -0400, Chuck Lever wrote:
For some time, I’ve been considering deferring ib_unmap_fmr() to
a work queue, but FMR is operational and is a bit of an antique
so I haven’t put much effort into bettering it.
Okay, I think I get it..
The fmr unmap could be made
On Wed, Jul 15, 2015 at 01:12:57PM -0600, Jason Gunthorpe wrote:
I might find time to type this in, but I won't be able to find time to
do any testing on the ULPs..
Here is the typing, I'll look more carefully at it later and send it
via email:
https://github.com/jgunthorpe/linux/commits
On Thu, Jul 16, 2015 at 01:04:02AM -0700, 'Christoph Hellwig' wrote:
On Wed, Jul 15, 2015 at 01:12:57PM -0600, Jason Gunthorpe wrote:
This looks perfect to me. After this we can get rid of the
ib_get_dma_mr calls outside of ib_alloc_pd, and eventuall move
setting up -local_dma_lkey
On Thu, Jul 16, 2015 at 12:01:55PM +, Liran Liss wrote:
- Name space lookup is done based on BTH.pkey, private_data.IP, and
optionally GRH.DGID (if present, for extra validation)
Just changing the pkey to BTH.pkey would be fine by me.
Using GRH.DGID if available instead of the primary
On Thu, Jul 16, 2015 at 03:21:04PM +0300, Sagi Grimberg wrote:
I gotta say,
these suggestions of bool/write or supported_ops with a convert helper
seem (to me at least) to make things more complicated.
Why not just set the the access_flags as they are?
I want local use?
set
On Thu, Jul 16, 2015 at 04:07:04PM -0400, Chuck Lever wrote:
The MRs are registered only for remote read. I don’t think
catastrophic harm can occur on the client in this case if the
invalidation and DMA sync comes late. In fact, I’m unsure why
a DMA sync is even necessary as the MR is
On Wed, Jul 15, 2015 at 01:57:48PM +0300, Haggai Eran wrote:
On 13/07/2015 21:14, Jason Gunthorpe wrote:
On Mon, Jun 22, 2015 at 03:42:37PM +0300, Haggai Eran wrote:
+ switch (ib_event-event) {
+ case IB_CM_REQ_RECEIVED:
+ req-device = req_param-listen_id-device
On Wed, Jul 15, 2015 at 10:32:55AM -0400, Chuck Lever wrote:
I would rather not build a non-deterministic delay into the
unmap interface. Using a pool or having map do an implicit
unmap are both solutions I’d rather avoid.
Can you explain how NFS is using FMR today? When does it unmap a FMR
201 - 300 of 1197 matches
Mail list logo
