Re: [PATCH V6 1/9] RDMA/iser: Limit sg tablesize and max_sectors to device fastreg max depth

2015-07-26 Thread Sagi Grimberg
On 7/24/2015 10:14 PM, Jason Gunthorpe wrote: On Fri, Jul 24, 2015 at 01:40:17PM -0500, Steve Wise wrote: Huh. How does this relate to the max_page_list_len argument: struct ib_mr *ib_alloc_fast_reg_mr(struct ib_pd *pd, int max_page_list_len) Shouldn't max_fast_reg_page_list_len be checked

Re: [PATCH V6 4/9] svcrdma: Use max_sge_rd for destination read depths

2015-07-26 Thread Christoph Hellwig
On Sun, Jul 26, 2015 at 12:58:59PM +0300, Sagi Grimberg wrote: With the above patch change, we have no more users of the recently created rdma_cap_read_multi_sge(). Should I add a patch to remove it? Yes please. And in the long run this is another argument for killing the system-wide

Re: [PATCH WIP 01/43] IB: Modify ib_create_mr API

2015-07-26 Thread Sagi Grimberg
On 7/23/2015 10:08 PM, Jason Gunthorpe wrote: On Thu, Jul 23, 2015 at 01:07:56PM +0300, Sagi Grimberg wrote: On 7/22/2015 10:05 PM, Jason Gunthorpe wrote: The reason I named max_entries is because might might not be pages but real SG elements. It stands for maximum registration entries. Do you

Re: [PATCH V6 4/9] svcrdma: Use max_sge_rd for destination read depths

2015-07-26 Thread Sagi Grimberg
@@ -1059,6 +1062,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) ntohs(((struct sockaddr_in *)newxprt-sc_cm_id- route.addr.dst_addr)-sin_port), newxprt-sc_max_sge, + newxprt-sc_max_sge_rd,

Re: [PATCH 00/10] IB: Replace safe uses for ib_get_dma_mr with pd-local_dma_lkey

2015-07-26 Thread Sagi Grimberg
If we want security by default then I propose not only to change the default value of register_always from false into true but also to change the default value of prefer_fr from false into true such that fast registration becomes the default instead of FMR. Yes, I was frowning at that stuff

Re: [PATCH WIP 28/43] IB/core: Introduce new fast registration API

2015-07-26 Thread Sagi Grimberg
On 7/23/2015 9:51 PM, Jason Gunthorpe wrote: On Thu, Jul 23, 2015 at 07:47:14PM +0300, Sagi Grimberg wrote: So we force ULPs to think about what they are doing properly, and we get a chance to actually force lkey to be local use only for IB. The lkey/rkey decision is passed in the fastreg

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Sagi Grimberg
On 7/24/2015 7:18 PM, Steve Wise wrote: This is in preparation for adding new FRMR-only IO handlers for devices that support FRMR and not PI. Steve, I've given this some thought and I think we should avoid splitting logic from PI and iWARP. The reason (other than code duplication) is that

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Christoph Hellwig
On Sun, Jul 26, 2015 at 01:08:16PM +0300, Sagi Grimberg wrote: I've given this some thought and I think we should avoid splitting logic from PI and iWARP. The reason (other than code duplication) is that currently the iser target support only up to 1MB IOs. I have some code (not done yet) to

Re: [PATCH] mlx5: Expose correct page_size_cap in device attributes

2015-07-26 Thread Sagi Grimberg
On 7/24/2015 12:48 AM, Jason Gunthorpe wrote: On Thu, Jul 23, 2015 at 05:41:38PM -0400, Doug Ledford wrote: I assume this prevents the driver from working at all on certain arches (like ppc with 64k page size)? Nothing uses page_size_cap correctly, so it has no impact. Sagi, that is a good

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Sagi Grimberg
On 7/26/2015 1:43 PM, Christoph Hellwig wrote: On Sun, Jul 26, 2015 at 01:08:16PM +0300, Sagi Grimberg wrote: I've given this some thought and I think we should avoid splitting logic from PI and iWARP. The reason (other than code duplication) is that currently the iser target support only up to

Re: [PATCH WIP 28/43] IB/core: Introduce new fast registration API

2015-07-26 Thread Sagi Grimberg
I would like to see the kdoc for ib_map_mr_sg explain exactly what is required of the caller, maybe just hoist this bit from the ib_sg_to_pages I'll add the kdoc. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More

Re: [PATCH WIP 28/43] IB/core: Introduce new fast registration API

2015-07-26 Thread Sagi Grimberg
On 7/23/2015 8:55 PM, Jason Gunthorpe wrote: On Thu, Jul 23, 2015 at 01:15:16PM +0300, Sagi Grimberg wrote: I was hoping we'd move the DMA flush and translate into here and make it mandatory. Is there any reason not to do that? The reason I didn't added it in was so the ULPs can make sure

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Sagi Grimberg
Ideally, the post contains a chain of all 4 registrations and the rdma_read (and an opportunistic good scsi response). Just to be clear: This example is for IB only, correct? IW would require rkeys with REMOTE_WRITE and 4 read wrs. My assumption is that it would depend on max_sge_rd. IB

[PATCH v2 12/13] IB/cma: Share ib_cm_ids between rdma_cm_ids

2015-07-26 Thread Haggai Eran
Use ib_cm_insert_listen to create listening IB CM IDs or share existing ones if needed. When given a request on a specific CM ID, the code now matches the request to the RDMA CM ID based on the request parameters, so it no longer needs to rely on the ib_cm's private data matching capabilities.

[PATCH v2 09/13] IB/cma: Add net_dev and private data checks to RDMA CM

2015-07-26 Thread Haggai Eran
Instead of relying on a the ib_cm module to check an incoming CM request's private data header, add these checks to the RDMA CM module. This allows a following patch to to clean up the ib_cm interface and remove the code that looks into the private headers. It will also allow supporting namespaces

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Steve Wise
On 7/26/2015 12:40 PM, Sagi Grimberg wrote: Ideally, the post contains a chain of all 4 registrations and the rdma_read (and an opportunistic good scsi response). Just to be clear: This example is for IB only, correct? IW would require rkeys with REMOTE_WRITE and 4 read wrs. My assumption

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Steve Wise
On 7/26/2015 5:08 AM, Sagi Grimberg wrote: On 7/24/2015 7:18 PM, Steve Wise wrote: This is in preparation for adding new FRMR-only IO handlers for devices that support FRMR and not PI. Steve, I've given this some thought and I think we should avoid splitting logic from PI and iWARP. The

[PATCH v2 13/13] IB/cm: Remove compare_data checks

2015-07-26 Thread Haggai Eran
Now that there are no ib_cm clients using the compare_data feature for matching IB CM requests' private data, remove the compare_data parameter of ib_cm_listen and remove the code implementing the feature. Signed-off-by: Haggai Eran hagg...@mellanox.com --- drivers/infiniband/core/cm.c

[PATCH v2 10/13] IB/cma: Validate routing of incoming requests

2015-07-26 Thread Haggai Eran
Pass incoming request parameters through the relevant IPv4/IPv6 routing tables and make sure the network stack is configured to handle such requests. Signed-off-by: Haggai Eran hagg...@mellanox.com --- drivers/infiniband/core/cma.c | 95 +-- 1 file

[PATCH v2 01/13] IB/core: lock client data with lists_rwsem

2015-07-26 Thread Haggai Eran
An ib_client callback that is called with the lists_rwsem locked only for read is protected from changes to the IB client lists, but not from ib_unregister_device() freeing its client data. This is because ib_unregister_device() will remove the device from the device list with lists_rwsem locked

[PATCH v2 06/13] IB/cma: Refactor RDMA IP CM private-data parsing code

2015-07-26 Thread Haggai Eran
When receiving a connection request, rdma_cm needs to associate the request with a network device, in order to disambiguate requests. To do this, it needs to know the request's destination IP. For this the module needs to allow getting this information from the private data in the request packet,

[PATCH v2 04/13] IB/cm: Expose service ID in request events

2015-07-26 Thread Haggai Eran
Expose the service ID on an incoming CM or SIDR request to the event handler. This will allow the RDMA CM module to de-multiplex connection requests based on the information encoded in the service ID. Acked-by: Sean Hefty sean.he...@intel.com Signed-off-by: Haggai Eran hagg...@mellanox.com ---

Re: [PATCH v3 06/15] xprtrdma: Clean up rpcrdma_ia_open()

2015-07-26 Thread Christoph Hellwig
Jason has patches that provide a local_dma_lkey in the PD that is always available. Do you need this clean up for the next merge window? If not it might be worth to postponed it to avoid merge conflicts, specially as I assume the NFS changes will go in through Trond. On Mon, Jul 20, 2015 at

Re: [PATCH V6 9/9] isert: Support iWARP transports using FRMRs

2015-07-26 Thread Sagi Grimberg
On 7/24/2015 10:24 PM, Jason Gunthorpe wrote: On Fri, Jul 24, 2015 at 01:48:09PM -0500, Steve Wise wrote: The use of FRWR for RDMA READ should be iWarp specific, IB shouldn't pay that overhead. I am expecting to see a cap_rdma_read_rkey or something in here ? Ok. But cap_rdma_read_rkey()

[PATCH v2 11/13] IB/cma: Use found net_dev for passive connections

2015-07-26 Thread Haggai Eran
When receiving a new connection in cma_req_handler, we actually already know the net_dev that is used for the connection's creation. Instead of calling cma_translate_addr to resolve the new connection id's source address, just use the net_dev that was found. Signed-off-by: Haggai Eran

[PATCH v2 03/13] IB/ipoib: Return IPoIB devices matching connection parameters

2015-07-26 Thread Haggai Eran
From: Guy Shapiro gu...@mellanox.com Implement the get_net_device_by_port_pkey_ip callback that returns network device to ib_core according to connection parameters. Check the ipoib device and iterate over all child devices to look for a match. For each IPoIB device we iterate through all upper

Re: [PATCH v3 04/15] xprtrdma: Don't fall back to PHYSICAL memory registration

2015-07-26 Thread Christoph Hellwig
On Mon, Jul 20, 2015 at 03:03:02PM -0400, Chuck Lever wrote: PHYSICAL memory registration uses a single rkey for all of the client's memory, thus is insecure. It is still useful in some cases for testing. Retain the ability to select PHYSICAL memory registration capability via

Re: [PATCH v3 01/15] xprtrdma: Make xprt_setup_rdma() agnostic to family of server address

2015-07-26 Thread Christoph Hellwig
On Mon, Jul 20, 2015 at 03:02:33PM -0400, Chuck Lever wrote: In particular, recognize when an IPv6 connection is bound. Signed-off-by: Chuck Lever chuck.le...@oracle.com Tested-by: Devesh Sharma devesh.sha...@avagotech.com Looks good, Reviewed-by: Christoph Hellwig h...@lst.de -- To

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Steve Wise
On 7/26/2015 6:00 AM, Sagi Grimberg wrote: On 7/26/2015 1:43 PM, Christoph Hellwig wrote: On Sun, Jul 26, 2015 at 01:08:16PM +0300, Sagi Grimberg wrote: I've given this some thought and I think we should avoid splitting logic from PI and iWARP. The reason (other than code duplication) is that

Re: [PATCH v3 06/15] xprtrdma: Clean up rpcrdma_ia_open()

2015-07-26 Thread Christoph Hellwig
On Sun, Jul 26, 2015 at 02:21:23PM -0400, Chuck Lever wrote: No, this patch is not strictly needed in 4.3, but my read of Jason?s series is that he does not touch xprtrdma. I don?t believe there will be a merge conflict. The goal of this patch is to move xprtrdma forward so it will be

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Christoph Hellwig
On Sun, Jul 26, 2015 at 02:00:51PM +0300, Sagi Grimberg wrote: On the wire iser sends a single rkey, but the target is allowed to transfer the data however it wants to. So you're trying to get above the limit of a single RDMA READ, not above the limit for memory registration in the initiator?

Re: [PATCH v3 06/15] xprtrdma: Clean up rpcrdma_ia_open()

2015-07-26 Thread Chuck Lever
Hi Christoph- On Jul 26, 2015, at 12:53 PM, Christoph Hellwig h...@infradead.org wrote: Jason has patches that provide a local_dma_lkey in the PD that is always available. Do you need this clean up for the next merge window? If not it might be worth to postponed it to avoid merge

[PATCH for-4.2] iw_cxgb4: gracefully handle unknown CQE status errors

2015-07-26 Thread Hariprasad Shenai
c4iw_poll_cq_on() shouldn't fail the poll operation just because the CQE status is unknown. Rather, it should map this to the fatal error status and log the anomaly. Signed-off-by: Steve Wise sw...@opengridcomputing.com Signed-off-by: Hariprasad Shenai haripra...@chelsio.com ---

[PATCH for-4.2] iw_cxgb4: set the default MPA version to 2

2015-07-26 Thread Hariprasad Shenai
This enables ORD/IRD negotiation and its about time to enable it by default Signed-off-by: Steve Wise sw...@opengridcomputing.com Signed-off-by: Hariprasad Shenai haripra...@chelsio.com --- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git

[PATCH v2 02/13] IB/core: Find the network device matching connection parameters

2015-07-26 Thread Haggai Eran
From: Yotam Kenneth yota...@mellanox.com In the case of IPoIB, and maybe in other cases, the network device is managed by an upper-layer protocol (ULP). In order to expose this network device to other users of the IB device, let ULPs implement a callback that returns network device according to

[PATCH v2 05/13] IB/cm: Share listening CM IDs

2015-07-26 Thread Haggai Eran
Enabling network namespaces for RDMA CM will allow processes on different namespaces to listen on the same port. In order to leave namespace support out of the CM layer, this requires that multiple RDMA CM IDs will be able to share a single CM ID. This patch adds infrastructure to retrieve an

[PATCH v2 08/13] IB/cm: Expose BTH P_Key in CM and SIDR request events

2015-07-26 Thread Haggai Eran
The rdma_cm module will later use the P_Key from the BTH to de-mux requests. See discussion at: http://www.spinics.net/lists/netdev/msg336067.html Cc: Jason Gunthorpe jguntho...@obsidianresearch.com Cc: Liran Liss lir...@mellanox.com Signed-off-by: Haggai Eran hagg...@mellanox.com ---

[PATCH v2 07/13] IB/cma: Helper functions to access port space IDRs

2015-07-26 Thread Haggai Eran
Add helper functions to access the IDRs by port-space and port number. Pass around the port-space enum in cma.c instead of using pointers to port-space IDRs. Signed-off-by: Haggai Eran hagg...@mellanox.com Signed-off-by: Yotam Kenneth yota...@mellanox.com Signed-off-by: Shachar Raindel

[PATCH v2 00/13] Demux IB CM requests in the rdma_cm module

2015-07-26 Thread Haggai Eran
Thanks everyone for the review comments. I've updated the patch set accordingly. The changes are listed below. In addition to the changes discussed on the list I've made sure AF_IB continues to work by retrieving parameters from the listener ID when an AF_IB request is detected. Changes from v1:

Re: [PATCH V6 6/9] isert: Rename IO functions to more descriptive names

2015-07-26 Thread Sagi Grimberg
On 7/26/2015 6:53 PM, Christoph Hellwig wrote: On Sun, Jul 26, 2015 at 02:00:51PM +0300, Sagi Grimberg wrote: On the wire iser sends a single rkey, but the target is allowed to transfer the data however it wants to. So you're trying to get above the limit of a single RDMA READ, not above the