Re: [V3 PATCH 3/5] device property: Introduces device_dma_is_coherent()

On 5/8/2015 1:49 PM, Rafael J. Wysocki wrote: On Thursday, May 07, 2015 09:12:00 PM santosh.shilim...@oracle.com wrote: On 5/7/15 5:37 PM, Suravee Suthikulpanit wrote: Currently, device drivers, which support both OF and ACPI, need to call two separate APIs, of_dma_is_coherent() and

[PATCH 13/14] RDS: return EMSGSIZE for oversize requests before processing/queueing

those limits do not get to rds_send_queue_rm() code for processing. Signed-off-by: Mukesh Kacker mukesh.kac...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/send.c | 11 ++- 1 file changed, 6

[PATCH 14/14] RDS: check for valid cm_id before initiating connection

Connection could have been dropped while the route is being resolved so check for valid cm_id before initiating the connection. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim

[PATCH 00/14] RDS: Assorted bug fixes

/queueing Santosh Shilimkar (13): RDS: restore return value in rds_cmsg_rdma_args() RDS: always free recv frag as we free its ring entry RDS: destroy the ib state earlier during shutdown RDS: don't update ip address tables if the address hasn't changed RDS: make sure we post recv buffers

[PATCH 09/14] RDS: Make sure we do a signaled send for large-send

at least once per large-send, we can at least detect the problem in work completion handler there by avoiding sending more data to inactive remote. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh

[PATCH 01/14] RDS: restore return value in rds_cmsg_rdma_args()

the 'ret = 0' line which broke RDS RDMA mode. Fix it by restoring the return value on rds_pin_pages() success keeping the clean-up in place. Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/rdma.c | 2 ++ 1 file changed, 2

[PATCH 06/14] RDS: check for congestion updates during rds_send_xmit

Ensure we don't keep sending the data if the link is congested. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/send.c | 3 ++- 1 file changed, 2

[PATCH 04/14] RDS: don't update ip address tables if the address hasn't changed

If the ip address tables hasn't changed, there is no need to remove them only to be added back again. Lets fix it. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com

[PATCH 02/14] RDS: always free recv frag as we free its ring entry

and connection resets. This patch ensures that we free the frag as we mark the ring entry free. This should stop the refill path from finding allocated frags in ring entries that were marked free. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant

[PATCH 08/14] RDS: Mark message mapped before transmit

rds_send_xmit() marks the rds message map flag after xmit_[rdma/atomic]() which is clearly wrong. We need to maintain the ownership between transport and rds. Also take care of error path. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant

[PATCH 07/14] RDS: add a sock_destruct callback debug aid

This helps to detect the accidental processes/apps trying to destroy the RDS socket which they are sharing with other processes/apps. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar

[PATCH 10/14] RDS: Fix assertion level from fatal to warning

Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/ib_rdma.c | 2 +- net/rds/rdma.c| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/net/rds/ib_rdma.c b/net/rds/ib_rdma.c index e49c956..7b7aac8 100644 --- a/net/rds

[PATCH 11/14] RDS: Don't destroy the rdma id until after we're done using it

From: Santosh Shilimkar ssant...@kernel.org During connection resets, we are destroying the rdma id too soon. We can't destroy it when it is still in use. So lets move rdma_destroy_id() after we clear the rings. Reviewed-by: Ajaykumar Hotchandani ajaykumar.hotchand...@oracle.com Signed-off

[net-next PATCH] RDS: convert bind hash table to re-sizable hashtable

the flexibility to grow the hashtbable based on use and also comes up with inbuilt efficient bucket(chain) handling. Reviewed-by: David Miller <da...@davemloft.net> Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.co

Re: [PATCH net] RDS-TCP: Recover correctly from pskb_pull()/pksb_trim() failure in rds_tcp_data_recv

od one. Probably we should get this fix in stable versions as well. It seems to be applicable for all v2.6.32+ stable versions. FWIW, Acked-by: Santosh Shilimkar <santosh.shilim...@oracle.com> Regards, Santosh -- To unsubscribe from this list: send the line "unsubscribe netdev" in

Re: [net-next PATCH] RDS: convert bind hash table to re-sizable hashtable

On 10/30/2015 4:15 AM, David Miller wrote: From: Santosh Shilimkar <santosh.shilim...@oracle.com> Date: Wed, 28 Oct 2015 15:51:12 -0700 + >rs_bound_node, ht_parms)) { *port = rs->rs_bound_port;

[net-next PATCH v2] RDS: convert bind hash table to re-sizable hashtable

the flexibility to grow the hashtbable based on use and also comes up with inbuilt efficient bucket(chain) handling. Reviewed-by: David Miller <da...@davemloft.net> Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com&g

[RFC PATCH] RDS: convert bind hash table to re-sizable hashtable

From: Santosh Shilimkar <ssant...@kernel.org> To further improve the RDS connection scalabilty on massive systems where number of sockets grows into tens of thousands of sockets, there is a need of larger bind hashtable. Pre-allocated 8K or 16K table is not very flexible in terms of

Re: [RFC PATCH] RDS: convert bind hash table to re-sizable hashtable

Hi David L, On 10/14/2015 2:15 PM, Santosh Shilimkar wrote: From: Santosh Shilimkar <ssant...@kernel.org> To further improve the RDS connection scalabilty on massive systems where number of sockets grows into tens of thousands of sockets, there is a need of larger bind hashtabl

[PATCH net-next] RDS: fix rds-ping deadlock over TCP transport

sport like IB. Reported-by: Sowmini Varadhan <sowmini.varad...@oracle.com> Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/send.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff

[PATCH v2 net-next] RDS: fix rds-ping deadlock over TCP transport

sport like IB. Reported-by: Sowmini Varadhan <sowmini.varad...@oracle.com> Acked-by: Sowmini Varadhan <sowmini.varad...@oracle.com> Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- v2: Dropped the co

Re: [PATCH net-next] RDS: fix rds-ping deadlock over TCP transport

On 10/16/2015 6:45 PM, Sowmini Varadhan wrote: On (10/16/15 20:26), Santosh Shilimkar wrote: diff --git a/net/rds/send.c b/net/rds/send.c + if (!test_bit(RDS_LL_SEND_FULL, >c_flags)) + queue_delayed_work(rds_wq, >c_send_w, 0); A minor note- it would help to ad

Re: [RFC PATCH] RDS: convert bind hash table to re-sizable hashtable

On 10/18/2015 6:56 PM, David Miller wrote: From: Santosh Shilimkar <santosh.shilim...@oracle.com> Date: Wed, 14 Oct 2015 14:15:31 -0700 From: Santosh Shilimkar <ssant...@kernel.org> To further improve the RDS connection scalabilty on massive systems where number of sockets gro

[PATCH v3 14/14] RDS: IB: split mr pool to improve 8K messages performance

lead to 8k pull being exhausted, we fall-back to 1m pool till 8k pool recovers for use. This helps to at least push ~55 kB/s bidirectional data which is a nice improvement. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

[PATCH v3 11/14] RDS: IB: use already available pool handle from ibmr

rds_ib_mr already keeps the pool handle which it associates with. Lets use that instead of round about way of fetching it from rds_ib_device. No functional change. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com>

[PATCH v3 07/14] RDS: IB: ack more receive completions to improve performance

mutual exclusion locks are being performed. In next patch, send completion is also split which re-uses the poll_cq() and hence the code is moved to ib_cm.c Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> ---

[PATCH v3 13/14] RDS: IB: use max_mr from HCA caps than max_fmr

All HCA drivers seems to popullate max_mr caps and few of them do both max_mr and max_fmr. Hence update RDS code to make use of max_mr. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.c | 4

[PATCH v3 04/14] RDS: Use per-bucket rw lock for bind hash-table

-bucket lock to improve the scalability. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 2 ++ net/rds/bind.c | 47 --- net/rds/rds.h|

[PATCH v3 00/14] RDS: connection scalability and performance improvements

t.kernel.org/pub/scm/linux/kernel/git/ssantosh/linux.git net/rds/4.3-v3 As a side note, the IB HCA driver I used for testing misses at least 3 important patches in upstream to see the full blown IB performance and am hoping to get that in mainline with help of them. Santosh Shilimkar (14):

[PATCH v3 08/14] RDS: IB: split send completion handling and do batch ack

. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.h | 6 ++- net/rds/ib_cm.c| 45 -- net/rds/ib_send.c | 110 + net/rd

[PATCH v3 01/14] RDS: use kfree_rcu in rds_ib_remove_ipaddr

synchronize_rcu() slowing down un-necessarily the socket shutdown path. It is used just kfree() the ip addresses in rds_ib_remove_ipaddr() which is perfect usecase for kfree_rcu(); So lets use that to gain some speedup. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Sign

[PATCH v3 09/14] RDS: IB: handle rds_ibdev release case instead of crashing the kernel

Just in case we are still handling the QP receive completion while the rds_ibdev is released, drop the connection instead of crashing the kernel. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib_cm.

[PATCH v3 10/14] RDS: IB: fix the rds_ib_fmr_wq kick call

RDS IB mr pool has its own workqueue 'rds_ib_fmr_wq', so we need to use queue_delayed_work() to kick the work. This was hurting the performance since pool maintenance was less often triggered from other path. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shi

[PATCH v3 02/14] RDS: make socket bind/release locking scheme simple and more efficient

for bind hash-table. In subsequent patch, we also covert the global lock with per-bucket lock to reduce the global lock contention. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 6 ---

[PATCH v3 12/14] RDS: IB: mark rds_ib_fmr_wq static

Fix below warning by marking rds_ib_fmr_wq static net/rds/ib_rdma.c:87:25: warning: symbol 'rds_ib_fmr_wq' was not declared. Should it be static? Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/i

[PATCH v3 03/14] RDS: fix rds_sock reference bug while doing bind

One need to take rds socket reference while using it and release it once done with it. rds_add_bind() code path does not do that so lets fix it. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds

[PATCH v3 05/14] RDS: defer the over_batch work to send worker

to loop forever inside rds_send_xmit"). The idea is to process as big a batch as we can but at the same time we don't hold other waiting processes for send. Hence back-off after the send_batch_count limit (1024) to avoid soft-lock ups. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org&

[PATCH v3 06/14] RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL

we are seeing connections stuck with the LL_SEND_FULL bit getting set and never cleared. We kick krdsd after any time we see -ENOMEM or -EAGAIN from the ring allocation code. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

Re: [PATCH v3 05/14] RDS: defer the over_batch work to send worker

On 10/5/2015 10:56 AM, Santosh Shilimkar wrote: Darn. Header hunk remained in the repo. Resending it. From 4bebdd7a4d2960b2ff6c40b27156d041ea270765 Mon Sep 17 00:00:00 2001 From: Santosh Shilimkar <santosh.shilim...@oracle.com> Date: Thu, 10 Sep 2015 11:57:14 -0700 Subject: [PATCH v3

[PATCH v4 01/14] RDS: use kfree_rcu in rds_ib_remove_ipaddr

synchronize_rcu() slowing down un-necessarily the socket shutdown path. It is used just kfree() the ip addresses in rds_ib_remove_ipaddr() which is perfect usecase for kfree_rcu(); So lets use that to gain some speedup. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Sign

[PATCH v4 13/14] RDS: IB: use max_mr from HCA caps than max_fmr

All HCA drivers seems to popullate max_mr caps and few of them do both max_mr and max_fmr. Hence update RDS code to make use of max_mr. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.c | 4

[PATCH v4 05/14] RDS: defer the over_batch work to send worker

to loop forever inside rds_send_xmit"). The idea is to process as big a batch as we can but at the same time we don't hold other waiting processes for send. Hence back-off after the send_batch_count limit (1024) to avoid soft-lock ups. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org&

[PATCH v4 14/14] RDS: IB: split mr pool to improve 8K messages performance

lead to 8k pull being exhausted, we fall-back to 1m pool till 8k pool recovers for use. This helps to at least push ~55 kB/s bidirectional data which is a nice improvement. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

[PATCH v4 07/14] RDS: IB: ack more receive completions to improve performance

mutual exclusion locks are being performed. In next patch, send completion is also split which re-uses the poll_cq() and hence the code is moved to ib_cm.c Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> ---

[PATCH v4 11/14] RDS: IB: use already available pool handle from ibmr

rds_ib_mr already keeps the pool handle which it associates with. Lets use that instead of round about way of fetching it from rds_ib_device. No functional change. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com>

[PATCH v4 10/14] RDS: IB: fix the rds_ib_fmr_wq kick call

RDS IB mr pool has its own workqueue 'rds_ib_fmr_wq', so we need to use queue_delayed_work() to kick the work. This was hurting the performance since pool maintenance was less often triggered from other path. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shi

[PATCH v4 03/14] RDS: fix rds_sock reference bug while doing bind

One need to take rds socket reference while using it and release it once done with it. rds_add_bind() code path does not do that so lets fix it. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds

[PATCH v4 08/14] RDS: IB: split send completion handling and do batch ack

. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.h | 6 ++- net/rds/ib_cm.c| 45 -- net/rds/ib_send.c | 110 + net/rd

[PATCH v4 09/14] RDS: IB: handle rds_ibdev release case instead of crashing the kernel

Just in case we are still handling the QP receive completion while the rds_ibdev is released, drop the connection instead of crashing the kernel. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib_cm.

[PATCH v4 06/14] RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL

we are seeing connections stuck with the LL_SEND_FULL bit getting set and never cleared. We kick krdsd after any time we see -ENOMEM or -EAGAIN from the ring allocation code. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

[PATCH v4 02/14] RDS: make socket bind/release locking scheme simple and more efficient

for bind hash-table. In subsequent patch, we also covert the global lock with per-bucket lock to reduce the global lock contention. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 6 ---

[PATCH v4 00/14] RDS: connection scalability and performance improvements

blown IB performance and am hoping to get that in mainline with help of them. Santosh Shilimkar (14): RDS: use kfree_rcu in rds_ib_remove_ipaddr RDS: make socket bind/release locking scheme simple and more efficient RDS: fix rds_sock reference bug while doing bind RDS: Use per-bucket rw lock

[PATCH v4 12/14] RDS: IB: mark rds_ib_fmr_wq static

Fix below warning by marking rds_ib_fmr_wq static net/rds/ib_rdma.c:87:25: warning: symbol 'rds_ib_fmr_wq' was not declared. Should it be static? Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/i

[PATCH v4 04/14] RDS: Use per-bucket rw lock for bind hash-table

-bucket lock to improve the scalability. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 2 ++ net/rds/bind.c | 47 --- net/rds/rds.h|

[PATCH 2/6] RDS: Fix rds MR reference count in rds_rdma_unuse()

] irq_exit+0xa6/0xb0 [81613915] do_IRQ+0x65/0xf0 [816118ab] common_interrupt+0x6b/0x6b Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/rdma.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions

[PATCH 0/6] RDS: Few more fixes

As indicated in the earlier series [1], this is a follow-up series which addresses few issues around the RDS FMR code. With [1] and the subject series, now I can run many parallel threads with multiple sockets with N x N traffic. The stress tests has survived overnight runs. Santosh Shilimkar (5

[PATCH 6/6] RDS: remove superfluous from rds_ib_alloc_fmr()

Memory allocated for 'ibmr' uses kzalloc_node() which already initialises the memory to zero. There is no need to do memset() 0 on that memory. Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/ib_rdma.c | 2 -- 1 file

[PATCH 4/6] RDS: push FMR pool flush work to its own worker

RDS FMR flush operation and also it races with connect/reconect which happes a lot with RDS. FMR flush being on common rds_wq aggrevates the problem. Lets push RDS FMR pool flush work to its own worker. Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar

[PATCH 3/6] RDS: fix fmr pool dirty_count

From: Wengang Wang wen.gang.w...@oracle.com In rds_ib_flush_mr_pool(), dirty_count accounts the clean ones which is wrong. This can lead to a negative dirty count value. Lets fix it. Signed-off-by: Wengang Wang wen.gang.w...@oracle.com Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed

[PATCH 1/6] RDS: fix the dangling reference to rds_ib_incoming_slab

On rds_ib_frag_slab allocation failure, ensure rds_ib_incoming_slab is not pointing to the detsroyed memory. Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds/ib_recv.c | 5 +++-- 1 file changed, 3 insertions(+), 2

[PATCH 5/6] RDS: flush the FMR pool less often

FMR flush is an expensive and time consuming operation. Reduce the frequency of FMR pool flush by 50% so that more FMR work gets accumulated for more efficient flushing. Signed-off-by: Santosh Shilimkar ssant...@kernel.org Signed-off-by: Santosh Shilimkar santosh.shilim...@oracle.com --- net/rds

Re: [PATCH] RDS: verify the underlying transport exists before creating a connection

Sasha, On 9/4/2015 9:43 AM, Sasha Levin wrote: There was no verification that an underlying transport exists when creating a connection, this would cause dereferencing a NULL ptr. Signed-off-by: Sasha Levin --- net/rds/connection.c |6 ++ 1 file changed, 6

[PATCH v2 06/14] RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL

we are seeing connections stuck with the LL_SEND_FULL bit getting set and never cleared. We kick krdsd after any time we see -ENOMEM or -EAGAIN from the ring allocation code. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

[PATCH v2 07/14] RDS: IB: ack more receive completions to improve performance

mutual exclusion locks are being performed. In next patch, send completion is also split which re-uses the poll_cq() and hence the code is moved to ib_cm.c Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> ---

[PATCH v2 01/14] RDS: use kfree_rcu in rds_ib_remove_ipaddr

synchronize_rcu() slowing down un-necessarily the socket shutdown path. It is used just kfree() the ip addresses in rds_ib_remove_ipaddr() which is perfect usecase for kfree_rcu(); So lets use that to gain some speedup. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Sign

[PATCH v2 10/14] RDS: IB: fix the rds_ib_fmr_wq kick call

RDS IB mr pool has its own workqueue 'rds_ib_fmr_wq', so we need to use queue_delayed_work() to kick the work. This was hurting the performance since pool maintenance was less often triggered from other path. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shi

[PATCH v2 08/14] RDS: IB: split send completion handling and do batch ack

. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.h | 6 ++- net/rds/ib_cm.c| 45 -- net/rds/ib_send.c | 110 + net/rd

[PATCH v2 04/14] RDS: Use per-bucket rw lock for bind hash-table

-bucket lock to improve the scalability. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 2 ++ net/rds/bind.c | 47 --- net/rds/rds.h|

[PATCH v2 14/14] RDS: IB: split mr pool to improve 8K messages performance

lead to 8k pull being exhausted, we fall-back to 1m pool till 8k pool recovers for use. This helps to at least push ~55 kB/s bidirectional data which is a nice improvement. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

[PATCH v2 13/14] RDS: IB: use max_mr from HCA caps than max_fmr

All HCA drivers seems to popullate max_mr caps and few of them do both max_mr and max_fmr. Hence update RDS code to make use of max_mr. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.c | 4

[PATCH v2 05/14] RDS: defer the over_batch work to send worker

case where we reached to over_batch code for some other reason so just retrying again before giving up. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/send.c | 4 +++- 1 file changed, 3 insertions(+), 1 d

[PATCH v2 03/14] RDS: fix rds_sock reference bug while doing bind

One need to take rds socket reference while using it and release it once done with it. rds_add_bind() code path does not do that so lets fix it. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds

[PATCH v2 02/14] RDS: make socket bind/release locking scheme simple and more efficient

for bind hash-table. In subsequent patch, we also covert the global lock with per-bucket lock to reduce the global lock contention. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 6 ---

[PATCH v2 09/14] RDS: IB: handle rds_ibdev release case instead of crashing the kernel

From: Santosh Shilimkar <ssant...@kernel.org> Just in case we are still handling the QP receive completion while the rds_ibdev is released, drop the connection instead of crashing the kernel. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> --- net/rds/ib_cm.c | 3 ++- 1 fil

[PATCH v2 00/14] RDS: connection scalability and performance improvements

kernel.org/pub/scm/linux/kernel/git/ssantosh/linux.git net/rds/4.3-v2 As a side note, the IB HCA driver I used for testing misses at least 3 important patches in upstream to see the full blown IB performance and am hoping to get that in mainline with help of them. Santosh Shilimkar (14): RDS: use

Re: [PATCH v2 net-next 0/3] RDS: RDS-TCP perf enhancements

, straightforward perf bug fixes that align the RDS TCP socket with other parts of the kernel stack. v2: fix kbuild-test-robot warnings, comments from Sergei Shtylov and Santosh Shilimkar. Acked-by: Santosh Shilimkar <santosh.shilim...@oracle.com> -- To unsubscribe from this list: send th

[PATCH v2 11/14] RDS: IB: use already available pool handle from ibmr

rds_ib_mr already keeps the pool handle which it associates with. Lets use that instead of round about way of fetching it from rds_ib_device. No functional change. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com>

Re: [PATCH net-next 1/3] net/rds: Use a single TCP socket for both send and receive.

On 9/30/2015 8:58 AM, Sowmini Varadhan wrote: On (09/30/15 08:50), santosh shilimkar wrote: minor nit though not a strict rule. Just to be consistent based on what we are following. - core RDS patches "RDS:" - RDS IB patches "RDS: IB:" or "RDS/IB:" - RDS IW patch

Re: [PATCH net-next 1/3] net/rds: Use a single TCP socket for both send and receive.

On 9/30/2015 9:09 AM, Sowmini Varadhan wrote: On (09/30/15 08:50), santosh shilimkar wrote: rs_tcp = (struct rds_tcp_connection *)conn->c_transport_data; - WARN_ON(!rs_tcp || rs_tcp->t_sock); + if (rs_tcp->t_sock && inet->inet_sadd

Re: [PATCH net-next 1/3] net/rds: Use a single TCP socket for both send and receive.

ed, we had an issue. That guards it now. Am curious about WARN_ON() and hence the question. Rest of the patch looks fine to me. Acked-by: Santosh Shilimkar <santosh.shilim...@oracle.com> -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH net-next 2/3] RDS-TCP: Do not bloat sndbuf/rcvbuf in rds_tcp_tune

k. Almost ~3X lift in RDS TCP performance. Acked-by: Santosh Shilimkar <santosh.shilim...@oracle.com> -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH net-next 3/3] RDS-TCP: Set up MSG_MORE and MSG_SENDPAGE_NOTLAST as appropriate in rds_tcp_xmit

Your checkpatch.pl should have complained about commit reference in the change-log. You might want to fix that for consistency. Patch itself is fine. Acked-by: Santosh Shilimkar <santosh.shilim...@oracle.com> -- To unsubscribe from this list: send the line "unsubscribe netdev" in

[PATCH 03/15] RDS: fix rds_sock reference bug while doing bind

One need to take rds socket reference while using it and release it once done with it. rds_add_bind() code path does not do that so lets fix it. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds

[PATCH 13/15] RDS: mark rds_ib_fmr_wq static

Fix below warning by marking rds_ib_fmr_wq static net/rds/ib_rdma.c:87:25: warning: symbol 'rds_ib_fmr_wq' was not declared. Should it be static? Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/i

[PATCH 08/15] RDS: ack more receive completions to improve performance

mutual exclusion locks are being performed. In next patch, send completion is also split which re-uses the poll_cq() and hence the code is moved to ib_cm.c Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> ---

[PATCH 11/15] RDS: fix the rds_ib_fmr_wq kick call

RDS IB mr pool has its own workqueue 'rds_ib_fmr_wq', so we need to use queue_delayed_work() to kick the work. This was hurting the performance since pool maintenance was less often triggered from other path. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shi

[PATCH 00/15] RDS: connection scalability and performance improvements

at least 3 important patches in upstream to see the full blown RDS IB performance and am hoping to get that in mainline with help of them. Santosh Shilimkar (15): RDS: use kfree_rcu in rds_ib_remove_ipaddr RDS: make socket bind/release locking scheme simple and more efficient RDS: fix rds_sock

[PATCH 10/15] RDS: handle rds_ibdev release case instead of crashing the kernel

From: Santosh Shilimkar <ssant...@kernel.org> Just in case we are still handling the QP receive completion while the rds_ibdev is released, drop the connection instead of crashing the kernel. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> --- net/rds/ib_cm.c | 3 ++- 1 fil

[PATCH 01/15] RDS: use kfree_rcu in rds_ib_remove_ipaddr

synchronize_rcu() slowing down un-necessarily the socket shutdown path. It is used just kfree() the ip addresses in rds_ib_remove_ipaddr() which is perfect usecase for kfree_rcu(); So lets use that to gain some speedup. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Sign

[PATCH 12/15] RDS: use already available pool handle from ibmr

rds_ib_mr already keeps the pool handle which it associates with. Lets use that instead of round about way of fetching it from rds_ib_device. No functional change. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com>

[PATCH 15/15] RDS: split mr pool to improve 8K messages performance

lead to 8k pull being exhausted, we fall-back to 1m pool till 8k pool recovers for use. This helps to at least push ~55 kB/s bidirectional data which is a nice improvement. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

[PATCH 05/15] RDS: increase size of hash-table to 8K

as the bind hash-table size. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/bind.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/rds/bind.c b/net/rds/bind.c index bc6b93e..fb

[PATCH 06/15] RDS: defer the over_batch work to send worker

case where we reached to over_batch code for some other reason so just retrying again before giving up. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/send.c | 4 +++- 1 file changed, 3 insertions(+), 1 d

[PATCH 04/15] RDS: Use per-bucket rw lock for bind hash-table

-bucket lock to improve the scalability. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 2 ++ net/rds/bind.c | 47 --- net/rds/rds.h|

[PATCH 02/15] RDS: make socket bind/release locking scheme simple and more efficient

for bind hash-table. In subsequent patch, we also covert the global lock with per-bucket lock to reduce the global lock contention. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/af_rds.c | 6 ---

[PATCH 14/15] RDS: use max_mr from HCA caps than max_fmr

From: Santosh Shilimkar <ssant...@kernel.org> All HCA drivers seems to popullate max_mr caps and few of them do both max_mr and max_fmr. Hence update RDS code to make use of max_mr. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <

[PATCH 09/15] RDS: split send completion handling and do batch ack

. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@oracle.com> --- net/rds/ib.h | 6 ++- net/rds/ib_cm.c| 45 -- net/rds/ib_send.c | 110 + net/rd

[PATCH 07/15] RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL

we are seeing connections stuck with the LL_SEND_FULL bit getting set and never cleared. We kick krdsd after any time we see -ENOMEM or -EAGAIN from the ring allocation code. Signed-off-by: Santosh Shilimkar <ssant...@kernel.org> Signed-off-by: Santosh Shilimkar <santosh.shilim...@o

Re: [PATCH 05/15] RDS: increase size of hash-table to 8K

On 9/21/2015 4:05 PM, David Miller wrote: From: Santosh Shilimkar <santosh.shilim...@oracle.com> Date: Sat, 19 Sep 2015 19:04:42 -0400 Even with per bucket locking scheme, in a massive parallel system with active rds sockets which could be in excess of multiple of 10K, rds_bin_

Re: [PATCH 00/15] RDS: connection scalability and performance improvements

On 9/20/2015 1:37 AM, Sagi Grimberg wrote: On 9/20/2015 2:04 AM, Santosh Shilimkar wrote: This series addresses RDS connection bottlenecks on massive workloads and improve the RDMA performance almost by 3X. RDS TCP also gets a small gain of about 12%. RDS is being used in massive systems

  1   2   3   4   >