[I applied the patch below from Christoph, but I see now he sent it only to me and didn't cc the mailing list... so for completeness I'm including it below in case anyone else has concerns]
From: Christoph Lameter <c...@linux-foundation.org> The IPoIB drops multicast packets if more than 3 are queued while a multicast send is pending. The send queue can easily contain more than 3 packets and when the queue is processed only the first 3 packets will make it and the rest will be dropped. The IPoIB layer is unable to send a high speed stream of multicast traffic because of this. Since the packets are dropped the sender is never throttled and so continuous sending of data will lead to continual packet loss. To fix this, simply remove the code that drops packets, so the socket queue will build up and the sender will be throttled if it is continuously sending packets faster than the IPoIB layer can process them. This results in the IPoIB layer sending multicast traffic at the highest rate allowed by the underlying hardware and makes the IPoIB layer multicast behavior work like other ethernet NICs. (And it also means there's no risk of unbounded growth of pending packets, since they are accounted for as part of socket memory) We were able to send millions of UDP multicast messages at high rates after this patch. The tool for generating multicast traffic can be found on http://www.gentwo.org/ll. Signed-off-by: Christoph Lameter <c...@linux-foundation.org> Signed-off-by: Roland Dreier <rola...@cisco.com> --- drivers/infiniband/ulp/ipoib/ipoib.h | 1 - drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 7 +------ 2 files changed, 1 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h index 753a983..ddb5cd7 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib.h +++ b/drivers/infiniband/ulp/ipoib/ipoib.h @@ -79,7 +79,6 @@ enum { IPOIB_NUM_WC = 4, IPOIB_MAX_PATH_REC_QUEUE = 3, - IPOIB_MAX_MCAST_QUEUE = 3, IPOIB_FLAG_OPER_UP = 0, IPOIB_FLAG_INITIALIZED = 1, diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c index 425e311..e6ea76a 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c @@ -685,12 +685,7 @@ void ipoib_mcast_send(struct net_device *dev, void *mgid, struct sk_buff *skb) } if (!mcast->ah) { - if (skb_queue_len(&mcast->pkt_queue) < IPOIB_MAX_MCAST_QUEUE) - skb_queue_tail(&mcast->pkt_queue, skb); - else { - ++dev->stats.tx_dropped; - dev_kfree_skb_any(skb); - } + skb_queue_tail(&mcast->pkt_queue, skb); if (test_bit(IPOIB_MCAST_FLAG_BUSY, &mcast->flags)) ipoib_dbg_mcast(priv, "no address vector, " -- 1.6.0.4 _______________________________________________ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
