Hi Olga, Olga Shern (Voltaire) wrote: > Hi Marcel, > > We also did tests with multicast traffic and figured out that when > using Arbel HCAs
As far as I can tell, we're also using "Arbel" HCAs. > there is performance penalty when: > 1. Sending MC packets with NO QP attached on local Arble HCA. > 2. Receiving MC packet with more then 1 QP attached on a single HCA > which causes the back pressure that slows down the sender. > > Hope this helps This explains some of the effects I've seen. We only need one sender and n receivers, all on separate hosts. I've only tested all that other scenarios because I wanted to find out more about why the performance was so bad in that original scenario. Attaching the send QP to the group as a workaround shouldn't be a problem. However, my numbers suggest that attaching alone isn't enough, the data has to be received actually, which hasn't any negative impact on my benchmark, but is consuming up to another 1GByte/s of memory bandwidth, which could hurt the real application. Besides of that, there are still performance problems with n > 1, as I mentioned in the original post (c is the client sending the data, s is the server, receiving): | We could test on another cluster with 6 nodes (also with MT25204 HCAs, | I don't know the OFED version and switch type) and got the following | results: | | Host1 Host2 Host3 Host4 Host5 Host6 Throughput (MByte/s) | 1s 1s 1c 255,15 | 1s 1s 1s 1c 255,22 | 1s 1s 1s 1s 1c 255,22 | 1s 1s 1s 1s 1s 1c 255,22 | | 1s1c 1s 1s 738,64 | 1s1c 1s 1s 1s 695,08 | 1s1c 1s 1s 1s 1s 565,14 | 1s1c 1s 1s 1s 1s 1s 451,90 But maybe that was a problem of the switch or general setup. I will make some tests with our own switch as soon as I got installed an additional host. Testing multicast with only two hosts is a bit odd... ;-) Thank you! Regards, Marcel _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
