Grant Grundler wrote:
If it's NAPI that means nothing, here's probably the best summary:
http://lwn.net/Articles/30098/
Cool; I see now.
But IPoIB can't really implement NAPI since it's sending work to
a shared HCA.
Hmm. I'm not knowledgeable to know why; I'll have to take your word for it. I'm not sure yet all the conditions that the HCA can generate interrupts.
But if I sit back and look at the logic of this arguement then it seemed like:
Hey. is there was a way to not generate so many interrupts? That's handled by NAPI OK. That looks interesting. But, we can't do NAPI because we can't just disable interrupts. Darn. But wait, why can't we just not generate interrupts in the first place then?
Isn't that what the midas touch of netdev->poll() really is? e1000 has:
quit_polling: netif_rx_complete(netdev);
e1000_irq_enable(adapter);Maybe IB can mimic the concept here by acting intellegently for us? Have disable_rx_and_rxnobuff_ints() only disable interrupts for the IPoIB ULP? Anyway, my knowledge here still sucks so I'm probably so far off base I'm not even on the field. Either way it's fun digging around here.
NAPI, TOE, and lots of other things help ethernet with throughput.
Hmmm, TOE is interesting.
One can. Using SDP, netperf TCP_STREAM measured 650 MB/s using the
regular PCI-X card.
Yes, I have the same speed results using perf_main().
The perf_main() test isn't that interesting I think though. It really just transfers the exact same memory window across two nodes. (at least as far as I can tell that is what it does).
Anyway, I'm just noticing that this simple dd test from memory doesn't go much over 1GB/sec. So this is an interesting non-IB problem.
[EMAIL PROTECTED]:/# dd if=/dev/shm/test of=/dev/null bs=4K 196608+0 records in 196608+0 records out 805306368 bytes transferred in 0.628504 seconds (1281306571 bytes/sec)
I wonder if that could be faster with bigger socket buffers but don't know (IIRC, IB socket buffers are fixed at 256KB). Anyway, that's with about 60% CPU (1.5 Ghz ia64).
I've got a 3.2Ghz Xeon. 2GB ram.
I haven't checked to see what libibverbs can do with "native" mode (if one is willing/able to modify the application.)
42
ISTR Michael Tsirkin had another streaming test in mind to measure peak bandwidth. But I'm not quite done with rdma_lat yet anyway...
Enjoy, Jeff _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
