As Chris mentioned your talking about two very different methods. I think you can use netem with IPoIB but I have never tried it. If you use connected mode I think your still technically doing RDMA but the maximum size (MTU) is around 64k which isn't sufficient for higher latencies. In the first few slides of my LUG presentation last year I have some graphs that show how RDMA performance is affected by latency and need to be increased to compensate for the bandwidth delay product (BDP). If you do what to use IPoIB you can add a line similar the following in your /etc/modprobe.conf or a fille in /etc/modprobe.d directory:
options lnet networks=tcp(ib0) If you want to use RC QPs as ko2iblnd does, we use the following kernel parameters: options lnet networks=o2ib(ib0) options ko2iblnd map_on_demand=2 peer_credits=128 credits=256 concurrent_sends=256 ntx=512 fmr_pool_size=2048 fmr_flush_trigger=512 fmr_cache=1 I'm not aware of any patches that add delay to the OFED stack. At NRL we use real delay between sites and hardware simulated delay using Obsidian Longbow XRs (http://www.obsidianresearch.com). They support a single SDR connection and can add delay to the IB channel up to 1 second. Jeremy On Wed, Feb 23, 2011 at 5:19 PM, Andreas Dilger <[email protected]>wrote: > On 2011-02-23, at 8:44 AM, Alvaro Aguilera wrote: > > I want to do some tests injecting artificial latency to LNET using netem > over an InfiniBand network. This doesn't work out of the box, since LNET's > RDMA transfers aren't affected by the kernel latency. Does anyway know if > there is a way to disable LNET's RDMA support and force its packets through > the kernel? Other ideas are also welcomed. > > Depending on what you are trying to measure, you may want to contact some > of the folks at NRL, who have done a bunch of IB WAN latency testing. They > inject latency into the network using special hardware, so that probably > won't help you, but maybe they already measured what you are looking at :-). > > Cheers, Andreas > -- > Andreas Dilger > Principal Engineer > Whamcloud, Inc. > > > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
