Thanks for all your suggestions. TCPoIB seems to be doing the job. Regards, Alvaro.
On Thu, Feb 24, 2011 at 4:16 AM, Liang Zhen <[email protected]> wrote: > > On Feb 24, 2011, at 10:45 AM, Jeremy Filizetti wrote: > > > As Chris mentioned your talking about two very different methods. I > think you can use netem with IPoIB but I have never tried it. If you use > connected mode I think your still technically doing RDMA but the maximum > size (MTU) is around 64k which isn't sufficient for higher latencies. In > the first few slides of my LUG presentation last year I have some graphs > that show how RDMA performance is affected by latency and need to be > increased to compensate for the bandwidth delay product (BDP). If you do > what to use IPoIB you can add a line similar the following in your > /etc/modprobe.conf or a fille in /etc/modprobe.d directory: > > > > options lnet networks=tcp(ib0) > > > > If you want to use RC QPs as ko2iblnd does, we use the following kernel > parameters: > > > > options lnet networks=o2ib(ib0) > > options ko2iblnd map_on_demand=2 peer_credits=128 credits=256 > concurrent_sends=256 ntx=512 fmr_pool_size=2048 fmr_flush_trigger=512 > fmr_cache=1 > > > If you have peer_credits=128, then I would suggest increase credits=1024 > and ntx=2048, otherwise a couple of clients could consume all NI credits, > concurrent_sends is not necessary here because o2iblnd will estimate proper > value for it. > > Thanks > Liang > > > > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
