Since both the configuration and the IB link bandwidth looked fine, I'd suggest to measure lnet throughput by lnet selftest:
1. On both client and server: modprobe lnet_selftest 2. On the client: export LST_SESSION=$$ lst new_session --timeo 100000 test lst add_group s [EMAIL PROTECTED] lst add_group c [EMAIL PROTECTED] lst add_batch bw lst add_test --batch bw --loop 500 --concurrency 8 --distribute 1:1 --from c --to s brw write size=1M lst run bw lst stat c Isaac On Tue, Jun 03, 2008 at 07:39:44AM -0400, Murray Smigel wrote: > ...... > ** > On server: (I have tried both msi_x=1 and 0 with no difference) > modprobe.conf --> > options lnet networks="tcp,o2ib" > options mlx4_core msi_x=0 > alias ib0 ib_ipoib > alias ib1 ib_ipoib > [EMAIL PROTECTED] lustre-iokit-1.2]# lctl list_nids > [EMAIL PROTECTED] > [EMAIL PROTECTED] > On client: > /etc/modprobe.d/lustre --> > options lnet networks="tcp,o2ib" > options mlx4_core msi_x=1 > alias ib0 ib_ipoib > alias ib1 ib_ipoib > nasnu3:/home/murray/lustre-iokit-1.2# lctl list_nids > [EMAIL PROTECTED] > [EMAIL PROTECTED] > Thanks, > murray _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
