Hi, I have migrated my Lustre 2.2 to 2.5.1 and I have equipped my OSS/MDS and clients with Infiniband QDR interfaces. I have compile lustre with OFED 3.2 and I have configured lnet module with:
options lent networks=“o2ib(ib0),tcp(eth0)” But when I try to compare the lustre performance across Infiniband (o2ib), I get the same performance than across ethernet (tcp): INFINIBAND TEST: dd if=/dev/zero of=test.dat bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1,0 GB) copied, 5,88433 s, 178 MB/s ETHERNET TEST: dd if=/dev/zero of=test.dat bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1,0 GB) copied, 5,97423 s, 154 MB/s And this is my scenario: - 1 MDs with SSD RAID10 MDT - 10 OSS with 2 OST per OSS - Infiniband interface in connected mode - Centos 6.5 - Lustre 2.5.1 - Striped filesystem “lfs setstripe -s 1M -c 10" I know my infiniband running correctly, because if I use IPERF3 between client and servers I got 40Gb/s by infiniband and 1Gb/s by ethernet connections. Could you help me? Regards, Alfonso Pardo Diaz System Administrator / Researcher c/ Sola nº 1; 10200 Trujillo, ESPAÑA Tel: +34 927 65 93 17 Fax: +34 927 32 32 37 ---------------------------- Confidencialidad: Este mensaje y sus ficheros adjuntos se dirige exclusivamente a su destinatario y puede contener información privilegiada o confidencial. Si no es vd. el destinatario indicado, queda notificado de que la utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente respondiendo al mensaje y proceda a su destrucción. Disclaimer: This message and its attached files is intended exclusively for its recipients and may contain confidential information. If you received this e-mail in error you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited and may be unlawful. In this case, please notify us by a reply and delete this email and its contents immediately. ---------------------------- _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
