Chuck Hartley wrote: > What kind of machines are the cards installed in? - Intel Xeon Quadcore; CPU E5420 @ 2.50GHz - 4 GB RAM - Super X7DCL Mainboard
(The 10 GBit/s TCP bandwidth has also been meassured on superior 64 bit machines running Ubuntu 8.04, also using Mellanox Infinihost III Lx DDR HCA cards in a 8x PCIe slots, when using a basic Infiniband installation. When trying to set up Infiniband by manually installing the OFED 1.3. packages on one of the machines, the maximum bandwidth meassured locally was also 12 GBits/s. The trial-and-error installation ran into some problems when trying to set up the more advanced Infiniband features, though. So the decision was made to test Infiniband on a system that supported the OFED install script to get a cleaner installation; the results have been summarized in the previous mail.) > Do you have the cards in 8x PCIe slots? The cards are in 8x PCIe slots. BIOS settings for the slots are as follows: 8x PCIe, Slot 6: Option ROM Scan [Enabled] Enable Master [Enabled] Latency Timer [Default] No other PCI(e)-Slots have any cards installed. (According to the manual, "Each PCI Express port on the MCH provides 4 GB/s bidirectional bandwidth if configured as a x8 port[...]". This would equal 32 Gbit/s, and even if unidirectional speed was half of that (I'm no expert on the PCIe bus), it would still be above the meassured Infiniband bandwidth by about 4 Gbit/s unidirectional. So I'm somewhat hesitant to assume PCI Express as the bottleneck for an Infiniband bandwidth of 12 GBit/s.) -- Jan Ruffing Software Developer Motama GmbH Lortzingstraße 10 · 66111 Saarbrücken · Germany tel +49 681 940 85 50 · fax +49 681 940 85 49 [EMAIL PROTECTED] · www.motama.com Companies register · district council Saarbrücken · HRB 15249 CEOs · Dr.-Ing. Marco Lohse, Michael Repplinger This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
