On Mon, 2005-04-11 at 12:28, Roland Fehrenbacher wrote: > Hi, > > I got gen2 opensm running fine now (there was a problem with a wrong > include file), and managed to get IP running on a network of > currently 40 machines (final size will be 144). Performance is pretty > impressive (initial tests with a simple netpipe): I got a latency of > 18microsec, and a maximum throughput of approx. 400MB/sec at packet > size approx. 1MB which then levels of at about 340MB/s for larger > packets.
That's all good to hear :-) > One problem and two questions: > > Problem: When I reboot all the 40 nodes (apart from the one the opensm > is running), the network is non-functional (no pings go through, even > though ports show status "Active") for quite a while (more than 10 > minutes) after all the nodes have come up. It then recovers without > intervention. Is this normal? Single node reboots don't affect the > network operation. osm Log file is appended. Can you describe your topology ? Is it the following: the SM is connected to a switch/or switches with the 40 nodes connected off these switches ? I'll respond to the log (and these questions) in a separate email response. > Question 1: Can I run opensm in a master slave configuration? Yes. Others are doing this. > I noticed > that there is a priority commandline option, but am not sure how to > apply this. SM election occurs per high priority low GUID. So if you don't care which SM is the master than you don't need to do anything. If you want a specific order (and it is not in GUID order) then you need to specify priority. > Question 2: I plan to run the gen1/Mellanox IBGD drivers on the > compute nodes (need fast MPI), and gen2 on the control/storage nodes > (need only IP) with gen2 opensm running on the control nodes. Is there > any reason why this should not work reliably? So basically this appears to be an interop question: 1. Will gen2 OpenSM support IBGD nodes ? 2. Will gen2 IPoIB interoperate with IBGD IPoIB ? I haven't done this but know of no reasons this should not work. Perhaps others can add to this. -- Hal ________________________________________________________________________ _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
