Hi All - I dont think we ever got around to testing this with multiple ports active on each HCA when we wrote it, so I believe I hard coded it to just default to the first port... iirc we tried to bring up the 2nd port at one point and found that there were some memory exhaustion issues when using more than one port AND the default HCA buffers/MTUs/etc on the cards that this was primarily tested on so it went back to 1.
I wouldn't recommend changing this via a hard code for obvious reasons, but at the same time it probably wouldn't take more than 20-30 lines of code to fix this up to take more than one port. I'll try to take a look at it. Cheers, Kyle Schochenmaier On Mon, Jan 30, 2012 at 9:21 AM, Phil Carns <[email protected]> wrote: > Hi Vlad, > > Let me prefact this by saying that I don't know much about InfiniBand, but I > was curious about this and looked in the src/io/bmi/bmi_ib/openib.c source > code, where I saw this comment: > > od->nic_port = IBV_PORT; /* maybe let this be configurable */ > > IBV_PORT is hard coded to 1 with a #define at the top of the file. This is > really just a guess on my part, but maybe you could try modifying that > #define to 2 instead, recompile, and see if that works? As the comment > indicates, this port value should probably be configurable but it doesn't > look like it is right now. > > -Phil > > > On 01/30/2012 09:44 AM, vlad wrote: >> >> Hi! >> >> I`m evaluating now version 2.8.5 and I have been able to connect and >> use hosts with single port infiniband cards (Mellanox Infinihost III, ) >> using the ib0 Interface, connection set to >> >> ib://host:3335/pvfs2 >> >> Integrating of hosts with 2-port cards fails. On this hosts I have >> plugged in port ib1, configured IPoverIB and I'm able to ping between the >> infiniband hosts. I have also an OpenSM Subnet Manager running on each >> host. >> >> When starting the servers on these hosts they complain about that they >> cannot reach the other pvfs2-servers and I should check if my >> subnetmanager >> on that particular host is running. >> >> I suppose, the connection request is sent on ib0 instead of ib1. >> >> My question is, if there are any parameters to tell the server to use port >> 1 instead of port 0 >> >> Despite of all, everything runs very fine and we have very nice data >> transfers (the 1-port 5 Nodes are writing with 415 MB per second). >> >> >> I would appreciate any help. >> >> Greeting from Salzburg/Austria/Europe >> >> Vlad Popa >> >> University of Salzburg >> Jakob-Harringer-Str.2 >> 5020 Salzburg >> Austria/Europe >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
