Hi Vlad - Did you configure with the disable-tcp option for IB? -disable-bmi-tcp
In theory this would work aorund the problem since you wont be using TCP anymore Cheers, Kyle Schochenmaier On Mon, Mar 19, 2012 at 9:58 AM, vlad <[email protected]> wrote: > Hi ! > > I'm evaluating now orangefs 2.8.5 on ROCKS Cluster (Centox 5.x) with 13 > Opteron Nodes > All nodes (compute-0-1 to compute-0-14, but omitting compute-0-2) are > clients, metaservers, storage servers. They are connected via Infiniband , > so I configured > > a cluster configuration using the cluster-quick-howto to connect via > ib://compute-0-X:3335/pvfs2-fs > > The metaservers have their files stored under > /state/partition1/pvfs2-meta while the storage servers are homed under > /state/partition1/pvfs2-server. > > Each node hast its own hostname configured as storage-/meta- host in the > pvfs2tab, since I thought it is the nearest and probably fastest point to > reach. > > > After the 1st initiation on all nodes to create the structure needed by > the servers, I have started the servers simultaneusly (using the cluster > ssh cssh) in debug mode and now I'm receiving following errors on some > nodes: > > "... > [E 03/19/2012 15:19:55] Warning: ib_tcp_client_connect: create tcp socket: > Too many open files. > [E 03/19/2012 15:19:55] Receive immediately failed: Too many open files > ..." > > The PVFS2-ping test also stucks on point 3 in its list. > > I wonder, how could I fix this "too many open files" error and how to get > on to the 1st real test. I was going to use the 2.6 kernel module. > > I once had a setup working with approx 3 nodes, but this "too many open > files" often reoccurs. I have also noticed, that a performance test with > bonnie++ failed. > > Thanks for any help in advance! > > Greetings from Europe/Austria > > Vlad Popa > > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
