Hello, Louis-Claude being working in my lab I just answered him in person, but I thought the answer could be interesting for other users and more particularly for the developers:
It seems that the script provided to stop the server simply calls the "kill" command, but pvfs creates some shared memory objects using shmget and these objects are not deleted. A call to ipcrm solves the problem, but it could be good to have the script properly cleaning these shared memory objects too. Matthieu 2012/2/27 Louis-Claude Canon <[email protected]> > Hello, > > I am facing a "no space left" problem when configuring two servers with > OrangeFS 2.8.5. In my experiment, I configure two OrangeFS servers, perform > some transfers, clean the configuration and repeat the process > indefinitely. After around 1 thousand iterations with the same two servers > (same nodes and same configuration options), I have the following situation > when launching the first server: > > $ /usr/sbin/pvfs2-server -a paradent-19.rennes.grid5000.fr/tmp/pvfs2.conf -f > [S 02/27/2012 14:34:15] PVFS2 Server on node > paradent-19.rennes.grid5000.fr version 2.8.5-orangefs starting... > [D 02/27/2012 14:34:15] PVFS2 Server: storage space created. Exiting. > $ /usr/sbin/pvfs2-server -a paradent-19.rennes.grid5000.fr /tmp/pvfs2.conf > [S 02/27/2012 14:34:17] PVFS2 Server on node > paradent-19.rennes.grid5000.fr version 2.8.5-orangefs starting... > $ cd /tmp > $ cat pvfs2-server.log > [D 02/27/2012 14:34:17] PVFS2 Server version 2.8.5-orangefs starting. > [E 02/27/2012 14:34:17] (null): shmget: key: 2043294211: unable to create > shared system memory region: No space left on device > [E 02/27/2012 14:34:17] dbpf_getdb_env(//tmp/pvfs2-**storage/53405cc2/): > No space left on device > [E 02/27/2012 14:34:17] Error initializing trove for filesystem pvfs2-fs > [E 02/27/2012 14:34:17] Error: Could not initialize server subsystems > [E 02/27/2012 14:34:17] Error: Could not initialize server; aborting. > [E 02/27/2012 14:34:17] Trove collection not defined. > $ df -h . > Filesystem Size Used Avail Use% Mounted on > /dev/sda5 279G 192M 265G 1% /tmp > $ cat pvfs2.conf > <Defaults> > UnexpectedRequests 50 > EventLogging none > EnableTracing no > LogStamp datetime > BMIModules bmi_tcp > FlowModules flowproto_multiqueue > PerfUpdateInterval 1000 > ServerJobBMITimeoutSecs 30 > ServerJobFlowTimeoutSecs 30 > ClientJobBMITimeoutSecs 300 > ClientJobFlowTimeoutSecs 300 > ClientRetryLimit 5 > ClientRetryDelayMilliSecs 2000 > PrecreateBatchSize 0,32,512,32,32,32,0 > PrecreateLowThreshold 0,16,256,16,16,16,0 > > DataStorageSpace /tmp/pvfs2-storage > MetadataStorageSpace /tmp/pvfs2-storage > > LogFile /tmp/pvfs2-server.log > </Defaults> > > <Aliases> > Alias paradent-19.rennes.grid5000.fr tcp://paradent-19.rennes.** > grid5000.fr:3334 <http://paradent-19.rennes.grid5000.fr:3334> > Alias paradent-2.rennes.grid5000.fr tcp://paradent-2.rennes.** > grid5000.fr:3334 <http://paradent-2.rennes.grid5000.fr:3334> > </Aliases> > > <Filesystem> > Name pvfs2-fs > ID 1396726978 > RootHandle 1048576 > FileStuffing yes > <MetaHandleRanges> > Range paradent-19.rennes.grid5000.fr 3-2305843009213693953 > Range paradent-2.rennes.grid5000.fr 2305843009213693954-** > 4611686018427387904 > </MetaHandleRanges> > <DataHandleRanges> > Range paradent-19.rennes.grid5000.fr 4611686018427387905-** > 6917529027641081855 > Range paradent-2.rennes.grid5000.fr 6917529027641081856-** > 9223372036854775806 > </DataHandleRanges> > <StorageHints> > TroveSyncMeta yes > TroveSyncData no > TroveMethod alt-aio > </StorageHints> > </Filesystem> > > This is not a blocking problem for me, but I would prefer to understand > what is going on. Any ideas? > > Regards, > > Louis > ______________________________**_________________ > Pvfs2-users mailing list > Pvfs2-users@beowulf-**underground.org<[email protected]> > http://www.beowulf-**underground.org/mailman/**listinfo/pvfs2-users<http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users> > -- Matthieu Dorier ENS Cachan, Brittany (Computer Science dpt.) IRISA Rennes, Office C113 http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
