Hello,
I am facing a "no space left" problem when configuring two servers with
OrangeFS 2.8.5. In my experiment, I configure two OrangeFS servers,
perform some transfers, clean the configuration and repeat the process
indefinitely. After around 1 thousand iterations with the same two
servers (same nodes and same configuration options), I have the
following situation when launching the first server:
$ /usr/sbin/pvfs2-server -a paradent-19.rennes.grid5000.fr
/tmp/pvfs2.conf -f
[S 02/27/2012 14:34:15] PVFS2 Server on node
paradent-19.rennes.grid5000.fr version 2.8.5-orangefs starting...
[D 02/27/2012 14:34:15] PVFS2 Server: storage space created. Exiting.
$ /usr/sbin/pvfs2-server -a paradent-19.rennes.grid5000.fr /tmp/pvfs2.conf
[S 02/27/2012 14:34:17] PVFS2 Server on node
paradent-19.rennes.grid5000.fr version 2.8.5-orangefs starting...
$ cd /tmp
$ cat pvfs2-server.log
[D 02/27/2012 14:34:17] PVFS2 Server version 2.8.5-orangefs starting.
[E 02/27/2012 14:34:17] (null): shmget: key: 2043294211: unable to
create shared system memory region: No space left on device
[E 02/27/2012 14:34:17] dbpf_getdb_env(//tmp/pvfs2-storage/53405cc2/):
No space left on device
[E 02/27/2012 14:34:17] Error initializing trove for filesystem pvfs2-fs
[E 02/27/2012 14:34:17] Error: Could not initialize server subsystems
[E 02/27/2012 14:34:17] Error: Could not initialize server; aborting.
[E 02/27/2012 14:34:17] Trove collection not defined.
$ df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 279G 192M 265G 1% /tmp
$ cat pvfs2.conf
<Defaults>
UnexpectedRequests 50
EventLogging none
EnableTracing no
LogStamp datetime
BMIModules bmi_tcp
FlowModules flowproto_multiqueue
PerfUpdateInterval 1000
ServerJobBMITimeoutSecs 30
ServerJobFlowTimeoutSecs 30
ClientJobBMITimeoutSecs 300
ClientJobFlowTimeoutSecs 300
ClientRetryLimit 5
ClientRetryDelayMilliSecs 2000
PrecreateBatchSize 0,32,512,32,32,32,0
PrecreateLowThreshold 0,16,256,16,16,16,0
DataStorageSpace /tmp/pvfs2-storage
MetadataStorageSpace /tmp/pvfs2-storage
LogFile /tmp/pvfs2-server.log
</Defaults>
<Aliases>
Alias paradent-19.rennes.grid5000.fr
tcp://paradent-19.rennes.grid5000.fr:3334
Alias paradent-2.rennes.grid5000.fr
tcp://paradent-2.rennes.grid5000.fr:3334
</Aliases>
<Filesystem>
Name pvfs2-fs
ID 1396726978
RootHandle 1048576
FileStuffing yes
<MetaHandleRanges>
Range paradent-19.rennes.grid5000.fr 3-2305843009213693953
Range paradent-2.rennes.grid5000.fr
2305843009213693954-4611686018427387904
</MetaHandleRanges>
<DataHandleRanges>
Range paradent-19.rennes.grid5000.fr
4611686018427387905-6917529027641081855
Range paradent-2.rennes.grid5000.fr
6917529027641081856-9223372036854775806
</DataHandleRanges>
<StorageHints>
TroveSyncMeta yes
TroveSyncData no
TroveMethod alt-aio
</StorageHints>
</Filesystem>
This is not a blocking problem for me, but I would prefer to understand
what is going on. Any ideas?
Regards,
Louis
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
