I am having trouble installing pvfs2 2.8.0 to use IB RDMA under centos 5.3
with 2.6.18-128, and hope that someone on the mailing list can offer some help.
1. The build: I ran ./configure --with-openib=/usr --without-bmi-tcp
--with-kernel=/usr/src/kernels/2.6.18-128.el5-x86_64.
2. I configured the meta and I/O server on a single node (dn2). And here is
the content of my /etc/pvfs2-fs.conf:
<Defaults>
UnexpectedRequests 50
EventLogging none
EnableTracing no.
LogStamp datetime
BMIModules bmi_ib
FlowModules flowproto_multiqueue
PerfUpdateInterval 1000
ServerJobBMITimeoutSecs 30
ServerJobFlowTimeoutSecs 30
ClientJobBMITimeoutSecs 300
ClientJobFlowTimeoutSecs 300
ClientRetryLimit 5
ClientRetryDelayMilliSecs 2000
PrecreateBatchSize 512
PrecreateLowThreshold 256
StorageSpace /pvfs2-storage-space
LogFile /tmp/pvfs2-server.log
</Defaults>
<Aliases>
Alias dn2-ib ib://dn2-ib:3335
</Aliases>
<Filesystem>
Name pvfs2-fs
ID 695113894
RootHandle 1048576
FileStuffing yes
<MetaHandleRanges>
Range dn2-ib 3-4611686018427387904
</MetaHandleRanges>
<DataHandleRanges>
Range dn2-ib 4611686018427387905-9223372036854775806
</DataHandleRanges>
<StorageHints>
TroveSyncMeta yes
TroveSyncData no
TroveMethod alt-aio
</StorageHints>
</Filesystem>
3. I built an xfs filesystem on /dev/md0 and mounted it on /pvfs2.
4. I then started the server with "/usr/local/sbin/pvfs2-server
/etc/pvfs2-fs.conf -f " for the 1st time and without "-f" subsequent times.
4. On a second node (dn3, the pvfs client) I created /etc/pvfs2tab with the
following line:
ib://dn2-ib:3335/pvfs2 /mnt/pvfs2 pvfs2 defaults,noauto 0 0
5. Pvfs2-ping -m /mnt/pvfs2 failed. Unfortunately I did'nt capture the erroe
message in time before the nodes were grabbed by my colleague. I remembered
vaguely that it complained about the exported pvfs2 filesystem not initialized.
6. I verified that the IB RDMA network worked using rping.
Please let me know if I have done something wrong and whether you need more
info to troubleshoot. Your help will be appreciated.
[email protected]
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
