Actually when I use the standard names (parapide-X or parapide-X.rennes.grid5000.fr), the kernel module uses the eth0 interface, so I switched again to using the names ending with "-ib0". I attach the configuration file.
Matthieu 2011/3/23 Matthieu Dorier <[email protected]> > Ok actually my nodes are connected to 2 switches: one 1G Eth and one IB, so > the names "parapide-X-ib0" were to be used for IP over IB (the real > hostnames being "parapide-X", or even "parapide-X.rennes.grid5000.fr", > which is the output of the hostname command). I realized it was useless > working with the names ending in "-ib0" because I'm not using IP over IB in > this case. > > So I deployed again PVFS using the real hostnames, but I still get the same > errors (and again, the kernel module works fine), the configuration line is > now: > /usr/bin/pvfs2-genconfig --quiet --protocol ib --ioservers "parapide-5" > --metaservers "parapide-5" --storage /tmp/pvfs2-storage-space --logfile > /tmp/pvfs2-log --fsname pvfs2 pvfs2-fs.conf > And the configuration file is attached. > > I also changed the limits.conf as advised by Kevin Harms, by adding the > following line: > * hard memlock unlimited > But it does change the warning (when I'm logged as user the ulimit -l > unlimited says I don't have the permissions). Also tried > * hard memlock 1073741824 > Doesn't work either. > > > Matthieu > > > 2011/3/22 Becky Ligon <[email protected]> > >> The kernel module is not required for MPI. >> >> Becky >> -- >> Becky Ligon >> HPC Admin Staff >> PVFS Developer >> Clemson University >> 864-650-4065 >> >> > Is make kmod_install also required? >> > >> > -crispy >> > >> > >> > On 03/22/2011 03:04 PM, Matthieu Dorier wrote: >> >> If you mean the command line for pvfs2-genconfig, here it is: >> >> >> >> /usr/bin/pvfs2-genconfig --quiet --protocol ib --ioservers >> >> "parapide-7-ib0" >> >> --metaservers "parapide-7-ib0" --storage /tmp/pvfs2-storage-space >> >> --logfile >> >> /tmp/pvfs2-log --fsname pvfs2 pvfs2-fs.conf >> >> >> >> I'm basically following the "Quick start guide". >> >> >> >> If you mean the configuration before compiling, here it is: >> >> >> >> ./configure --prefix=/usr >> >> --with-kernel=/lib/modules/2.6.26-2-amd64/build >> >> --with-openib=/usr --without-bmi-tcp >> >> (then make, make kmod, make install) >> >> >> >> Matthieu >> >> >> >> 2011/3/22 Becky Ligon<[email protected]> >> >> >> >>> Can you give us the full command line when you configure PVFS? Seems >> >>> that >> >>> you are doing this in two steps and must be done in one. >> >>> >> >>> Becky >> >>> -- >> >>> Becky Ligon >> >>> HPC Admin Staff >> >>> PVFS Developer >> >>> Clemson University >> >>> 864-650-4065 >> >>> >> >>>> Hello, >> >>>> >> >>>> I have installed PVFS (version 2.8.2) with InfiniBand support on a >> set >> >>>> of >> >>>> nodes running Debian (linux 2.6). The configuration line being the >> >>>> following: >> >>>> ./configure --with-openib=/usr --without-bmi-tcp --prefix=/usr >> >>>> The filesystem works fine with the kernel module, I'm able to write >> >>>> and >> >>>> read >> >>>> to/from the filesystem. >> >>>> Then I compiled Mpich2 (1.2.1p1) with PVFS support using >> >>>> --enable-romio >> >>>> --with-pvfs2=/usr (among other options which doesn't matter). >> >>>> Running the IOR benchmark with POSIX, it works fine, but when using >> >>>> MPIIO >> >>>> I >> >>>> got the following errors (this is from process 0, but all the >> >>>> processes >> >>>> are >> >>>> printing the same) >> >>>> >> >>>> 0: libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. >> >>>> 0: This will severely limit memory registrations. >> >>>> 0: [E 14:12:05.314874] Warning: openib_ib_initialize: >> ibv_open_device. >> >>>> 0: [E 14:12:05.314908] PVFS_isys_fs_add: Failed to initialize any >> >>>> appropriate BMI methods for addresses: >> >>>> 0: [E 14:12:05.314921] ib:// >> parapide-7-ib0.rennes.grid5000.fr:3335 >> >>>> 0: [E 14:12:05.314952] PVFS_isys_fs_add call: No such device >> >>>> 0: [E 14:12:05.314969] ERROR: could not initialize any file systems >> in >> >>>> /etc/mtab. >> >>>> 0: PVFS_util_init_defaults: No such device (error class: 0) >> >>>> >> >>>> Do you have any idea where it comes from? I'm unsuccefully trying to >> >>>> reinstall OFED-1.5, recompiling PVFS, Mpich, but I always get the >> same >> >>>> thing. >> >>>> Note: previsouly everything worked well with Ethernet, I was able to >> >>>> have >> >>>> kernel module AND MPI-IO working. >> >>>> >> >>>> Thank you, >> >>>> >> >>>> Regards >> >>>> >> >>>> Matthieu Dorier >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Matthieu Dorier >> >>>> ENS Cachan, Brittany (Computer Science dpt.) >> >>>> IRISA Rennes, Office E324 >> >>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/ >> >>>> _______________________________________________ >> >>>> Pvfs2-users mailing list >> >>>> [email protected] >> >>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> >>>> >> >>> >> >>> >> >>> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> Pvfs2-users mailing list >> >> [email protected] >> >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > >> >> >> > > > -- > Matthieu Dorier > ENS Cachan, Brittany (Computer Science dpt.) > IRISA Rennes, Office E324 > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/ > -- Matthieu Dorier ENS Cachan, Brittany (Computer Science dpt.) IRISA Rennes, Office E324 http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
pvfs2-fs.conf
Description: Binary data
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
