Thanks Becky! .. It looks like IBV_EVENT_GID_ has been around since about 2006, but we don't actually use it except for reporting WHEN that GID change actually happens, if your library doesnt have it then in theory it wont come do it...so it may be safe to remove that line just to test long enough to see if the mopid errors go away as expected.
If you can try that out on your test-bed, I can try to come up with a more permanent solution for excluding those calls. It goes without saying that I'd also recommend updating your ofed distro so both server and client are running on the same libraries for maximum stability if possible, though I do understand that there is a bit of effort involved in that. Cheers, Kyle Schochenmaier On Thu, Jul 19, 2012 at 10:39 AM, Becky Ligon <[email protected]> wrote: > Thanks, Kyle, for helping out with this problem. Just wanted to throw out > that 2.8.6 DOES correct the original problem with IB, unless a variant of > this problem has been uncovered. > > Becky > > > On Thu, Jul 19, 2012 at 11:28 AM, Kyle Schochenmaier <[email protected]> > wrote: >> >> Which version of OFED are you using? >> >> Kyle Schochenmaier >> >> >> On Thu, Jul 19, 2012 at 10:27 AM, Yves Revaz <[email protected]> wrote: >> > ooops !, >> > >> > recompiling 2.8.6 on a client machine, I get: >> > >> > >> > In file included from ./src/common/misc/pint-event.h:11, >> > from ./src/io/bmi/bmi-method-support.h:16, >> > from src/io/bmi/bmi_ib/ib.c:21: >> > ./src/common/quickhash/quickhash.h: In function 'qhash_add': >> > ./src/common/quickhash/quickhash.h:133: warning: declaration of 'link' >> > shadows a global declaration >> > /usr/include/unistd.h:757: warning: shadowed declaration is here >> > ./src/common/quickhash/quickhash.h:135: warning: declaration of 'index' >> > shadows a global declaration >> > /usr/include/string.h:304: warning: shadowed declaration is here >> > ./src/common/quickhash/quickhash.h: In function 'qhash_search': >> > ./src/common/quickhash/quickhash.h:160: warning: declaration of 'index' >> > shadows a global declaration >> > /usr/include/string.h:304: warning: shadowed declaration is here >> > ./src/common/quickhash/quickhash.h: In function 'qhash_search_at_index': >> > ./src/common/quickhash/quickhash.h:190: warning: declaration of 'index' >> > shadows a global declaration >> > /usr/include/string.h:304: warning: shadowed declaration is here >> > ./src/common/quickhash/quickhash.h: In function >> > 'qhash_search_and_remove': >> > ./src/common/quickhash/quickhash.h:221: warning: declaration of 'index' >> > shadows a global declaration >> > /usr/include/string.h:304: warning: shadowed declaration is here >> > ./src/common/quickhash/quickhash.h: In function >> > 'qhash_search_and_remove_at_index': >> > ./src/common/quickhash/quickhash.h:252: warning: declaration of 'index' >> > shadows a global declaration >> > /usr/include/string.h:304: warning: shadowed declaration is here >> > CC src/io/bmi/bmi_ib/util-server.o >> > CC src/io/bmi/bmi_ib/mem-server.o >> > CC src/io/bmi/bmi_ib/openib-server.o >> > src/io/bmi/bmi_ib/openib.c: In function 'async_event_type_string': >> > src/io/bmi/bmi_ib/openib.c:748: error: 'IBV_EVENT_GID_CHANGE' undeclared >> > (first use in this function) >> > src/io/bmi/bmi_ib/openib.c:748: error: (Each undeclared identifier is >> > reported only once >> > src/io/bmi/bmi_ib/openib.c:748: error: for each function it appears in.) >> > make: *** [src/io/bmi/bmi_ib/openib-server.o] Error 1 >> > >> > >> > There was no errors with 2.8.5 ! >> > >> > I used : >> > >> > ./configure --prefix /cvos/shared/apps/orangefs-2.8.6 --with-openib=/usr >> > --without-bmi-tcp >> > -with-kernel=/usr/src/linux-2.6.22.9-clustervision-192_cvos3.1/ >> > >> > and get: >> > >> > ***** Displaying PVFS Configuration Information ***** >> > ------------------------------------------------------ >> > PVFS2 configured to build karma gui : yes >> > PVFS2 configured to build visualization tools : no >> > PVFS2 configured to perform coverage analysis : no >> > PVFS2 configured for aio threaded callbacks : yes >> > PVFS2 configured to use FUSE : no >> > PVFS2 configured for the 2.6/3 kernel module : yes >> > PVFS2 configured for the 2.4.x kernel module : no >> > PVFS2 configured for using the mmap-ra-cache : no >> > PVFS2 configured for resetting file position : no >> > PVFS2 will use workaround for redhat 2.4 kernels : no >> > PVFS2 will use workaround for buggy NPTL : no >> > PVFS2 server will be built : yes >> > PVFS2 user interface libraries will be built : yes >> > PVFS2 user interface library cache enabled : no >> > configure: WARNING: The detected version of Berkeley DB is not at least >> > 4.8.30. >> > Metadata corruption has been documented in versions >> > prior >> > to 4.8.30. It is highly recommended that you update >> > the >> > version of Berkeley DB you are building against. >> > >> > >> > >> > Something to do ? >> > >> > >> > yves >> > >> > >> > >> > >> > >> > On 07/19/2012 05:17 PM, Kyle Schochenmaier wrote: >> >> >> >> Hi Yves - >> >> >> >> Yes, we shouldn't try to mix client/server versions - iirc I think >> >> there is also a check to prevent this. >> >> >> >> Regards, >> >> Kyle Schochenmaier >> >> >> >> >> >> On Thu, Jul 19, 2012 at 10:10 AM, Yves Revaz<[email protected]> >> >> wrote: >> >>> >> >>> On 07/19/2012 04:28 PM, Kyle Schochenmaier wrote: >> >>>> >> >>>> Hi Yves - >> >>>> >> >>>> I believe this is fixed in orangefs2.8.6. >> >>> >> >>> >> >>> do I need to update also the pvfs2 kernel >> >>> modules for the clients ? >> >>> >> >>>> Regards, >> >>>> Kyle Schochenmaier >> >>>> >> >>>> >> >>>> On Thu, Jul 19, 2012 at 9:25 AM, Yves Revaz<[email protected]> >> >>>> wrote: >> >>>>> >> >>>>> Dear list, >> >>>>> >> >>>>> I get the following error in one of my 6 pvfs server, >> >>>>> while I was copying files using rsync: >> >>>>> >> >>>>> [D 07/19/2012 14:47:25] PVFS2 Server version 2.8.5-orangefs >> >>>>> starting. >> >>>>> [E 07/19/2012 16:13:20] Error: encourage_recv_incoming: mop_id >> >>>>> f58c80 >> >>>>> in >> >>>>> RTS_DONE message not found. >> >>>>> [E 07/19/2012 16:13:20] [bt] >> >>>>> /opt/orangefs-2.8.5/sbin/pvfs2-server(error+0xca) [0x48b62a] >> >>>>> [E 07/19/2012 16:13:20] [bt] >> >>>>> /opt/orangefs-2.8.5/sbin/pvfs2-server() >> >>>>> [0x4889d4] >> >>>>> [E 07/19/2012 16:13:20] [bt] >> >>>>> /opt/orangefs-2.8.5/sbin/pvfs2-server() >> >>>>> [0x48a5a5] >> >>>>> [E 07/19/2012 16:13:20] [bt] >> >>>>> /opt/orangefs-2.8.5/sbin/pvfs2-server(BMI_testunexpected+0x28b) >> >>>>> [0x4787db] >> >>>>> [E 07/19/2012 16:13:20] [bt] >> >>>>> /opt/orangefs-2.8.5/sbin/pvfs2-server() >> >>>>> [0x45bfbe] >> >>>>> [E 07/19/2012 16:13:20] [bt] /lib64/libpthread.so.0() >> >>>>> [0x3ca70077f1] >> >>>>> [E 07/19/2012 16:13:20] [bt] /lib64/libc.so.6(clone+0x6d) >> >>>>> [0x3ca6ce570d] >> >>>>> >> >>>>> >> >>>>> What does this means ? >> >>>>> After this error, the fs was blocked. >> >>>>> I get messages like this: >> >>>>> >> >>>>> rsync: recv_generator: mkdir >> >>>>> "/SCRATCH/./arnaudon/PyTrans/dSph-test-staronly" failed: Connection >> >>>>> refused >> >>>>> (111) >> >>>>> >> >>>>> And I had to restart all servers to get the fs back. >> >>>>> May I change something ? >> >>>>> >> >>>>> >> >>>>> Thanks, >> >>>>> >> >>>>> yves >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> (o o) >> >>>>> --------------------------------------------oOO--(_)--OOo------- >> >>>>> Dr. Yves Revaz >> >>>>> Laboratory of Astrophysics >> >>>>> Ecole Polytechnique Fédérale de Lausanne (EPFL) >> >>>>> Observatoire de Sauverny Tel : ++ 41 22 379 24 28 >> >>>>> 51. Ch. des Maillettes Fax : ++ 41 22 379 22 05 >> >>>>> 1290 Sauverny e-mail : [email protected] >> >>>>> SWITZERLAND Web : http://www.lunix.ch/revaz/ >> >>>>> ---------------------------------------------------------------- >> >>>>> >> >>>>> _______________________________________________ >> >>>>> Pvfs2-users mailing list >> >>>>> [email protected] >> >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> >>> >> >>> >> >>> >> >>> -- >> >>> (o o) >> >>> --------------------------------------------oOO--(_)--OOo------- >> >>> Dr. Yves Revaz >> >>> Laboratory of Astrophysics >> >>> Ecole Polytechnique Fédérale de Lausanne (EPFL) >> >>> Observatoire de Sauverny Tel : ++ 41 22 379 24 28 >> >>> 51. Ch. des Maillettes Fax : ++ 41 22 379 22 05 >> >>> 1290 Sauverny e-mail : [email protected] >> >>> SWITZERLAND Web : http://www.lunix.ch/revaz/ >> >>> ---------------------------------------------------------------- >> >>> >> > >> > >> > -- >> > (o o) >> > --------------------------------------------oOO--(_)--OOo------- >> > Dr. Yves Revaz >> > Laboratory of Astrophysics >> > Ecole Polytechnique Fédérale de Lausanne (EPFL) >> > Observatoire de Sauverny Tel : ++ 41 22 379 24 28 >> > 51. Ch. des Maillettes Fax : ++ 41 22 379 22 05 >> > 1290 Sauverny e-mail : [email protected] >> > SWITZERLAND Web : http://www.lunix.ch/revaz/ >> > ---------------------------------------------------------------- >> > >> >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > -- > Becky Ligon > OrangeFS Support and Development > Omnibond Systems > Anderson, South Carolina > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
