HI Matthieu - The last time I worked on this we ran into this problem and I think we narrowed it down to a mopid reuse issue, we tried to insert some thread locking mechanisms into the mopid 'cache' but I dont think it ever got resolved. This was years ago and only occurred under very heavy load of relatively small messages.
That would be the place to start I would imagine. Cheers, Kyle Schochenmaier On Sat, Mar 26, 2011 at 4:21 PM, Matthieu Dorier < [email protected]> wrote: > Hello, > > I'm trying to evaluate the performance of my PVFS installation over an > InfiniBand network, but from time to time a server crashes with this trace > in the log: > > [E 03/26 21:58] Error: encourage_recv_incoming: mop_id 12952a0 in RTS_DONE > message not found. > [E 03/26 21:58] [bt] /usr/sbin/pvfs2-server(error+0xca) [0x46f55a] > [E 03/26 21:58] [bt] /usr/sbin/pvfs2-server [0x46c88c] > [E 03/26 21:58] [bt] /usr/sbin/pvfs2-server [0x46e485] > [E 03/26 21:58] [bt] /usr/sbin/pvfs2-server(BMI_testunexpected+0x384) > [0x421004] > [E 03/26 21:58] [bt] /usr/sbin/pvfs2-server [0x41cf4a] > [E 03/26 21:58] [bt] /lib/libpthread.so.0 [0x7f6422ff0fc7] > [E 03/26 21:58] [bt] /lib/libc.so.6(clone+0x6d) [0x7f642295164d] > > I've seen that some other users reported this kind of error in some > archives of the mailing list, but didn't find any answer to solve the > problem. Any idea how to solve this problem? > > If it can be of any use: I'm working with 16 PVFS servers (IO server and > metadata server at the same time), and I'm benchmarking with the IOR > program, for now I have 648 processes writing 8MB each in a shared file with > a transfer size that corresponds to the strip size (64KB). > > Thank you, > > Matthieu > > -- > Matthieu Dorier > ENS Cachan, Brittany (Computer Science dpt.) > IRISA Rennes, Office E324 > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/ > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
