Re: [Pvfs2-users] PVFS2 Client on ppc64 problem?

[Andrew Pochinsky]

Here's pvfs2-statfs output. There is, of course, a chance of a hardware problem, but the fact that pvfs2-cp works ten times faster than cp suggests otherwise, in my opinion. I have rebooted both servers and the client, and after about 15 hours I again see Badness... messages in dmesg and the system log. There is nothing new in the pvfs2 client log file.

--andrew

statfs
Description: Binary data

On Feb 15, 2006, at 4:50 PM, Robert Latham wrote:

On Tue, Feb 14, 2006 at 05:53:50PM -0500, Andrew Pochinsky wrote:

Either before that, or after that, pvfs2-client.log contains these
lines:

[E 23:50:20.804738] Object Type mismatch error: Bad file descriptor

[E 23:50:20.842488] pvfs2-client-core: caught signal 11
[E 23:51:20.800850] Object Type mismatch error: Bad file descriptor

[E 23:51:20.801054] pvfs2-client-core: caught signal 11
[E 23:52:20.800206] Object Type mismatch error: Bad file descriptor

[E 23:52:20.800382] pvfs2-client-core: caught signal 11
[E 23:53:20.800813] Object Type mismatch error: Bad file descriptor

[E 23:53:20.800992] pvfs2-client-core: caught signal 11
[E 23:54:20.800199] Object Type mismatch error: Bad file descriptor

[E 23:54:20.800379] pvfs2-client-core: caught signal 11

[E 23:50:20.842452] getattr_object_getattr_failure : Bad file descriptor [E 23:51:20.801025] getattr_object_getattr_failure : Bad file descriptor [E 23:52:20.800354] getattr_object_getattr_failure : Bad file descriptor [E 23:53:20.800966] getattr_object_getattr_failure : Bad file descriptor [E 23:54:20.800353] getattr_object_getattr_failure : Bad file descriptor

That's really odd: every 60 seconds between 23:50 and 23:54 (and only
for those 4 minutes), pvfs2-client-core caught a seg fault.

We don't get many reports of degraded performance (as opposed
to an outright crash), so we're going to have to do a little
exploring.

What does pvfs2-statfs say? (maybe one server is hitting swap or ran
out of space or ran out of handles)

Sometimes we get PVFS2 bug reports that turn out to be hardware
problems.  Are you confident your memory has no errors and that your
switches are all properly configured and didn't go into maintenance
mode (that happened to us once).

==rob

--
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B
_______________________________________________
Pvfs2-users mailing list
Pvfs2-users@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

_______________________________________________
Pvfs2-users mailing list
Pvfs2-users@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users