Maybe the memory accounting is just handled differently in RHEL5?
-Phil
David Metheny wrote:
I'm running the same types of tests I did with a RHEL3 and RHEL4 system. But
not seeing the same results on the RHEL5 systems. So... I'm not sure if it
really is a problem or not.
Watching 'top' or using 'ps' I can see the VIRT and RSS sizes pretty much
match on initial start for pvfs2-client-core. With pvfs2 idle, I'm running
some external program that eats up the available memory, using more than our
physical RAM.
On RHEL3/RHEL4 with the patch, the RSS sizes shows that memory gets swapped
out a bit, but leaves around 40MB which matches our size for I/O Buffers
(--desc-count 5 --desc-size 8388608).
Without the patch, on RHEL3, the RSS size would get really low, around 1K or
so, and when PVFS2 was used again, it would start putting it back in memory
and the RSS size would grow, however we would see that the data in the I/O
transfers at that point was wrong.
For the RHEL5 systems, I'm able to use the external program to eat up
memory, and see the RSS (shows up as RES in top) size get really low, but
I'm not able to push it back into memory, and can't get the data corruption
issue to trigger.
-----Original Message-----
From: Rob Ross [mailto:[EMAIL PROTECTED]
Sent: Friday, November 21, 2008 11:38 AM
To: [EMAIL PROTECTED]
Cc: [email protected]
Subject: Re: [Pvfs2-developers] Re: patch for client data corruption with
2.4kernels
is there any way to do a quick verification that the buffer is still
in the right place, on the kernel side, and then force a remapping?
rob
On Nov 21, 2008, at 11:19 AM, David Metheny wrote:
I know this fixes RHEL3 (2.4 kernels) and the RHEL4 U6/U7 (2.6.9
kernels), but testing this with RHEL5 U2 (2.6.18-92.el5PAE) shows
that the memory is still being swapped out which would result in
client data corruption.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
] On Behalf Of Bart Taylor
Sent: Thursday, November 20, 2008 4:32 PM
To: [email protected]
Subject: [Pvfs2-developers] Re: patch for client data corruption
with 2.4kernels
I neglected the header file; attached is an additional patch to
catch it. Both of these patches should apply cleanly to head.
Bart.
On Thu, Nov 20, 2008 at 1:54 PM, Bart Taylor <[EMAIL PROTECTED]> wrote:
Hey guys,
This patch fixes a client data corruption issue on Linux 2.4
systems. We worked with Phil off-list and determined that the
kernel's mapped buffers are not staying pinned in memory as they
should be. We were able to verify that under heavy swapping the
pvfs2-client-core memory was being moved, invalidating the pointers.
This fix uses mlock to hold the two buffers in place, and we were
able to verify that it fixes the swapping issue. There are compile
time flags to ensure it is only added for Linux 2.4 systems.
Bart.
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
