I'm running the same types of tests I did with a RHEL3 and RHEL4 system. But not seeing the same results on the RHEL5 systems. So... I'm not sure if it really is a problem or not.
Watching 'top' or using 'ps' I can see the VIRT and RSS sizes pretty much match on initial start for pvfs2-client-core. With pvfs2 idle, I'm running some external program that eats up the available memory, using more than our physical RAM. On RHEL3/RHEL4 with the patch, the RSS sizes shows that memory gets swapped out a bit, but leaves around 40MB which matches our size for I/O Buffers (--desc-count 5 --desc-size 8388608). Without the patch, on RHEL3, the RSS size would get really low, around 1K or so, and when PVFS2 was used again, it would start putting it back in memory and the RSS size would grow, however we would see that the data in the I/O transfers at that point was wrong. For the RHEL5 systems, I'm able to use the external program to eat up memory, and see the RSS (shows up as RES in top) size get really low, but I'm not able to push it back into memory, and can't get the data corruption issue to trigger. -----Original Message----- From: Rob Ross [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 11:38 AM To: [EMAIL PROTECTED] Cc: [email protected] Subject: Re: [Pvfs2-developers] Re: patch for client data corruption with 2.4kernels is there any way to do a quick verification that the buffer is still in the right place, on the kernel side, and then force a remapping? rob On Nov 21, 2008, at 11:19 AM, David Metheny wrote: > I know this fixes RHEL3 (2.4 kernels) and the RHEL4 U6/U7 (2.6.9 > kernels), but testing this with RHEL5 U2 (2.6.18-92.el5PAE) shows > that the memory is still being swapped out which would result in > client data corruption. > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > ] On Behalf Of Bart Taylor > Sent: Thursday, November 20, 2008 4:32 PM > To: [email protected] > Subject: [Pvfs2-developers] Re: patch for client data corruption > with 2.4kernels > > I neglected the header file; attached is an additional patch to > catch it. Both of these patches should apply cleanly to head. > > Bart. > > On Thu, Nov 20, 2008 at 1:54 PM, Bart Taylor <[EMAIL PROTECTED]> wrote: > Hey guys, > > This patch fixes a client data corruption issue on Linux 2.4 > systems. We worked with Phil off-list and determined that the > kernel's mapped buffers are not staying pinned in memory as they > should be. We were able to verify that under heavy swapping the > pvfs2-client-core memory was being moved, invalidating the pointers. > This fix uses mlock to hold the two buffers in place, and we were > able to verify that it fixes the swapping issue. There are compile > time flags to ensure it is only added for Linux 2.4 systems. > > Bart. > > _______________________________________________ > Pvfs2-developers mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
