RE: how to flush out cache.?

2004-04-24 Thread Jan Mikkelsen
Julian Elischer wrote: > Other than reading a few GB of data, is there a way to flush > out the cache copy of a file I've written? I don't know how this will fit into your application, but unmounting and remounting the filesystem is a way that springs to mind. Perhaps not as isloated as you'd lik

Re: Re99: how to flush out cache.?

2004-04-23 Thread Matthew Dillon
:When I get the time (probably not next week) I will write a patch :to release the cached buffers that would prevent page removal. I would appreciate a CC if/when you have something along these lines. It won't be easy. The VM system has no reliable way to determine the buffer cache bl

Re: Re99: how to flush out cache.?

2004-04-23 Thread Stephan Uphoff
Julian Elischer wrote: > with this bug could a user zero out /etc/group or similar? > I am not sure what the ramification of the bug is.. The bug affects only in memory modified file data. In memory modifications to the file can be deleted and the file data reverts to a state before the file mod

Re: Re99: how to flush out cache.?

2004-04-23 Thread Julian Elischer
with this bug could a user zero out /etc/group or similar? I am not sure what the ramification of the bug is.. On Fri, 23 Apr 2004, Stephan Uphoff wrote: > > Matthew Dillon wrote: > > >The idea with the fix is as follows: > > > >* It's hard to make the VM system invalidate buffers, s

Re: Re99: how to flush out cache.?

2004-04-23 Thread Stephan Uphoff
Matthew Dillon wrote: >The idea with the fix is as follows: > >* It's hard to make the VM system invalidate buffers, so don't try. > But it is easy to throw away clean buffers since they are nicely sorted and easy to release. Efficient locking is a bit tricky but should not be

Re: how to flush out cache.?

2004-04-23 Thread Julian Elischer
On Thu, 22 Apr 2004, Matthew Dillon wrote: > Sigh. Run this program. Note that the file contains an 'A' in the > first byte after you run it (hexdump -C test.dat). Thus, msync() > is not destroying the page until AFTER it finishes flushing it to > disk. > > /* > * x.c > */

Re99: how to flush out cache.?

2004-04-22 Thread Matthew Dillon
:(taking hackers off to reduce noise) oops. I guess I didn't take it off hackers. Sorry about that. Yah yah, I know, 'Matt's making a billion postings again'. In anycase, I sent a patch set to Stephan and julian, and since I didn't take it off the list like I said I would I gu

Re: how to flush out cache.?

2004-04-22 Thread Matthew Dillon
(taking hackers off to reduce noise) I think all that needs to happen to fix this problem is to call vm_object_page_remove() with the clean_only flag set to TRUE instead of FALSE. Wired pages, e.g. pages in the buffer cache, will not be removed, but that's ok because memor

Re: how to flush out cache.?

2004-04-22 Thread Matthew Dillon
Ok... I've done some more testing. Ahh! I finally was able to reproduce the problem with PROT_READ, and I now see the code bit you were talking about. That's definitely a bug. Here's the test program that reproduces the problem. This is definitely a serious bug.

Re: how to flush out cache.?

2004-04-22 Thread Matthew Dillon
Sigh. Run this program. Note that the file contains an 'A' in the first byte after you run it (hexdump -C test.dat). Thus, msync() is not destroying the page until AFTER it finishes flushing it to disk. /* * x.c */ #include #include #include #include #include int main(in

Re: how to flush out cache.?

2004-04-22 Thread Stephan Uphoff
Matthew Dillon wrote: > :Yes - but FreeBSD then calls vm_object_page_remove to remove the pages > :from the vnode object. (vm_object_sync for 5.x or vm_map_clean for 4.x ) > < ... SNIP ...> > > I don't quite see that. Could you point out the code in question? > (And, of course, a test p

Re: how to flush out cache.?

2004-04-21 Thread Matthew Dillon
:Yes - but FreeBSD then calls vm_object_page_remove to remove the pages :from the vnode object. (vm_object_sync for 5.x or vm_map_clean for 4.x ) : :Mhhh .. 4.X removes the pages without flushing if the file is not write :mapped. :This means that anyone with read access can screw up a file. :(

Re: how to flush out cache.?

2004-04-21 Thread Stephan Uphoff
Stephan Uphoff wrote: > Matthew Dillon wrote: > > : > > :> > > :> mmap() and msync(..MS_INVALIDATE..) should work. > > : > > :hmmm that is rather interesting.. > > :I wonder if it would work > > :Maybe a vm guru could confirm this.. (under 4.x) > > : > > > > Huh. If I hadn't looked at

Re: how to flush out cache.?

2004-04-21 Thread Stephan Uphoff
Matthew Dillon wrote: > : > :> > :> mmap() and msync(..MS_INVALIDATE..) should work. > : > :hmmm that is rather interesting.. > :I wonder if it would work > :Maybe a vm guru could confirm this.. (under 4.x) > : > > Huh. If I hadn't looked at the code I would have said that > MS_INVA

Re: how to flush out cache.?

2004-04-21 Thread Matthew Dillon
:Actually what I'm looking for is :"throw away clean data" : :I want to dump the cached version of a file so that I can force a reread :of the disk. MS_INVALIDATE doesn't do that. madvise()'s MADV_FREE does what you want, BUT it does not currently work (at least on 4.x or in DFly

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Matthew Dillon wrote: > : > :> > :> mmap() and msync(..MS_INVALIDATE..) should work. > : > :hmmm that is rather interesting.. > :I wonder if it would work > :Maybe a vm guru could confirm this.. (under 4.x) > : > > Huh. If I hadn't looked at the code I would have

Re: how to flush out cache.?

2004-04-21 Thread Matthew Dillon
: :> :> mmap() and msync(..MS_INVALIDATE..) should work. : :hmmm that is rather interesting.. :I wonder if it would work :Maybe a vm guru could confirm this.. (under 4.x) : Huh. If I hadn't looked at the code I would have said that MS_INVALIDATE doesn't work in FreeBSD, but when I lo

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Stephan Uphoff wrote: > > mmap() and msync(..MS_INVALIDATE..) should work. hmmm that is rather interesting.. I wonder if it would work Maybe a vm guru could confirm this.. (under 4.x) > > Stephan > > > > > Ok so I have an application where I need to > > rere

Re: how to flush out cache.?

2004-04-21 Thread Stephan Uphoff
mmap() and msync(..MS_INVALIDATE..) should work. Stephan > > Ok so I have an application where I need to > reread a file I have just written to ensure that it went to disk > correctly.. > > Other than reading a few GB of data, is there a way to flush > out the cache copy of a file I'v

Re: how to flush out cache.?

2004-04-21 Thread David Malone
On Wed, Apr 21, 2004 at 01:47:43PM -0700, Julian Elischer wrote: > ingenious, but, no we can't do that.. > As I mentionned.. someone suggested using 'dump' > to extract the file from raw disk and checksumming that... :-) If it is SCSI, you could probably put a second controler on the SCSI bus, and

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Garance A Drosihn wrote: > At 12:23 PM -0700 4/21/04, Julian Elischer wrote: > >Ok so I have an application where I need to > >reread a file I have just written to ensure that it went to disk > >correctly.. > > > >Other than reading a few GB of data, is there a way to flush

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Steve Watt wrote: > On Apr 21, 13:28, Julian Elischer wrote: > } Subject: Re: how to flush out cache.? > } > } On Wed, 21 Apr 2004, Steve Watt wrote: > } > } > In article <[EMAIL PROTECTED]> you write: > } > > > } > &g

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Steve Watt wrote: > On Apr 21, 13:28, Julian Elischer wrote: > } Subject: Re: how to flush out cache.? > } > } On Wed, 21 Apr 2004, Steve Watt wrote: > } > } > In article <[EMAIL PROTECTED]> you write: > } > > > } > &g

Re: how to flush out cache.?

2004-04-21 Thread Garance A Drosihn
At 12:23 PM -0700 4/21/04, Julian Elischer wrote: Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include:

Re: how to flush out cache.?

2004-04-21 Thread Steve Watt
On Apr 21, 13:28, Julian Elischer wrote: } Subject: Re: how to flush out cache.? } } On Wed, 21 Apr 2004, Steve Watt wrote: } } > In article <[EMAIL PROTECTED]> you write: } > > } > >Ok so I have an application where I need to } > >reread a file I have just writte

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Julian Elischer wrote: > > > On Wed, 21 Apr 2004, Steve Watt wrote: > > > In article <[EMAIL PROTECTED]> you write: > > > > > >Ok so I have an application where I need to > > >reread a file I have just written to ensure that it went to disk > > >correctly.. > > > > > >Ot

Re: how to flush out cache.?

2004-04-21 Thread Julian Elischer
On Wed, 21 Apr 2004, Steve Watt wrote: > In article <[EMAIL PROTECTED]> you write: > > > >Ok so I have an application where I need to > >reread a file I have just written to ensure that it went to disk > >correctly.. > > > >Other than reading a few GB of data, is there a way to flush > >out the

Re: how to flush out cache.?

2004-04-21 Thread Steve Watt
In article <[EMAIL PROTECTED]> you write: > >Ok so I have an application where I need to >reread a file I have just written to ensure that it went to disk >correctly.. > >Other than reading a few GB of data, is there a way to flush >out the cache copy of a file I've written? You can flush the fil

how to flush out cache.?

2004-04-21 Thread Julian Elischer
Ok so I have an application where I need to reread a file I have just written to ensure that it went to disk correctly.. Other than reading a few GB of data, is there a way to flush out the cache copy of a file I've written? possibilities include: a file flag saying "don't keep a copy after it