Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Juan José del Río
On Tue, 2008-08-19 at 18:25 -0700, John Caruso wrote: On Tuesday 05:59 PM 8/19/2008, Juan José del Río wrote: If you don't want to deactivate it, and have some C skills, I would recommend you to make the needed changes to fastpath code to enable it to use the kernel facilities of the operating

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Titi Alailima
Could someone document ns_returnfp while we're talking about it? Titi Ala'ilima Lead Architect MedTouch LLC 1100 Massachusetts Avenue Cambridge, MA 02138 617.621.8670 x309 -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Jim Davidson Sent:

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Gustaf Neumann
Titi Alailima schrieb: This sounds like the problem. Not a bug with fastpath, Come on folks, the discussion wether or the behavior shown by John Caruso is a bug or not is completely fruitless. Most aolserver users are not novices, and if it takes some of us several weeks to find, what

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread John Caruso
On Wednesday 08:45 AM 8/20/2008, Jim Davidson wrote: Overall, it seems one thing to do would be to switch to filename-based cache keys by default, leaving the dev/inode pair as an option for folks who run sites with large symlinks and want to benefit from caching objects just once. I think that

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Derek
On Aug 20, 2008, at 11:54 AM, John Caruso wrote: Actually that wouldn't have fixed the problem in the code that led us to find out about this in the first place. The change that I suggested does fix that problem, though, and it directly addresses the limitation of mtime's one-second

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Jeff Rogers
John Caruso wrote: BTW, Jeff, the scenario you'd outlined that you thought would trip this up...: 13:50:21 - create file 13:50:21 - serve file (gets cached) 13:50:21 - delete file 13:50:21 - create file again (reuses inode) ... time passes ... 13:55:11 - serve file

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Jim Davidson
Hmm... I may be confused and have to re-read all the past messages (except I've deleted them) but my understanding was it was rapid re- use of inodes within the 1-second resolution of mtime and the same file size, all confusing the cache code that caused the problem. With the fix below with

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Jeff Rogers
Jim Davidson wrote: BTW: Which OS is re-using inodes so quickly? I can't get my Mac OS/X laptop to do that -- figured the inode re-use/prediction thing was plugged years ago, e.g., when fsirand was introduced for scrambling NFS vnodes. Linux. This tcl page: set fn /tmp/tmpfile[expr

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread John Caruso
On Wednesday 10:58 AM 8/20/2008, Jim Davidson wrote: With the fix below with resolution for mtime at 1 second and the grace period at 2 seconds I can see how it would work but it would make me a bit queasy -- fixes which have assumptions of timing can be fragile. Perhaps, but not in this

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread John Caruso
By the way, Jeff, regarding this...: On Wednesday 10:56 AM 8/20/2008, Jeff Rogers wrote: And it still hurts the optimization of using a 404 adp page to generate a heavyweight file only once that gets cached. ...which you'd explained elsewhere as...: There is also at least one clever

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Derek
On Aug 20, 2008, at 1:29 PM, John Caruso wrote: Whether or not that's so, the fact is that everyone on this list appeared to share the same utterly natural assumption that ns_returnfile X really will return file X--which turns out to be untrue solely because of fastpath caching's design

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Tom Jackson
John, The patch below is not at all unreasonable as far as your stated goal of not caching a newly modified file. BTW, I think that fastpath is not enabled by default, your config has to have the fastpath config section with the cache parameter set to true. Also, ns_returnfile is really just an

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Eric Larkin
On 8/20/08 11:29 AM, John Caruso [EMAIL PROTECTED] wrote: Whether or not that's so, the fact is that everyone on this list appeared to share the same utterly natural assumption that ns_returnfile X really will return file X All, I've been on vacation or I would have chimed in earlier, but as

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Jeff Rogers
John Caruso wrote: No, you're still not understanding how the patch works. Ok, I'll admit that I misread it at first, but you're also not understanding why I'm saying why it will still break. I'm surprised you're taking such an all-or-nothing view now, I don't think I'm taking an

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Jeff Rogers
Derek wrote: how is this thread still alive. I think this bikeshed should be painted blue. See http://www.bikeshed.com/ if you don't understand this. -J -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Tom Jackson
On Wed, 2008-08-20 at 12:23 -0700, Eric Larkin wrote: On 8/20/08 11:29 AM, John Caruso [EMAIL PROTECTED] wrote: Whether or not that's so, the fact is that everyone on this list appeared to share the same utterly natural assumption that ns_returnfile X really will return file X All, I've

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread John Caruso
On Wednesday 12:30 PM 8/20/2008, Jeff Rogers wrote: I can very easily come up with a scenario that breaks your patched fastpath just as easily as the original, to which you can rightly say, but why would you do it that way?. And you would be right. Do it, then. This is the simplest example

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Jim Davidson
Hi Folks, I agree with Eric, even though I wrote the original code and was one of the first to suggest is wasn't a bug. This thread has surprised me in a few ways: -- The bug was indeed subtle and curious -- The debate on dynamic vs. static and underlying assumptions and performance was

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Juan José del Río
Hello John, I just think that you're introducing complexity to a common and widely used infraestructure (with a hack), just to hide the flaws on your side. I don't like this hack. It's not the correct thing to do, not even if you make some parameters (like the timeout) changeable via the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread John Caruso
On Wednesday 01:45 PM 8/20/2008, Juan José del Río wrote: But, since Linux version 2.6.13, inotify is into the kernel, and aolserver can subscribe to a path, so know if that file has been deleted, modified, or anything else. That's the way a cache can know if the file has been altered in any

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Michael A. Cleverly
On Wed, Aug 20, 2008 at 12:22 PM, Jeff Rogers [EMAIL PROTECTED] wrote: Linux. This tcl page: set fn /tmp/tmpfile[expr rand()] set f [open $fn w] puts $f [ns_queryget data] close $f after 2 ns_returnfile 200 text/plain $fn ns_unlink $fn [after 2] would wait 2 milliseconds. [after 2000]

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Juan José del Río
Hello again John, Oops, sorry. I thought RedHat Linux had everything an Enterprise needed ;-) (Take the above line with a lot of salt. I don't want to discuss about linux distros... yet ;-) Then what about adding a -nocache parameter to the function? That way you won't modify the original

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread Tom Jackson
John, Your last patch suggestion seems good, not caching something that looks like a new file is fully in line with the intent of fastpath and ns_returnfile. I'm not sure everyone is commenting on this new patch idea, maybe a previous idea? Anyway if the cache is for serving static (and

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-20 Thread russell muetzelfeldt
On 21/08/2008, at 1:45 AM, Jim Davidson wrote: I looked at the code a bit closer. The ns_returnfile and ns_respond commands both call Ns_ConnReturnFile, the public API to the underlying FastPath. It does more than just blast the content -- it handles: ... -- caches, mmap's, or