Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Tom Jackson
On Mon, 2008-08-18 at 19:20 -0700, John Caruso wrote: I'd say it's still better, because it requires explicit action on the user's part to enable the flawed caching mechanism in that case. And actually I don't think fastpath in its default configuration would be of much help in

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Andrew Piskorski
On Mon, Aug 18, 2008 at 06:06:23PM -0700, John Caruso wrote: That'd be an improvement over the current situation, but it's still the case that AOLserver as currently shipped has a file cache mechanism built into it which 1) may return incorrect data and 2) is enabled by default. Given the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Titi Alailima
This would be a wonderful addition to the documentation. As a matter of fact, I just added it: http://panoptic.com/wiki/aolserver/Fastpath For what it's worth, it seems to me that if it has a measurable benefit, it's worth leaving on by default, as long as developers are properly educated

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Juan José del Río
I agree with Titi. The vast majority of times, having Fastpath on does not harm at all. Having it disabled by default would be like not using computers because they fail sometimes. That would be too extreme, isn't it? ;-) As long as it's well documented, and there are alternatives to avoid the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Tom Jackson
Andrew, This is not a corner case. The exact same thing could happen without fastpath. What is that thing? That the contents of a file changes after a request is made and before the file is returned. In fact, there is no guarantee that it won't change mid-return. This is a fact of life with

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Juan José del Río
What about using epoll (or equivalent) in Linux, and kqueue in FreeBSD to tell the kernel to notify AOLServer in change a file has changed? That'd be a pretty easy and efficient way to discard fastpath items in case they have been deleted and/or modified. Just my two cents ;-) - Juan José

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Jeff Rogers
Tom Jackson wrote: If you want to avoid use of fastpath, just set the configuration lower than your dynamic content: # # Fastpath # ns_section ns/server/${server}/fastpath ns_param cache[set cache 10] ;# max entries ?? ns_param cachemaxsize [set cachemaxsize [expr 5 *

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Jim Davidson
Hi folks, I wrote the code. The explanation below is correct -- I chose inode/ dev combination to cache the same file even with multiple names which was the case at AOL -- hundreds of symlinks and hard links to the same file. The same strategy is used for ADP templates. I think the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread John Caruso
On Tuesday 10:40 AM 8/19/2008, Jim Davidson wrote: I would suggest the code snippet of create temp file and use fastpath to return contents is not a use case I was solving for or recommend. It's also not the use case in question--just a simple illustration of the problem. Here's a more

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Tom Jackson
John, It is not a bug in ns_returnfile. tom jackson On Tue, 2008-08-19 at 11:52 -0700, John Caruso wrote: On Tuesday 10:40 AM 8/19/2008, Jim Davidson wrote: I would suggest the code snippet of create temp file and use fastpath to return contents is not a use case I was solving for or

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread John Caruso
On Monday 05:53 PM 8/18/2008, Jeff Rogers wrote: russell muetzelfeldt wrote: fastpath is making assumptions about what means something is the same file, and those assumptions are not consistent with unix filesystem semantics - how is this not a bug? It's not a bug because no one ever said

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Jeff Rogers
John Caruso wrote: Think of it as a if-modified-since or if-none-match conditional GET. Actually that's not analogous, ... I didn't mean to say it was exactly the same, just similar in that given a particular system that makes particular assumptions it is possible to construct a

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread John Caruso
On Tuesday 02:10 PM 8/19/2008, Jeff Rogers wrote: A default configuration change was suggested which seems generally viewed as undesirable. My impression was that support was split about evenly, actually. I take it that means you're against changing the default? I'm a bit surprised, since

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Tom Jackson
John, This isn't a democracy. You have to demonstrate some understanding of how things work. The only real security issue is your misuse/abuse of ns_returnfile to serve dynamic data. Nobody is going to guarantee that you can't shoot yourself in the foot due to your lack of understanding of

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Rusty Brooks
Personally I can't imagine any persuasive argument that a caching mechanism that can easily confuse /usr/local/private/var/rootpass and /var/tmp/verisign/certs/webcert.txt should be enabled by default in a web server. Oh, come on. Only if you're rapidly creating and deleting these files.

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread John Caruso
On Tuesday 04:57 PM 8/19/2008, Rusty Brooks wrote: Personally I can't imagine any persuasive argument that a caching mechanism that can easily confuse /usr/local/private/var/rootpass and /var/tmp/verisign/certs/webcert.txt should be enabled by default in a web server. Oh, come on. Only

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Jim Davidson
Hi, I haven't looked at a directory change notification type scheme in a long time but that could be very clever. Aside from addressing issues discussed here, the key benefit would be to avoid the repeated stat syscalls. Those stat calls always bothered me conceptually but the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread russell muetzelfeldt
On 20/08/2008, at 9:57 AM, Rusty Brooks wrote: I think it's interesting, at least, that this topic has created more traffic than the list usually sees in a whole year. most of which isn't actually about the issue at hand, but rather whether John is an idiot for expecting ns_returnfile to

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread John Caruso
On Tuesday 05:39 PM 8/19/2008, Jim Davidson wrote: Your right, the code snippet below could trip over a race condition as you've described. It's not a race condition, actually; the code in that example was serialized, so there's no race involved. ...fastpath isn't broken. It's designed

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-19 Thread Jim Davidson
One other idea could be to modify the code so that the Win32 behavior (cache by filename) could be made a configurable option (perhaps default on) instead of compile-time. Would be easy to fiddle with code snippets like this to make that happen: #ifdef _WIN32 key = file; #else