Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Don Baccus
On Aug 21, 2008, at 8:14 AM, Dossy Shiobara wrote: I've remained silent on this issue because I didn't want to be accused of stifling the community, etc. ... End of discussion. Accused. Guilty. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org --

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Dossy Shiobara
I think there has been more than enough discussion around this issue which I did not interfere with nor influence in any way. However, to let it continue to spin over and over is unproductive. I'd love to hear other solutions other than configurabe cache key strategies or a time-based delay

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread John Caruso
On Thursday 08:14 AM 8/21/2008, Dossy Shiobara wrote: 4) I see the simplest (best?) solution here being a configurable parameter that controls fastpath's cache key generation. As Jim points out, one can quickly test whether this would solve the problem at hand by temporarily #define'ing

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Jeff Rogers
Dossy Shiobara wrote: However, to let it continue to spin over and over is unproductive. I'd love to hear other solutions other than configurabe cache key strategies or a time-based delay caching strategy, but the time to debate whether this is a defect or not is officially over: it is a

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Tom Jackson
On Thu, 2008-08-21 at 11:14 -0400, Dossy Shiobara wrote: 4) I see the simplest (best?) solution here being a configurable parameter that controls fastpath's cache key generation. As Jim points out, one can quickly test whether this would solve the problem at hand by temporarily #define'ing

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread John Caruso
On Thursday 09:25 AM 8/21/2008, Tom Jackson wrote: Why put newly created files into a cache, if the point of the cache is to handle static files? We can wait for evidence that it is static. In this case, we can wait until it is a few seconds old, at least. This is a very good point, actually.

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Titi Alailima
I agree that John's patch is worth doing. It satisfies both his requirements and the stated design goals of fastpath. The remaining issue is whether something called ns_returnfile which takes a pathname as a parameter should have some guarantee that you will return what at least at some point

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Rusty Brooks
I don't have any opinion on the fix, but I think the actual objection to using the filename in the fix is that this would cause hard links to files, which are for all intents and purposes The Same File, to be considered different files by fastpath. (Hard links have different names, but the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Jeff Rogers
Titi Alailima wrote: what you were looking for. Even with the mtime fix there's no guarantee that systems which muck around with mtime (such as tar) won't cause separate files to collide. For a contrived example: I think the best you can do is to use ctime instead of mtime, or maybe btime

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Titi Alailima
Right, I forgot that one. But the potential resolution is the same, allow a configurable unsafe mode. If you want the inode-only optimization and are willing to take on the resulting unpredictability of ns_returnfile, go right ahead. But the majority of developers who don't know or don't

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Tom Jackson
I'm lost. If you are interested in serving the same content, mtime tells you the last time the content was modified. ctime changes for reasons all unrelated to the content. But this is a cache, which is a copy. There is never any way to guarantee that the content is the same as what is currently

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Jim Davidson
Hi, Yes -- the original reason for dev/inode on Unix instead of filename was to reduce memory consumed in the case of a large # of symlinks or hardlinks to the same file. This was the case for AOL's digitalcity.com back in 1999. For better or worse, the AOL in AOLserver means AOL was

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread John Caruso
On Thursday 12:08 PM 8/21/2008, Titi Alailima wrote: It's perfectly acceptable in dealing with caching systems that the cached value could be out of sync, but not that the cached value could be for something entirely different from what you were looking for. Yep. I think that aspect of the

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Tom Jackson
On Thu, 2008-08-21 at 17:34 -0400, Jim Davidson wrote: So, technically this was a case where we dynamically created code which was later read by ADP (which had the same dev/inode cache stuff as fastpath). However, this was done carefully: -- Tcl-level mutex/condition variables to

Re: [AOLSERVER] Data corruption with fastpath caching

2008-08-21 Thread Tom Jackson
On Thu, 2008-08-21 at 13:52 -0700, John Caruso wrote: On Thursday 12:08 PM 8/21/2008, Titi Alailima wrote: It's perfectly acceptable in dealing with caching systems that the cached value could be out of sync, but not that the cached value could be for something entirely different from what