Hi,

I haven't looked at a "directory change notification" type scheme in a long time but that could be very clever. Aside from addressing issues discussed here, the key benefit would be to avoid the repeated "stat" syscalls. Those stat calls always bothered me conceptually but the performance of the underlying systems always improved faster than my irritation would grow to do something about it. However, we were always careful to run websites against local filesystems - I would be more concerned with the overhead if we were using NFS or some other shared filesystem thing.

Somewhat related, the "dci module" (a series of AOL extensions we open sourced awhile back) includes some content fetch/caching features called "sob". That had the model you described -- things stayed in the cache until either space was needed or the server received an explicit flush message on a publish event. That approach worked well and scaled well but it wasn't entirely general nor naive, i.e., it was key that we understood how it worked under the covers and to make sure the flush message links were reliable to avoid stale content problems.

Anyway, I've been pondering this whole discussion some more and agree with Tom -- the fastpath isn't broken. It just does a certain thing -- serves static files with a reasonable balance of performance and stability -- and shouldn't be modified except to add notes about how it works in the docs. I'm having trouble thinking through how it could be modified to plug all possible race conditions. I'd suggest the code snippets using fastpath for dynamic content should be modified, perhaps some new Tcl commands could be added to make it convenient, but otherwise it seems a mismatch between capabilities and requirements.

-Jim





On Aug 19, 2008, at 1:03 PM, Juan José del Río wrote:

What about using epoll (or equivalent) in Linux, and kqueue in FreeBSD
to tell the kernel to notify AOLServer in change a file has changed?

That'd be a pretty easy and efficient way to discard fastpath items in
case they have been deleted and/or modified.

Just my two cents ;-)

-
Juan José del Río    |
(+34) 616 512 340    |  [EMAIL PROTECTED]


Simple Option S.L.
 Tel: (+34) 951 930 122
 Fax: (+34) 951 930 122
 http://www.simpleoption.com


On Tue, 2008-08-19 at 09:20 -0700, Tom Jackson wrote:
Andrew,

This is not a corner case. The exact same thing could happen without
fastpath.

What is that thing? That the contents of a file changes after a request is made and before the file is returned. In fact, there is no guarantee
that it won't change mid-return. This is a fact of life with files on
any filesystem.

In fact, with the HTTP caching mechanisms, you could fail to get
up-to-date contents of a file, since the If-Modified-Since mechanism
will also fail.

The problem here is that the application is using this static file
handling API to serve dynamic content. Wondering why it doesn't work is
pointless.

Just to summarize again, this case requires that a file is created then destroyed and another file created within the same second that has the same size. Also, the original file must get into the cache, and the only way that can happen is for the application to treat it as a long lived
static file.

We have other means to cache dynamic data, and large chunks of dynamic
content saved as a file can avoid the fastpath cache by setting the
cachemaxsize parameter. Writing smaller content to disk doesn't make any
sense if your goal is speed...or security.

It is probably even more important to tamp down these misconceptions
about how AOLserver works. Static and dynamic content are handled by
different API. The reason is that it has long been recognized by the
developers of AOLserver that different techniques are required to
maintain high performance based upon how the content is generated, its
expected lifespan, its size, and its potential for reuse.

tom jackson

On Tue, 2008-08-19 at 03:00 -0400, Andrew Piskorski wrote:
On Mon, Aug 18, 2008 at 06:06:23PM -0700, John Caruso wrote:

That'd be an improvement over the current situation, but it's still the case that AOLserver as currently shipped has a file cache mechanism built
into it which 1) may return incorrect data and 2) is enabled by
default. Given the risk, I'd say fastpath caching should be disabled by
default rather than enabled.

Sounds right to me.  Either robustify Fastpath somehow against this
corner case, or don't have Fastpath turned on by default.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED] > with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Reply via email to