Hi,
I haven't looked at a "directory change notification" type scheme in a
long time but that could be very clever. Aside from addressing issues
discussed here, the key benefit would be to avoid the repeated "stat"
syscalls. Those stat calls always bothered me conceptually but the
performance of the underlying systems always improved faster than my
irritation would grow to do something about it. However, we were
always careful to run websites against local filesystems - I would be
more concerned with the overhead if we were using NFS or some other
shared filesystem thing.
Somewhat related, the "dci module" (a series of AOL extensions we open
sourced awhile back) includes some content fetch/caching features
called "sob". That had the model you described -- things stayed in
the cache until either space was needed or the server received an
explicit flush message on a publish event. That approach worked well
and scaled well but it wasn't entirely general nor naive, i.e., it was
key that we understood how it worked under the covers and to make sure
the flush message links were reliable to avoid stale content problems.
Anyway, I've been pondering this whole discussion some more and agree
with Tom -- the fastpath isn't broken. It just does a certain thing
-- serves static files with a reasonable balance of performance and
stability -- and shouldn't be modified except to add notes about how
it works in the docs. I'm having trouble thinking through how it
could be modified to plug all possible race conditions. I'd suggest
the code snippets using fastpath for dynamic content should be
modified, perhaps some new Tcl commands could be added to make it
convenient, but otherwise it seems a mismatch between capabilities and
requirements.
-Jim
On Aug 19, 2008, at 1:03 PM, Juan José del Río wrote:
What about using epoll (or equivalent) in Linux, and kqueue in FreeBSD
to tell the kernel to notify AOLServer in change a file has changed?
That'd be a pretty easy and efficient way to discard fastpath items in
case they have been deleted and/or modified.
Just my two cents ;-)
-
Juan José del Río |
(+34) 616 512 340 | [EMAIL PROTECTED]
Simple Option S.L.
Tel: (+34) 951 930 122
Fax: (+34) 951 930 122
http://www.simpleoption.com
On Tue, 2008-08-19 at 09:20 -0700, Tom Jackson wrote:
Andrew,
This is not a corner case. The exact same thing could happen without
fastpath.
What is that thing? That the contents of a file changes after a
request
is made and before the file is returned. In fact, there is no
guarantee
that it won't change mid-return. This is a fact of life with files on
any filesystem.
In fact, with the HTTP caching mechanisms, you could fail to get
up-to-date contents of a file, since the If-Modified-Since mechanism
will also fail.
The problem here is that the application is using this static file
handling API to serve dynamic content. Wondering why it doesn't
work is
pointless.
Just to summarize again, this case requires that a file is created
then
destroyed and another file created within the same second that has
the
same size. Also, the original file must get into the cache, and the
only
way that can happen is for the application to treat it as a long
lived
static file.
We have other means to cache dynamic data, and large chunks of
dynamic
content saved as a file can avoid the fastpath cache by setting the
cachemaxsize parameter. Writing smaller content to disk doesn't
make any
sense if your goal is speed...or security.
It is probably even more important to tamp down these misconceptions
about how AOLserver works. Static and dynamic content are handled by
different API. The reason is that it has long been recognized by the
developers of AOLserver that different techniques are required to
maintain high performance based upon how the content is generated,
its
expected lifespan, its size, and its potential for reuse.
tom jackson
On Tue, 2008-08-19 at 03:00 -0400, Andrew Piskorski wrote:
On Mon, Aug 18, 2008 at 06:06:23PM -0700, John Caruso wrote:
That'd be an improvement over the current situation, but it's
still the
case that AOLserver as currently shipped has a file cache
mechanism built
into it which 1) may return incorrect data and 2) is enabled by
default. Given the risk, I'd say fastpath caching should be
disabled by
default rather than enabled.
Sounds right to me. Either robustify Fastpath somehow against this
corner case, or don't have Fastpath turned on by default.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]
> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.