I looked at the code a bit closer. The ns_returnfile and ns_respond commands both call Ns_ConnReturnFile, the public API to the underlying FastPath. It does more than just blast the content -- it handles:

-- respond "not found" if file doesn't exist
-- sets "last modified header" based on mtime
-- returns "not modified" if client sent "if-modified-since"
-- returns headers only if a HEAD request
-- sets mimetype and contentlength
-- caches, mmap's, or simply opens the fd and sends, chunk by chunk

The code is a bit nested, with a few layers of code which handle similar cases (send file, FILE *, fd, etc.). It's not 100% perfect because the file could change between the stat to check existence and get size/mtime for header construction and the actual send. There's even a note that say something like "silently ignore truncated file..." deeper down so that if fastpath stats a file of size x but opens and starts sending a smaller file, it'll just silently quit after sending what it found. By comparison, the ADP template read/ parse/cache code attempts to catch that case -- it tries up to 10 times in ParseFile to get a consistent fstat/read on the open file (you can see that in adpeval.c).

Overall, this means ns_returnfile is designed to mimic the core file- send/fastpath stuff as if there was no Tcl code involved at all -- performance, pitfalls and all. Whether fastpath is significantly smarter or dumber sending static content than, say, Apache, I can't say. This also means that a new "-cache 1/0" option to ns_returnfile (default 0 or 1, subject to debate) could be useful but not quite the same as "just send this file..." One would still get the result AOLserver would have generated (not found, not modified, etc.) unless all that was disabled as well.

By comparison, ns_returnfp is more basic -- it assumes it's sending stuff from an open channel, just sets basic headers, and has none of the "not found", "not modified", etc. type logic. Because the docs don't mention this, I think some folks expect ns_returnfile to be just this:

set fp [open file]
ns_returnfp $fp
close $fp

BTW: I mentioned that I thought it worked that way in the past but I think I was wrong. Checking the CVS logs and diff'ing the files with rev 1.1 way back in 2000 ns_returnfile was calling the fastpath and fastpath was using the dev/inode pair for the cache key on Unix.

Overall, it seems one thing to do would be to switch to filename-based cache keys by default, leaving the dev/inode pair as an option for folks who run sites with large symlinks and want to benefit from caching objects just once. I think that should avoid the data corruption cases John pointed out with minimal downside. We could also modify ns_returnfile to behave as above, calling ns_returnfp, but I'm thinking that may break more than it fixes if folks are assuming the behavior it has now (e.g, an ns_returnfile of an image where they expect the not-modified responses).


On Aug 20, 2008, at 9:23 AM, Titi Alailima wrote:

-----Original Message-----
From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On
Behalf Of Jim Davidson
Sent: Tuesday, August 19, 2008 8:39 PM
Subject: Re: [AOLSERVER] Data "corruption" with fastpath caching

Your right, the code snippet below could trip over a race condition as
you've described.  But, that's not reason enough to change the
fastpath, it's reason to better document the behavior so folks don't
write code that uses ns_returnfile for temporary, dynamic content.
Although fastpath takes care to be correct in most cases (e.g.,
stat'ing the file on each request and serializing read on cache miss),
the "fast" in fastpath is because it's primarily designed to return
simple static content with minimal overhead.

BTW:  I believe the ns_returnfile command didn't use the fastpath
originally -- I think it just opened and sent the content.  It was
changed because folks asked for it to go faster I think -- can't

This sounds like the problem. Not a bug with fastpath, but a questionable decision to take an apparently generic command and apply a situation-specific optimizer to it that returns bad results when applied outside of its design parameters. It seemed like a valid assumption at the time, just as John's code seemed to be making valid assumptions, but it just seems to have led to asses being made left and right.

Is there any way to make ns_returnfile's use of fastpath configurable, and turn _that_ off by default? Or would that be essentially the same as just turning fastpath off entirely? I must confess my ignorance of where fastpath gets used.

Titi Ala'ilima
Lead Architect
MedTouch LLC
1100 Massachusetts Avenue
Cambridge, MA 02138
617.621.8670 x309

AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED] > with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.

AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.

Reply via email to