A portable way to handle really huge files would be to let the conn
service threads (I'm thinking primarily of fastpath) co-routine with
the HTTP driver:
HTTP socket driver reads all of the request input
Initial request is dispatched to fastpath:
fastpath opens the file
if new request
stat file
do ifmod
set headers, etc.
if file size is "small"
add file to cache, return to output socket driver
else if file size is "large", but < max cache entry size
add file to cache, set quick expiration, return to output socket driver
else if file size is > max cache entry size
read large-sized hunk (250KB, or some configurable chunk size)
set file offsets in conn structure & "incomplete" flag
set quick expiration
return to output socket driver
fi
else (continuation of incomplete request)
position to last offset
read a large hunk
update file offsets in conn structure & set/clear "incomplete" flag
return to output socket
fi
close file
IMO, with this kind of setup, AOLServer could serve a thousand movie
downloads or CD image downloads in an efficient manner, using just a
handful of connection service threads and 250MB of memory (1000 x the
max chunk size) for the output buffers. As a side benefit, it
conserves (disk) file descriptors in a big way, because they are not
in use while the data is being spooled to the socket.
I'm not saying AS should be designed/optimized for serving movies or
CD images, but IMO this setup would also make it a better (more
scalable) image server, and that is an important webserver feature.
I'm continuing to post on this subject because if changes are being
considered to the design of the socket drivers to support multiple
protocols, maybe it's a good time to consider output spooling too.
To me, tuning the cache is a totally separate issue, but may also be
useful.
Jim
>
> On 2004.08.19, Jim Wilcoxson <[EMAIL PROTECTED]> wrote:
> > For fastpath (static file serving), reading a file into a memory
> > buffer would seem to scale well for typical web size images: 250K or
> > less, especially considering that fastpath already uses a memory
> > cache. I can see 3 scenarios:
> >
> > 1. File is small: put it in the cache, leave it there after spooling
> > 2. File exceeds cache size limit, but is less than 1MB: put it in the
> > cache, delete it after spooling or make it expire "very soon"
> > 3. File is huge: let the connection thread spool it, like it does now,
> > and don't spool it, or
>
> The Netscape Enterprise Server (NES) web server has a static file cache
> that's tunable ... "nfsc" rings a bell. You specify things like the
> max number of entries in the cache, the byte size ranges that define
> "small", "medium" and "large" files, etc.
>
> Could be an interesting idea to accelerate fastpath static content
> serving from AOLserver. Maybe.
>
> -- Dossy
>
> --
> Dossy Shiobara mail: [EMAIL PROTECTED]
> Panoptic Computer Network web: http://www.panoptic.com/
> "He realized the fastest way to change is to laugh at your own
> folly -- then you can let go and quickly move on." (p. 70)
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with
> the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field
> of your email blank.
>
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
