Am 02.01.2006 um 09:21 schrieb Stephen Deasey:
POST request with small amounts of data are really common. Think of a
user logging in via a web form. Couple of bytes.
The way you've coded it at the mo (and I realize it's just a first
cut), all requests with more than 0 bytes of body content will cause
the upload stats code to fire. That's why I suggested we may want to
have some lower threshold, or restrict it to certain URLs via
urlspecific data. We don't want to track every form submission.
No. That is of course not good.
One problem i see with lazy uploads that if you have multiple clients
doing large POSTs, spawning multiple clients for a long time reading
that content will waste resources, each conn thread is heavy, with
Tcl
interp.
Not necessarily. You can create another thread pool and then ensure
that those threads never run Tcl code. Remember, Tcl interps are
allocated lazily on first use. We could also adjust stacksize per
thread pool.
But then, how are you going to interpret the stats? You need to
run some Tcl code for that and this will load the Tcl interp?
Using driver thread reading small chunks from the connections
and putting it into file will keep everything smooth. But with small
uploads on fast network this may not be an issue, so it needs a
compromize solution here, may be configurable options. Currently,
spooling into file can be disabled/enabled, lazy spooling may be
implemented similar way. Actually, lazy file spooling can be easily
done, because even Ns_ConnContent calls SockRead which does
spooling, we
just need to introduce an option that tells how much we should
spool in
the main thread and then continue in the conn thread.
We already have maxreadahead, which is the amount of data read by the
driver thread into a memory buffer. I think you're either happy
letting the driver thread block writing to disk, or you're not. Why
would you set a threshold on this?
Right. We already have that knob: maxreadahead. It is the question
what happens when we reach that point. At the moment, extra data is
spooled into the file and file is mmaped at the end of reading
when all the content from the client is received. I'm not that happy
with the driver thread pumping the data into the file as this might
affect the overall performance, therefore I suggested kernel AIO.
This feature of OS is developed precisely for that purpose.
Zoran
