Jeff, Interps are confined to a specific thread. You can transfer the sock around, but not the interp. But the big reason for different interps is that they are or can be specialized. The prequeue interp could be very simple. Conn interps tend to be big and expensive so you don't want to use them any more than you have to. This is why we have a driver thread handling all i/o prior to queuing the conn. The driver thread is the upload equivalent of the download helper. (plus a bunch of other stuff).
As an example of an additional prequeue filter, maybe the new http client (written in C) using the ?ns_task? API. Basically seems like you could dole out a task and return once the task (getting something via event driven http) is complete. Maybe a good use of the prequeue filter would be to actually return this download progress information. Since it is a filter, it would fire on a particular url. Maybe you could do a quick return and abort the connection, never using a conn thread. (Note this is a separate http request from the upload request.) It should be very fast if you never fire up an interp. tom jackson On Tue, Dec 1, 2009 at 1:45 PM, Jeff Rogers <[email protected]> wrote: > Jim Davidson wrote: >> >> Right -- the pre-queue thing operates within the driver thread only, >> after all content is read, before it's dispatched to a connection. >> The idea is that you may want to use the API to fetch using >> event-style network I/O some other bit of context to attach to the >> connection using the "connection specific storage" API. So, it won't >> fire during the read. > > Can you share any specific examples of how it has been used? It's always > struck me as an unfinished (or very specific-purpose) API since it's > undocumented and it seems that doing anything nontrivial is liable to gum up > the whole works since the driver is just a single thread. > >> However, a progress callback API could be good -- wouldn't be called >> unless someone wrote something special and careful enough to not bog >> things down. Maybe an "onstart", "onend", and "on %" and/or "on #" >> callback which could do something like dump the progress in some >> shared (process or machine wide) thinger for other threads to check. >> I like the idea... >> >> Oh, and the pre-queue leak sounds like a bug -- should just be >> re-using the same Tcl interp for all callbacks. > > In the case of a tcl filter proc, ProcFilter gets the server/thread specific > interpreter for the connection and expects NsFreeConnInterp to be called > later, but in the case of pre-queue filters NsFreeConnInterp is never called > in the driver thread so it allocates (PopInterp) a new interp every time. > Adding in a call to NsFreeConnInterp after the prequeue filters looks like > it fixes the problem. If a filter proc is added into SockRead the same > thing would need to happen (potentially in the reader thread instead of the > driver thread). > > One thing I am confused about tho, is why without calling NsFreeConnInterp > in the driver thread it just leaks the interps rather than crashing when it > tries to use the interp in the conn thread, since it looks like a new conn > interp wouldn't get allocated in that case. > > I also don't understand why there can be multiple interps per server+thread > combo in the first place (PopInterp/PushInterp); I'd expect that only one > conn can be in a thread at a time and that it always releases the interp > when it leaves the thread. > > -J > >> -Jim >> >> >> >> >> On Nov 25, 2009, at 5:46 PM, Jeff Rogers wrote: >> >>> It looks like the pre-queue filters are run after the message body >>> has been read, but before it is passed off to the Conn thread, so >>> no help there. However it looks like it would not be hard to add >>> in a new callback to the middle of the read loop, tho it's >>> debatable if that's a good idea or not (for one, it would get >>> called a *lot*). >>> >>> Curious about that tcl prequeue leak. I guess no one uses or cares >>> about it, since the symptom is serious (more than just a really big >>> memory leak, it crashes the server too), the cause is pretty >>> obvious and the fix appears on the surface to be pretty obvious, >>> although it does result in prequeue filters working differently >>> from all the others, in particular that it would use a different >>> interp from the rest of the request. >>> >>> -J >>> >>> Tom Jackson wrote: >>> >>>> There is one possibility. There is a pre-queue filter in >>>> AOLserver (run inside the driver thread). It works from the Tcl >>>> level, but creates a memory leak equal to the size of an interp, >>>> in other words a huge memory leak. However, maybe at the C level, >>>> you could create a handler which would do something interesting >>>> before returning control back to the driver thread and ultimately >>>> the conn thread. I'm not sure exactly when the pre-queue filters >>>> are activated, but if it is before reading the message body, it >>>> might be useful. >>> >>> -- AOLserver - http://www.aolserver.com/ >>> >>> To Remove yourself from this list, simply send an email to >>> <[email protected]> with the body of "SIGNOFF AOLSERVER" in >>> the email message. You can leave the Subject: field of your email >>> blank. >> >> >> -- AOLserver - http://www.aolserver.com/ >> >> To Remove yourself from this list, simply send an email to >> <[email protected]> with the body of "SIGNOFF AOLSERVER" in >> the email message. You can leave the Subject: field of your email >> blank. > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to > <[email protected]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: > field of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[email protected]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
