Hi, I just checked in some changes to hopefully fix the pre-queue interp leak muck (and other bugs). I also added read and write filter callbacks -- the read callbacks can be used to report file upload progress somewhere. And, I added new ns_cls and ns_quewait commands to work with the curious Ns_QueueWait interface. Some man pages were updated to reflect the changes including an example in ns_quewait.n. There's not yet an HTTP protocol interface on top of ns_quewait but it could be added, letting you do some REST things, for example, in an event-driven manner before your connection gets running.
-Jim On Dec 1, 2009, at 4:45 PM, Jeff Rogers wrote: > Jim Davidson wrote: >> Right -- the pre-queue thing operates within the driver thread only, >> after all content is read, before it's dispatched to a connection. >> The idea is that you may want to use the API to fetch using >> event-style network I/O some other bit of context to attach to the >> connection using the "connection specific storage" API. So, it won't >> fire during the read. > > Can you share any specific examples of how it has been used? It's always > struck me as an unfinished (or very specific-purpose) API since it's > undocumented and it seems that doing anything nontrivial is liable to gum up > the whole works since the driver is just a single thread. > >> However, a progress callback API could be good -- wouldn't be called >> unless someone wrote something special and careful enough to not bog >> things down. Maybe an "onstart", "onend", and "on %" and/or "on #" >> callback which could do something like dump the progress in some >> shared (process or machine wide) thinger for other threads to check. >> I like the idea... >> Oh, and the pre-queue leak sounds like a bug -- should just be >> re-using the same Tcl interp for all callbacks. > > In the case of a tcl filter proc, ProcFilter gets the server/thread specific > interpreter for the connection and expects NsFreeConnInterp to be called > later, but in the case of pre-queue filters NsFreeConnInterp is never called > in the driver thread so it allocates (PopInterp) a new interp every time. > Adding in a call to NsFreeConnInterp after the prequeue filters looks like it > fixes the problem. If a filter proc is added into SockRead the same thing > would need to happen (potentially in the reader thread instead of the driver > thread). > > One thing I am confused about tho, is why without calling NsFreeConnInterp in > the driver thread it just leaks the interps rather than crashing when it > tries to use the interp in the conn thread, since it looks like a new conn > interp wouldn't get allocated in that case. > > I also don't understand why there can be multiple interps per server+thread > combo in the first place (PopInterp/PushInterp); I'd expect that only one > conn can be in a thread at a time and that it always releases the interp when > it leaves the thread. > > -J > >> -Jim >> On Nov 25, 2009, at 5:46 PM, Jeff Rogers wrote: >>> It looks like the pre-queue filters are run after the message body >>> has been read, but before it is passed off to the Conn thread, so >>> no help there. However it looks like it would not be hard to add >>> in a new callback to the middle of the read loop, tho it's >>> debatable if that's a good idea or not (for one, it would get >>> called a *lot*). >>> Curious about that tcl prequeue leak. I guess no one uses or cares >>> about it, since the symptom is serious (more than just a really big >>> memory leak, it crashes the server too), the cause is pretty >>> obvious and the fix appears on the surface to be pretty obvious, >>> although it does result in prequeue filters working differently >>> from all the others, in particular that it would use a different >>> interp from the rest of the request. >>> -J >>> Tom Jackson wrote: >>>> There is one possibility. There is a pre-queue filter in >>>> AOLserver (run inside the driver thread). It works from the Tcl >>>> level, but creates a memory leak equal to the size of an interp, >>>> in other words a huge memory leak. However, maybe at the C level, >>>> you could create a handler which would do something interesting >>>> before returning control back to the driver thread and ultimately >>>> the conn thread. I'm not sure exactly when the pre-queue filters >>>> are activated, but if it is before reading the message body, it >>>> might be useful. >>> -- AOLserver - http://www.aolserver.com/ >>> To Remove yourself from this list, simply send an email to >>> <[email protected]> with the body of "SIGNOFF AOLSERVER" in >>> the email message. You can leave the Subject: field of your email >>> blank. >> -- AOLserver - http://www.aolserver.com/ >> To Remove yourself from this list, simply send an email to >> <[email protected]> with the body of "SIGNOFF AOLSERVER" in >> the email message. You can leave the Subject: field of your email >> blank. > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to > <[email protected]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: > field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[email protected]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
