On Dec 1, 2009, at 4:45 PM, Jeff Rogers wrote: > Jim Davidson wrote: >> Right -- the pre-queue thing operates within the driver thread only, >> after all content is read, before it's dispatched to a connection. >> The idea is that you may want to use the API to fetch using >> event-style network I/O some other bit of context to attach to the >> connection using the "connection specific storage" API. So, it won't >> fire during the read. > > Can you share any specific examples of how it has been used? It's always > struck me as an unfinished (or very specific-purpose) API since it's > undocumented and it seems that doing anything nontrivial is liable to gum up > the whole works since the driver is just a single thread.
Nope, no examples I'm aware of. You're right it's unfinished, mostly because it's missing a curl-like simple HTTP interface on top and a Tcl interface to configure it to do the common thing of fetching something over the net via REST or SOAP and shoving the result into CLS for later use. And, there's no documentation so how would anyone know what or why it would be useful? Also, it's a bit "advanced" as it were for just the reason you mentioned -- if you mucked up your callback, you would stall all accept and read-ahead. Frankly, I always thought it was a cool idea but could never get anyone else interested. In practice folks had various "search database here..." or "fetch xml there..." type stuff riddled throughout their existing ADP scripts and it wasn't strictly necessary or a priority to re-factor that stuff to move it all to the pre-queue interface. My goal of getting all the "context" into the connection before processing began, eliminating any potential long-waiting stalls in the connection threads was never something that got others super excited. Even so, the concept is still cool. > >> However, a progress callback API could be good -- wouldn't be called >> unless someone wrote something special and careful enough to not bog >> things down. Maybe an "onstart", "onend", and "on %" and/or "on #" >> callback which could do something like dump the progress in some >> shared (process or machine wide) thinger for other threads to check. >> I like the idea... >> Oh, and the pre-queue leak sounds like a bug -- should just be >> re-using the same Tcl interp for all callbacks. > > In the case of a tcl filter proc, ProcFilter gets the server/thread specific > interpreter for the connection and expects NsFreeConnInterp to be called > later, but in the case of pre-queue filters NsFreeConnInterp is never called > in the driver thread so it allocates (PopInterp) a new interp every time. > Adding in a call to NsFreeConnInterp after the prequeue filters looks like it > fixes the problem. If a filter proc is added into SockRead the same thing > would need to happen (potentially in the reader thread instead of the driver > thread). Ah ... this does sound like a bug. Since the interface is undocumented and not used, I don't think it would hurt to add a call to NsFreeConnInterp as a special case of a pre-queue filter. NsFreeConnInterp does call a "free conn" Tcl trace which assumes end of connection but I guess that's ok to call pre-queue (if there has ever been one registered). > > One thing I am confused about tho, is why without calling NsFreeConnInterp in > the driver thread it just leaks the interps rather than crashing when it > tries to use the interp in the conn thread, since it looks like a new conn > interp wouldn't get allocated in that case. Hmm... appears it's working when it shouldn't, Interps are supposed to be per-thread but maybe in this case the interp is floating from one thread to the next and still working, avoiding things that wouldn't work. Anyway, calling NsFreeConnInterp should clear this out. > > I also don't understand why there can be multiple interps per server+thread > combo in the first place (PopInterp/PushInterp); I'd expect that only one > conn can be in a thread at a time and that it always releases the interp when > it leaves the thread. I think there was an edge case that led to a need for a cache of possible interps instead of just holding one. In practice, it's always just one unless someone writes some weird code to do a pop/push directly to have some alternate interp for a special purpose. -Jim > > -J > >> -Jim >> On Nov 25, 2009, at 5:46 PM, Jeff Rogers wrote: >>> It looks like the pre-queue filters are run after the message body >>> has been read, but before it is passed off to the Conn thread, so >>> no help there. However it looks like it would not be hard to add >>> in a new callback to the middle of the read loop, tho it's >>> debatable if that's a good idea or not (for one, it would get >>> called a *lot*). >>> Curious about that tcl prequeue leak. I guess no one uses or cares >>> about it, since the symptom is serious (more than just a really big >>> memory leak, it crashes the server too), the cause is pretty >>> obvious and the fix appears on the surface to be pretty obvious, >>> although it does result in prequeue filters working differently >>> from all the others, in particular that it would use a different >>> interp from the rest of the request. >>> -J >>> Tom Jackson wrote: >>>> There is one possibility. There is a pre-queue filter in >>>> AOLserver (run inside the driver thread). It works from the Tcl >>>> level, but creates a memory leak equal to the size of an interp, >>>> in other words a huge memory leak. However, maybe at the C level, >>>> you could create a handler which would do something interesting >>>> before returning control back to the driver thread and ultimately >>>> the conn thread. I'm not sure exactly when the pre-queue filters >>>> are activated, but if it is before reading the message body, it >>>> might be useful. >>> -- AOLserver - http://www.aolserver.com/ >>> To Remove yourself from this list, simply send an email to >>> <[email protected]> with the body of "SIGNOFF AOLSERVER" in >>> the email message. You can leave the Subject: field of your email >>> blank. >> -- AOLserver - http://www.aolserver.com/ >> To Remove yourself from this list, simply send an email to >> <[email protected]> with the body of "SIGNOFF AOLSERVER" in >> the email message. You can leave the Subject: field of your email >> blank. > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to > <[email protected]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: > field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[email protected]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
