I ran some benchmarks about 6 months ago and saw very different
results when our TCL library was present vs. when it wasn't present.
We finally nailed the performance difference down to this thread
startup issue. I don't remember the numbers, but it was striking
enough that I posted here about it and Rob (and others) explained what
was happening. I seem to recall thread startup being 2-3 seconds,
which is an eternity compared to 2.3.3.
I think the idea of redefining the unknown command is good. We have
something like 600 procs in our library. On average, any given proc
may be used in only 5 places (I'm guessing here), so if a new thread
is launched, for us I believe it would be better to take the dynamic
proc definition hit rather than define 600 procedures. I would guess
that a new thread would need fewer than 25 procs out of our library to
complete a random request.
It seems this setup would handle traffic spikes, web attacks, and
poorly-engineered search engine spiders much more smoothly. In these
cases it is likely that a bunch of new threads will start to handle
the temporary load, service only a few requests, and then die (if
configured) after being idle a long time. It would be a waste of
resources for all of these short-lived threads to know about our
entire library. Taking the (small) dynamic proc hit 10-25 at a time
for each request would be much better, IMO, especially if a way
is devised to cache the compiled TCL using the same mechanism.
With 2.3.3, we just bumped maxthreads whenever we were hitting that
limit a few times a day. With 3.X, we will have to keep in mind the
long thread startup time and realize that most of the time it will be
better for a new request to wait until another thread is finished
rather than starting a new thread. We should only bump maxthreads
when the wait time starts to exceed 3 seconds. (I have no idea how
we measure this yet...)
Jim
>
> On Thursday, September 20, 2001, at 09:46 AM, Jim Wilcoxson wrote:
>
> > Seems like some kind of dynamic proc definition mechanism similar to
> > autoloading would be useful here...
>
> I was just reading the Ousterhout book on the unknown command and
> autoloading. I don't think the default Tcl autoloading mechanism is
> appropriate, because it depends on building an index file on disk, but I
> think it might be useful to create an AOLserver-specific unknown command,
> and perhaps redefine "proc" so that AOLserver stores the proc defs in a
> hash; then, when an unknown command is invoked, the AOLserver-specific
> unknown command could grab the def from the hash and eval it, and then
> execute the command.
>
> One issue I'm concerned about here is the overhead of such an approach --
> the first time through an interpreter, you could be hitting the unknown
> command a lot, and it could be really slow. On the other hand, maybe it
> wouldn't be any slower than eval'ing a big script of all the procs.
>
> Jim, do you know if it's the interp startup script that's taking the time
> in your application?
>
> I was also thinking it would be useful if we could have some form of
> shared libraries for Tcl, so that a compiled proc could be shared among
> interpreters. This would improve memory footprint, and might help reduce
> interp startup time. Does anyone know if modern Tcl supports sharing
> compiled procs? It didn't seem to as of 8.3, but I could be misreading
> the docs.
>
