Re: [AOLSERVER] interpreter lifecycle

[Jim Davidson]

Howdy,

I've been following this thread the last few days -- great discussion.  Let me add a few points and suggest how we can move forward.


First, I'm responsible for the current approach of initializing Tcl interps in AOLserver and I agree with Zoran -- it's clumsy.  Aside from failure on my part to keep things simple, the current state reflects AOLserver's age (10 years now, should we have a celebration?) which pre-dates useful Tcl features, in particular packages and namespaces.  In lieu of packages, AOLserver used it's "module" initialization framework to handle Tcl initialization which is non-standard, not namespace-aware, and complicated by "shared" and "virtual server"   confusion.

I also agree with Zoran that "cloning" would be hard to get right.  I understand how low-level C-copying of most things could be fast.  In fact, pre-opensource versions of AOLserver went a step beyond cloning and actually linked interps together, sharing command and global variable tables so new interps would flash into existance with (most) state ready to go.  Everything was great except I had to hack too much of the Tcl core, too many locks where involved, and there was confusion between global and "ns_share" variables.  More importantly, cloning wouldn't solve the full problem which includes not only initialization but regular garbage collection and re-init between transactions/connections.


I've done two things in the current 4.5 code to try and simply things.  First, I added a bunch of comments in nsd/tclinit.c so one can follow the arcane flow of virtual-server startup init and then interp lifetime. More importantly I added a new API, Ns_TclRegisterTrace, (unfortunately undocumented) which allows one to add C callbacks at the key Tcl interp state transition points: create, allocate, deallocate, and attach and detach from a connnection. You can use the "ns_ictl" command to register Tcl-script level traces as well.  The legacy code (e.g., Ns_TclInitInterps) has been modified to use the new API.  While technically possible, it turns out without the new interface it was difficult to really control these points. Another problem with the pre-4.5 model is it set the wrong expectation, that either the default would do the right thing with all the state extraction and copying or that the code could not be easily extended (I don't get the sense many folks modify the "ns_init" and "ns_cleanup" procs).


So, here's a suggestion on how to move forward:

1. Avoid the cloning idea as we've been there before and it never really worked perfectly.  And, when interps are well managed and are retained transaction to transaction, the time spent creating the interp tends to zero when amortized over the long lifetime of the interp so the overhead gain, if any, isn't so critical.

2. Modify existing code to be actual packages.  I've started this already with AOLserver core itself, e.g., the following should work:

# /usr/local/aolserver/bin/tclsh8.5
% lappend auto_path /usr/local/aolserver/lib
% pkg_mkIndex /usr/local/aolserver/lib
% package require Nsd
4.5
% info comm ns*
... a bunch, but not all, AOLserver commands ...


3. Use the package interface along with ns_ictl to manage state clearly, e.g., replace the confusing code in bin/init.tcl with the following and expect folks to own this config for their own install as they do with nsd.tcl:

ns_ictl trace create {
  package require Nsd
  package require Foo
  package require Bar
}

ns_ictl trace dealloc {
  ... default cleanup for Nsd, clear ns_sets, etc...
  ... cleanup, if any, for Foo, Bar, etc.
}


4. Give up on general dynamic reconfig via ns_eval, instead use ns_ictl again for specific interfaces:

ns_ictl trace alloc {
   ... check for some update, e.g., a control file change or something to act on...
}


5.  Finally, the last thing we should consider is giving up on virtual servers.  It turns out that there is still quite a bit of code complexity to support virtual servers and more importantly it confuses the "process" init with "server" init, limiting our ability to turn AOLserver completely into a Tcl loadable module which would really simplifiy interp management and align with normal Tcl practice.  This is something to consider later.


-Jim
2">