Stephan Richter wrote:
On Tuesday 09 May 2006 07:22, Jim Fulton wrote:

I guess we need to make this a priority for the next release.

Python simply does not support a general robust reload, other than

I think that there are 2 ways we can make progress in this area:

- Speed up restart.  I think there are a lot of ways that restarts
  can be made faster:

  o Optimizae what we're doing now.  I suspect that there are some
    opportunuties here.

I have applied for the SoC with a proposal to enhance ZCML. My proposal is attached. It discusses some of the optimization options we talked of before.


  I propose to allow local components to be configurable through ZCML. This
  goal became feasible with the recent component architecture refactorings by
  Jim Fulton. Any site (local or global) can now have a set of base sites that
  are used to provide additional components. I want to allow ZCML to specify
  any number of base sites and add components to it. Here is an example of how
  I imagine it to look like (ZCML)::

    <site name="my-base-site" />

I'll note that I'd like site-name to be a convenience directive that is
equivalent to defining a GlobalComponents object and registering it as
an IComponents utility.  Note that GlobalComponents doesn't exist yet. :)
It should be very similar to BaseGlobalComponents, except that it pickles
and unpickles as a named utilities registered with a BaseGlobalComponents.

    <configure use-site="my-base-site">

  The new ``site`` directive creates a new site. The ``configure`` directive
  will grow an ``use-site`` attribute that specifies the site to put in the
  components. By default,

Actually, the site to put the components in. :)

  There will be a new registry of all ZCML-defined sites.

No!  Use the base utility registry.

>   All existing ZCML
  directives have to be reviewed and it must be ensured that they are
  multi-site aware. The tricky part of the implementation will be to hook in
  those sites as bases to local sites. It must be ensured that the ZODB can
  load having filesystem-based sub-sites, error handling must be carefully
  considered and an UI must be written.

The pickling aspects are pretty trivial.  I'm not sure what UI you are refering 


  Another big problem in Zope 3 is the startup time. Some code profiling has
  determined that most of the time in the startup process is lost in parsing,
  converting and validating ZCML directives with their schemas. Thus, this
  startup problem is not purely a Zope 3 problem, but one that affects
  everyone using ZCML.

  This problem can be addressed in several ways. The most obvious one to Zope
  2 developers would be not to restart the application server, but only reload
  the packages and modules that were affected by the code changes. This
  approach has been used in Zope 2 for many years, but it several serious
  problems and some of the smartest people I know have not been able to
  completely solve the problems. Based on that, I do not think that a proposal
  suggesting this approach would be accepted.

  The second approach is to reduce the ZCML processing time, which could be
  integrated into the reload mechanism for Zope 2. This can be accomplished by
  storing some binary representation of the ZCML, similarly to ``*.pyc`` files
  in Python. Again there are several choices to consider and they should
  probably all be tried. The first solution would be to store a pickle of each
  parsed directive, namely the action and its arguments. There would be one
  pickle file fore each ZCML file. When the ZCML file changed, the pickle
  would be updated. Pickle loading would be much faster than pure ZCML
  loading, since no XML-parsing, value conversion and schema validation would
  be necessary.

Note that this will require a refactoring of ZCML handlers to define picklable
actions.  This will also require refactoring so that work now done by handlers
be defered to action execution.

  On the other hand, ZCML creates actions that are eventually
  executed. Actions are created by executing the directive handlers. Thus the
  optimization in this approach would be even greater. The problem with this
  approach is that not all directives are easily pickable. Directive handlers
  often create new types/classes on the fly. This problem could be solved by
  ensuring that directives only create pickable actions. Clearly, this would
  require a lot more work, since you would have to go through all directives
  to ensure their pickability and also provide fallbacks for 3rd-party

Yup.  One possibility is to have a mechanism in which the pickle files
are created only when possible.  That is, we try to pickle the actions
and give up if pickling fails. If we update the core directives to
work this way, that will account for most configuration files and
could provide a significant speedup even if not all configuration files
are handled.  Note that, if this is pursued. then a prototype should
test the speedup by updating some core directives and measuring the

  Overall, I believe that the performance of ZCML could be improved
  drastically either way and that the benefit would be immediately visible for
  anyone using ZCML, including Zope 2 developers using Zope 3 components via

IMO, the action pickling work should be deferred until we allow configuration
to be expressed in Python and assess the impacts.  If many people switch to
defining configuration actions in Python, then speeding action generation from
XML could be moot.


Jim Fulton           mailto:[EMAIL PROTECTED]       Python Powered!
CTO                  (540) 361-1714  
Zope Corporation
Zope3-dev mailing list

Reply via email to