[tg-trunk] Re: TG2 stack explained quickly

Timur Izhbulatov Thu, 15 Jan 2009 07:59:39 -0800

2009/1/14 Mark Ramm <[email protected]>:
> My ultimate plan is to take this document and explode it out to discussion
> of what happens at each WSGI middleware layer, and how to overide the setup
> of each of these things, so that people have a clear picture of what's going
> on.
>
> Hopefully this snipit helps.   Also, if you have good ideas on how we can
> best visually represent all of this, I'd be very interested.


Hi Mark,

I have a draft of an article about TG2 and Pylons (attached) which, I
think, could be helpful. It briefly explains the WSGI stack and
request handling.

It started as a collection of random notes that I wrote for myself
when I was trying to come up with tests for #1999. I realized that I
needed to setup some infrastructure  (the stack, configuration etc.)
for the tests, but I was lacking detailed understanding of how
everything was supposed to work.

In the article I used tables which you might find interesting, plus,
included several useful references.

-- 
Timur Izhbulatov -- www.timka.org

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"TurboGears Trunk" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to 
[email protected]
For more options, visit this group at 
http://groups.google.com/group/turbogears-trunk?hl=en
-~----------~----~----~----~------~----~------~--~---

Title: Application Configuration Modules

Comparison of TurboGears 2 and Pylons

TurboGears 2 is a full stack WSGI web framework built from best-of-breed components on top of Pylons. TurboGears adds a relatively thin integration layer which creates a comfortable environment for developers already familiar with TurboGears 1.x.

When I started learning TurboGears, I only had basic understanding of WSGI and was not familiar with Pylons. Thus I was a bit confused by some extra indirection introduced by the framework for no obvious reasons. Apparently, this stems from the fact that TurboGears is still Pylons, but with some remarkable differences, and that is why the understanding of how TurboGears works is impossible without getting familiar with Pylons.

Here I will try to summarize the key points of TurboGears' operation in WSGI sense with the emphasis on where it differs from Pylons.

NOTE: Make sure you read the Pylons' Internal Architecture chapter of The Pylons Book [pylons-arch]. This is an invaluable source of knowledge!

Overview

The application modules responsible for configuration in Pylons [pylons-conf] and TurboGears [tg-conf] project are presented in the following table:

Module	Function	Pylons	TG	Comments
`config/middleware.py`	`make_app`	Y	Y	WSGI middleware stack. Calls `load_environment`
`config/environment.py`	`load_environment`	Y	Y	Application runtime environment (PylonsConfig etc.). Called from `make_app`
`config/routing.py`	`make_map`	Y	N	Pylons URL mapping. Called from `load_environment`. TurboGears uses object dispatch.
`config/app_cfg.py`	N/A	N	Y	TurboGears application configuration. Creates and configures an `AppConfig` instance. The instance's `setup_tg_wsgi_app` method is later called from `config/middleware.py` to create the `make_base_app` application factory function which is called from the `make_app` function.

Pylons is built upon Paste WSGI meta-framework. Every Pylons (and thus TurboGears) application defines a paste.app_factory entry point [pylons-arch] in its setup.py that is used to create an application instance. Usually, this is the make_app function from config/middleware.py.

In turn, make_app calls the load_environment function internally to do necessary initialization before building the application WSGI stack.

In Pylons the configuration modules are generated from template when a new project is created using the paster create command. The functions in these modules do everything necessary to setup the application, and developers are supposed to directly edit the corresponding function's body, if customization is needed.

In TurboGears configuration modules are also generated from project template, but, unlike Pylons, the generated files don't directly define the necessary functions. Instead, they call special framework factory methods which, in turn, return dynamically created function objects. Thus application developers don't have to change their code every time they update TurboGears.

The config/app_cfg.py module is special. There is no any specific function required to be defined there. The purpose of this module is to create an instance of the tg.configuration.AppConfig class and set its attributes as necessary.

That is, basic customization can be achieved by simply changing the AppConfig instance's attributes. More advanced cases may require subclassing AppConfig and overriding some of its methods.

WSGI Middleware Stack

The default stack in Pylons and TurboGears looks like this (outermost is the topmost):

Default WSGI Middleware Stack

Component	Pylons	TG	In `tg.configuration.AppConfig` class	Comments
Registry Manager	Y	Y	setup_tg_wsgi_app()	Always added. [paste-registry]
Status Code Redirect	Y	Y	`add_error_middleware()`	[pylons-wsgi] [pylons-full-stack]
Error Handler	Y	Y	`add_error_middleware()`	[pylons-wsgi] [pylons-full-stack]
DB Session Remover	N	Y	`use_sqlalchemy`, `add_dbsession_remover_middleware(),setup_sqlalchemy()`	TODO
Transaction Manager	N	Y	`use_transaction_manager`, `add_tm_middleware()`	TODO
Auth	N	Y	`auth_backend` ('sqlalchemy' is supported OOTB), `add_auth_middleware()`, `setup_sa_auth_backend()`	TODO
ToscaWidgets	N	Y	`use_toscawidgets`, `add_tosca_middleware()`	TODO
Cache	Y	Y	`add_core_middleware()`	TODO
Session	Y	Y	`add_core_middleware()`	TODO
Routes	Y	Y	`add_core_middleware()`, `setup_routes()`	TODO
Application	Y	Y	N/A	Pylons or TurboGears WSGI application

Pylons default WSGI stack is described in [pylons-wsgi]. Pylons middleware components are described in [pylons-arch]. The make_app function in config/middleware.py builds the stack by simply wrapping more and more middleware components around the application object. The application developer can add custom middleware components by directly inserting the appropriate lines of code into the make_app function.

In TurboGears the tg.configuration.AppConfig class responsible for building the WSGI stack has a number of add_*_middleware methods, which can be overridden in subclasses to change how the corresponding middleware component is added.

Application Runtime Environment

First, this should not be confused with WSGI environment. The config/environment.py module basically initializes the necessary parts of the application which are not middleware:

PylonsConfig instance (pylons.configuration.config)
templating
model
AppGlobals instance
helpers module

This is done in a way similar to that of middleware. The load_environment function is responsible for initializing the PylonsConfig object using the data from the deployment .ini file and defaults.

In TurboGears, again, the load_environment function is created in run-time by the AppConfig.make_load_environment() method which is called inside the config/environment.py.

Request Handling

Key points:

TurboGears application is a Pylons application (isinstance(tg.wsgiapp.TGApp, pylons.wsgiapp.PylonsApp) == True)
Pylons application is a WSGI application
WSGI application is a callable object which accepts two arguments

The pylons.wsgiapp.PylonsApp class is responsible for:

registering context, request, and globals request specific objects in Registry Manager [pylons-arch]
dispatching request to controller

The pseudocode for request handling control flow is shown bellow. It includes doc-strings from the respective methods:

__call__(environ, start_response)
    self.setup_app_env(environ, start_response)
    """Setup and register all the Pylons objects with the registry

    After creating all the global objects for use in the request,
    ``PylonsApp.register_globals`` is called to register them
    in the environment.

    """
        self.register_globals(environ)
 
    controller = self.resolve(environ, start_response)
    """Uses dispatching information found in ``environ['wsgiorg.routing_args']``
    to retrieve a controller name and return the controller instance from the
    appropriate controller module.

    """
        return self.find_controller(controller)
        """Locates a controller by attempting to import it then grab
        the SomeController instance from the imported module.

        """
 
    response = self.dispatch(controller, environ, start_response)
    """Dispatches to a controller, will instantiate the controller
    if necessary.
 
    """
        return controller(environ, start_response)

    return response

The tg.wsgiapp.TGApp class is the same as PylonsApp except that it overrides the find_controller method because TG doesn't check the Pylons' __controller__ special variable [pylons-ctrl].