Alberto Valverde wrote:
fumanchu wrote: > More discussion to follow. I just spent a man-year of work making CP 3 > do all this and more and be beautiful at the same time, and I'll be > damned if I let TG devolve back to preferring the ugliness of bare > WSGI and Paste.As I've mentioned, the only problem I've seen when experimenting with it is how it handles module globals because of the way I'd like to build the WSGI stack for the application. I know that CherryPy can already build a tree composed of CP and wsgi apps, however, from what I've seen (I hope I'm wrong but I believe I'm not) it seems to me that CherryPy wants to be the only one building the stack. What I mean with this is that by the way I've seen a CP app is mounted and the server started, the whole tree is built by CP mechanisms, mounted under a given prefix and served. This is what the quickstart function looks like: def quickstart(root, script_name="", config=None): """Mount the given app, start the engine and builtin server, then block.""" if config: cherrypy.config.update(config) cherrypy.tree.mount(root, script_name, config) cherrypy.server.quickstart() cherrypy.engine.start() CP and wsgi apps are mounted or grafted into a global cherrypy.tree, and then a global server and a global engine are started.
This is merely the default, because most users, especially new ones, want to deploy a single application. Hence the name "quickstart". If you want to do more than that, then naturally you must take more manual control over the mounting and startup process, and CherryPy 3 gives you LOTS of control over that (which CP 2 did not).
* It's not tied to any particular framework. Pylons is using it and it looks like any wsgi compliant framework/app could use it too (just implement it's interface and provide entry points... could even be done by a user wanting to integrate a non "paste-enabled" wsgi app into it's workflow).
Despite the rhetoric, frameworks are not evil per se; they provide a common interface to all components and are crucial for performing expensive operations _once_. WSGI does provide a lowest-common-denominator interface, but few people want to program to the WSGI interface all day long. So alternatives (like CherryPy, or paste.wsgiwrappers) are recommended to make working with WSGI more pleasant. But there's a vicious circle which accompanies the desire to "not have a framework": 1. You build your initial offering using middleware from project A. 2. The developers of project A find that parsing headers, params, and various calculated data is slow, so they cache behind the scenes in library Z. 3. The holy grail of WSGI middleware is that we should be able to steal code from lots of different projects, so you add another bit of WSGI middleware from project B. 4. The developers of project B find that parsing headers, params, and various calculated data is slow, so they cache behind the scenes in library Y. 5. As the number of libraries Z, Y, X... increases, the number of times the same data is parsed rises (increasing CPU time) and ALSO the number of places it is cached (increasing memory footprint). 6. Turbogears finds this state of affairs unacceptable, and decrees, "we will only support middleware built with library Z (or no library)". 7. Library Z therefore plays the role of a framework, no matter how much "proof by repeated assertion" that it is "not a framework" floats around in its marketing. Why go through that entire loop when CherryPy is ready now to play the part of library Z?
The last point I believe to be very important because I'd like to see other framework's apps easy to deploy along TG apps and the opposite way around too, TG apps being easy to deploy along other frameworks' apps. I'd like to stress the fact that this deployment should be easy and as standardized as possible for the end user too (sysadmins, etc..), not only developers. To achieve this it looks obvious to me that we (wsgi frameworks) shall agree on a "standard", framework-independent interface and paste deploy's is the only one I know of (is there any other?)
That goal should be addressed on web-sig, I think. Choosing a current implementation (of any portion of a web stack) and expecting each web tool to declare it a "standard" has a pretty poor track record in the world of Python web apps. The WSGI standard, by contrast, was hashed out on web-sig for quite some time, taking input from all possible players in that space.
So I began experimenting and I got to this paste.app_factory implementation for deploying a CP3 app: http://paste.turbogears.org/ paste/769. cputils looks like this: http://paste.turbogears.org/paste/ 770.
I only get the following for those: 770 HTTP/1.1 500 Internal Server ErrorContent-Type: text/plain Content-Length: 35 Date: Fri, 05 Jan 2007 23:54:07 GMT Server: CherryPy/2.2.1 Unrecoverable error in the server.
What I'm trying to achieve with this are self contained CP apps (with their own tree, config, etc...) which can be mounted by a paste.URLMap (or any other composite app factory) at *any* point in the tree.
Then I haven't communicated the architecture of CherryPy properly. A "self-contained app" (CP or otherwise) shouldn't have a Tree. The CherryPy Tree class is WSGI middleware for app dispatch, much like paste.urlparser.URLResolver. And it can dispatch to CP apps or arbitrary WSGI apps equally well. Tree dispatch is separate from the "handler dispatch" that occurs once we've finished with the WSGI stack and are inside a request object.
However, I haven't found easy to isolate each CP app because of the mentioned use of globals. cherrypy.config is easy to monkey patch to be app specific. cherrypy.request and reponse too. I know they're already thread-local, but what if a CP app dispatches (in the same thread) to a wsgi app that turns out to be another CP app wrapped into an opaque WSGI app?
First, I should clarify by repeating that CP _applications_ should not "dispatch to a wsgi app". A CP Tree instance might, because that's what it's made for. But a CP3 Application instance is just that: a WSGI application object and not middleware. There is a wsgiapp Tool in the distro, but frankly, I don't recommend it--there are much better ways to compose WSGI components. Most of CherryPy makes web application development easier, not middleware development. And this is IMO a proper focus, because the VAST majority of CP users (and TG users) are going to spend the VAST majority of their time writing web apps, not WSGI middleware.
would they get app-specific cp globals? To put it in other words: _tree = Tree() # mount stuff # global tree cherrypy.tree.graft(_tree, '/foo') cherrypy.tree.graft(another_app, '/bar') Would CP controllers under _tree see a different cp.config, cp.request, cp.response, etc... than those under cp.tree?
Yes, they would. The request and response objects are in a threadlocal; you get a different one for each request. You even get a different one when you InternalRedirect, now. The config is in three parts: global config, app config, and a "request.config" which is "all the items in the first two that apply to the current request" in a single dict. The global config is truly global, but that's as it should be. Anything you don't want to put in all applications should go in each app's config. This is nothing more than proper separation.
This is why some sort of stacked, thread-local proxy is needed.
I agree it needs to be thread-local, but it doesn't need to be "stacked" unless one *application* (not middleware) truly calls another *application*, which isn't AFAICT a use-case of the WSGI spec. In addition, CherryPy's expectation is that WSGI middleware should not use cherrypy globals, and no CherryPy middleware does.
So it seems to me that if TG wants to use CP3's engine and use paste.deploy's interfaces some patching or monkey-patching is needed... I'm really not very comfortable with this sort of monkey- patching because we seem to be poking too many CP internals again. Experience with 1.0 has taught us that this leads to a hard time if we want to migrate to newer CP versions.
I don't think you should use paste.deploy's interfaces. They're not a standard yet by any stretch of the imagination, and I've written extensively on why CherryPy's deployment interface is better at http://www.cherrypy.org/wiki/CherryPyAndPaste.
TG 2.0 want's to build on independent components which can be stolen, adapted or written from scratch and be reused in other frameworks/apps so it looks like WSGI middleware is a better alternative to CP's tools/hooks because the latter can only be used in CP apps.
This is a non-sequitur. Nothing about CP keeps you from *using* WSGI components from anywhere, whether stolen, adapted or written from scratch. Please, beg, borrow and steal good code from these sources. But at the end of the day, middleware cannot implement all functionality (and I would argue little of the interesting functionality) for extending the request/response process. So you'll need some code to allow plugins inside the application itself, after all the WSGI layers have done their thing. CherryPy is ready, willing, and able to provide that extension system today. Why reinvent it? Now, if you want to argue that CP Tools deny you the ability to share your code with others, that's a different tangent. But IMO that desire pales in comparison to the desire to have a useful, beautiful, fast web framework. Not to mention that the vast majority of Tools are short bits of code and could be ported to uglier middleware versions with a couple quick greps. I have no problem making those who want to steal MY code do that work; it's silly to make ME do it (and make my toolset ugly in the process in order to support it).
This brings us to a point that we should consider what CP3 provides us with. If we don't need config, deployment, tools and filters we're left with the server and it's dispatching mechanisms which is what most TG users are used to and love.
There's something missing from the above list: management. You are going to end up reinventing the entire set of management layers, like CherryPy's Server class (which governs the HTTP server, and can now start/stop/manage multiple HTTP servers), the Engine class (which governs the OS process) and the Toolbox (which provides a plugin architecture). You do need Tools in some form, as I've argued above. You might decide that you want more of CherryPy's builtin tools to be implemented as middleware, which is fine. Whether you used CP or not, you'd have to supply those. But CP already has solid, tested, versions of these features that already exist, and they are manageable in code, by code, like any good component-management and plugin architecture. Why reinvent all that? My guess is that, using a single page of CherryPy 3 code, TG could provide an admin webapp which allowed a new developer to compose apps, middleware, and tools into a complete web stack without ever taking the server down. This is possible because the arrangement of components in CherryPy 3 is done using objects, not config-file entries (although you may use config files to declare those objects). Borrowing the terminology of the http://en.wikipedia.org/wiki/Capability_Maturity_Model (despite its weaknesses) may shed some light on what I'm describing here. Organizational processes, including software projects, can be arranged in the following rough order: 1 - Initial 2 - Repeatable 3 - Defined 4 - Managed 5 - Optimizing Turbogears has been moderately successful thus far, but has only partially reached Level 3. "...standard processes are used to establish consistency across the organization. Projects establish their defined processes by the organization's set of standard processes according to tailoring guidelines." Turbogears is largely in this position (in the HTTP arena) because of its use of a framework; namely, CherryPy 2. "At level 3, the standards, process descriptions, and procedures for a project are tailored from the organization's set of standard processes to suit a particular project or organizational unit." HTTP standards for Turbogears 1.0 apps are largely tailored from CherryPy 2. CherryPy 3 was made to take Turbogears (and ad-hoc users of CP) to Level 4 and beyond, by providing a layer of management to the components of the HTTP process. "A critical distinction between maturity level 3 and maturity level 4 is the predictability of process performance." Using CherryPy 3 as the basis for TG 2 will provide you with that predictability. Switching to Paste at this point will take Turbogears back to Level 1, because performance will be less predictable, standards go back to being specified per-component, and the discipline of repeatability is actively discouraged by the "take code from anywhere" mindset.
Taken all this into account I believe it's less risky to use CP3 server and emulate CP3's dispatching rather than heavily monkey-patching CP3's internals.
I put a LOT of work into CherryPy 3 making sure that, at every possible point where TG has or might monkeypatch CP, there is a beautiful object or attribute which is designed to be replaced with a subclass or other delegate to meet your needs. Since Paste provides neither application management nor application extension tools, TG will have to rewrite those. Why do so when CherryPy already has proven solutions in this space? Robert Brewer System Architect Amor Ministries [EMAIL PROTECTED] --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "TurboGears Trunk" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/turbogears-trunk?hl=en -~----------~----~----~----~------~----~------~--~---
