Re: Regex based publisher proposal
* DB connection pooling Got that. It's actually very simple. Just derive from the Queue class, it's already synchronized. Override pop with the logic for handing out connections. Simply use put to return it when you're done. I recommend putting the db information in a config file like rails does, nobody wants to have to put that all over the place. * Generic rendering template system (empowering the developer to use virtually ANY templating system avail in python) That's the way mod_python is as far as I know, it pretty much lets you do your own thing for templates. Since everyone seems to have different ideas about what templates they like, I think that's great. * Remote DB Sessions (for scalability) Got that for mysql, but is based on a different session model than mod_python uses. Someone else posted a mysql session here recently, I do not know what became of it, I would recommend you follow up before it goes cold. * Advanced Debugging Get wingide, I've never seen a better debugger for python. It handles remote debugging on mod_python flawlessly. When PythonDebug is on, I automaticaly connect to the debugger from python. I tried Komodo first for the same, but it is worse than useless. If you want help getting that working let me know. -Dan
Re: Regex based publisher proposal
What Sébastien proposes is the kind of dispatching mechanism used in Django, and (without regular expressions) in Rails and Routes (http://routes.groovie.org/). This tends to prove that some people find it useful.
The proposed implementation, however, is another story. There are the problems that Graham noticed, plus the fact that the current publishing system is richer than the simple controller + action couple (you can buidl hierarchies of objects, have callable object anywhere in the hierarchy, etc.).
Also, the publisher supports skipping the extension part of the module, so you CAN have /controller/action URIs that already behave quite like what Sébastien suggests.Regards,Nicolas
2006/9/7, Jorey Bump <[EMAIL PROTECTED]>:
Sébastien Arnaud wrote:> > AddHandler mod_python .py .html> PythonHandler mod_python.pubre> PythonOption "pubregex"> "(?P[\w]+)?(\.(?P[\w]+))?(/(?P[^/]+))?(\?$)?"
>> >> I know that not all grammars will work with the current version attached> (due to some code being still dependent on the conservative url> structure /path/file.ext), eventually though, I hope I can get this
> solved and allow any regex grammar to work.>> Anyway, please share your comments and feedback to make sure I am headed> in the right direction by keeping in mind that my first goal is to be
> able to publish using a defined regex url grammar a callable class> within a module. I believe that once this first step is accomplished the> real design of the web framework can begin.Is regex essential to what you're trying to accomplish? I do something
similar in a minimal handler I've written that simply makes a list ofthe URL parts: resource = req.path_info.strip('/').split('/')Then I walk the list in order to determine the action that needs to be
taken. A very primitive example would be:if resource[0] == 'time': # use first item only, returns formatted local date & time # i.e. http://host/path/to/handler/time
import time now = time.strftime('%Y-%m-%d %H:%M:%S') req.content_type = "text/plain" req.write(now)elif resource[0] == 'color': # use second URL part as a key in a data structure
# i.e. http://host/path/to/handler/color/Lance data = "" 'blue', 'Robin': 'idunno', 'Galahad': 'yellow'}
req.content_type = "text/plain" try: req.write("%s's favorite color is %s." % (resource[1],data[resource[1]])) except KeyError: req.write("%s's favorite color is unknown." % (resource[1],))
except IndexError: req.write('Specify a name (/color/Someone).')Then http://host/handler/time returns: 2006-09-07 11:17:19and
http://host/handler/color/Lance returns: Lance's favorite color is blue.My motivation was to support friendly dynamic URLs(http://host/handler/foo/bar/baz
) in the simplest way possible. Sincethe action taken is handled in logic (instead of a strict mapping toeither functions, modules, or objects), it's extremely flexible. Thetrick is in determining if an argument should be part of the abs_path or
handled as a query (http://host/handler/foo?bar=baz).Of course, there's nothing stopping you from using the remainder of thelist as values for your app:
if resource[0] == 'order': # use remaining resources as multiple values for single argument # http://host/path/to/handler/order/item/item/item
if len(resource) > 2: # there was a list of items order = '%s and %s' % (', '.join(resource[1:-1]), resource[-1]) elif len(resource) == 2: # there was a single item
order = resource[1] else: # no items in URL order = 'nothing' req.content_type = "text/plain" req.write('You have ordered %s.' % (order))Then this URL:
http://host/handler/order/spam/spam/spam/eggs/spamwould return: You have ordered spam, spam, spam, eggs and spam.This isn't always appropriate, and one needs to be careful (too much
rope, as they say). It's normally better to handle arbitrarily longlists of values with query arguments or POST variables.Anyway, I don't mean to hijack your thread, Sébastien. I'm justwondering if you are trying to accomplish the same thing, or have a
specific reason for using regex, which really can slow an applicationdown. You could easily adapt this simpler approach to enforce a strictermapping of your own convention.
Re: Regex based publisher proposal
Sébastien Arnaud wrote:
AddHandler mod_python .py .html
PythonHandler mod_python.pubre
PythonOption "pubregex"
"(?P[\w]+)?(\.(?P[\w]+))?(/(?P[^/]+))?(\?$)?"
I know that not all grammars will work with the current version attached
(due to some code being still dependent on the conservative url
structure /path/file.ext), eventually though, I hope I can get this
solved and allow any regex grammar to work.
Anyway, please share your comments and feedback to make sure I am headed
in the right direction by keeping in mind that my first goal is to be
able to publish using a defined regex url grammar a callable class
within a module. I believe that once this first step is accomplished the
real design of the web framework can begin.
Is regex essential to what you're trying to accomplish? I do something
similar in a minimal handler I've written that simply makes a list of
the URL parts:
resource = req.path_info.strip('/').split('/')
Then I walk the list in order to determine the action that needs to be
taken. A very primitive example would be:
if resource[0] == 'time':
# use first item only, returns formatted local date & time
# i.e. http://host/path/to/handler/time
import time
now = time.strftime('%Y-%m-%d %H:%M:%S')
req.content_type = "text/plain"
req.write(now)
elif resource[0] == 'color':
# use second URL part as a key in a data structure
# i.e. http://host/path/to/handler/color/Lance
data = {'Lance': 'blue',
'Robin': 'idunno',
'Galahad': 'yellow'}
req.content_type = "text/plain"
try:
req.write("%s's favorite color is %s." % (resource[1],
data[resource[1]]))
except KeyError:
req.write("%s's favorite color is unknown." % (resource[1],))
except IndexError:
req.write('Specify a name (/color/Someone).')
Then http://host/handler/time returns:
2006-09-07 11:17:19
and http://host/handler/color/Lance returns:
Lance's favorite color is blue.
My motivation was to support friendly dynamic URLs
(http://host/handler/foo/bar/baz) in the simplest way possible. Since
the action taken is handled in logic (instead of a strict mapping to
either functions, modules, or objects), it's extremely flexible. The
trick is in determining if an argument should be part of the abs_path or
handled as a query (http://host/handler/foo?bar=baz).
Of course, there's nothing stopping you from using the remainder of the
list as values for your app:
if resource[0] == 'order':
# use remaining resources as multiple values for single argument
# http://host/path/to/handler/order/item/item/item
if len(resource) > 2:
# there was a list of items
order = '%s and %s' % (', '.join(resource[1:-1]), resource[-1])
elif len(resource) == 2:
# there was a single item
order = resource[1]
else:
# no items in URL
order = 'nothing'
req.content_type = "text/plain"
req.write('You have ordered %s.' % (order))
Then this URL:
http://host/handler/order/spam/spam/spam/eggs/spam
would return:
You have ordered spam, spam, spam, eggs and spam.
This isn't always appropriate, and one needs to be careful (too much
rope, as they say). It's normally better to handle arbitrarily long
lists of values with query arguments or POST variables.
Anyway, I don't mean to hijack your thread, Sébastien. I'm just
wondering if you are trying to accomplish the same thing, or have a
specific reason for using regex, which really can slow an application
down. You could easily adapt this simpler approach to enforce a stricter
mapping of your own convention.
Re: Regex based publisher proposal
On 07/09/2006, at 2:59 PM, Sébastien Arnaud wrote:
Anyway, please share your comments and feedback to make sure I am
headed in the right direction by keeping in mind that my first goal
is to be able to publish using a defined regex url grammar a
callable class within a module. I believe that once this first step
is accomplished the real design of the web framework can begin.
A few comments while I work out what your code actually does.
class Mapper:
""" This is the object to cache the regex engine """
regex = "(?P[\w]+)?(\.(?P[\w]+))?(/(?P
[^/]+))?(\?$)?"
regex_compared = 0
def __init__(self):
self.reobj = re.compile(self.regex)
def __call__(self, uri, cre):
if(cre!=None and not self.regex_compared and cre!=self.regex):
self.regex = cre
self.reobj = re.compile(self.regex)
self.regex_compared = 1
m = self.reobj.match(uri)
if m:
return (m.group('controller'), m.group('extension'), m.group
('action'))
else:
return (None, None, None)
mapper_cache = Mapper()
This is not thread safe and use in a multithreaded MPM, ie., winnt
and worker, may
result in failure. I also suspect if you would have problems where
two different parts
of the URL namespace use different regex's and they aren't executing
in different
Python interpreter instances.
path,module_name = os.path.split(req.filename)
# Trimming the front part of req.uri
if module_name=='':
req_url = ''
else:
req_url = req.uri[req.uri.index(module_name):]
This is not a very robust way of doing this can technically could
fail in certain cases.
# Now determine the actual Python module code file
# to load. This will first try looking for the file
# '/path/.py'.
req.filename = path + '/' + controller + '.py'
if not exists(req.filename):
raise apache.SERVER_RETURN, apache.HTTP_NOT_FOUND
I am not really sure why you go to all this trouble. For the way the
default regex
is written, this could possibly just as easily be achieved using
standard mod_python.publisher,
using subdirectories in document tree and use of MultiViews matching
in an appropriate
way.
In other words, am not convinced that your code is required at all
and you may be
able to achieve the same thing as default regex using standard
mod_python.publisher.
At worst case, you might need to use Apache RewriteRule. In
mod_python 3.3, you
could probably do all this with a very simple fixup handler as well.
Thus, as already requested, can you actually supply some examples of
how this is
used in practice.
BTW, you could also have done this by using a wrapper handler around the
existing mod_python.publisher handler as well, thereby avoiding
having to cut and
paste all the code.
Graham
Re: Regex based publisher proposal
On 07/09/2006, at 2:59 PM, Sébastien Arnaud wrote: Hi, I have been following with passion mod_python development for quite a while now, and in the light of a few emails over the past few months discussing web frameworks in mod_python, I decided I would attempt to contribute to the project in order to move towards a fast, flexible MVC mod_python only based web framework. I have written 2 or 3 different ones along the past couple of years, but nothing worthy of sharing by any mean. They have helped me however to define what would be the "dream" web framework for mod_python, but more importantly to identify the needed plumbing improvements to mod_python. One of the first needed improvements, in my opinion, is the capacity to route web requests in a more flexible manner than via the current publisher module. So, I would like to propose the following module (pubre.py). It is basically a copy of the mod_python.publisher module to the exception that a lot of the core handler code has been modified to use regex in order to route a web request to the appropriate module/function. I have been developing against mod_python/trunk and I attached the file for whoever wants to review it and give it a try. Keep in mind though it is still probably rough around the edges and not any solid testing has been performed yet. I only performed some trivial benchmarking/stress testing to make sure that performance wise it was on par with the current mod_python.publisher. The default behavior is suppose to be 100% compatible with the way mod_python.publisher behaves. Eventually though you would be able to pass as a PythonOption the grammar of the urls in your web application, by simply declaring something like: AddHandler mod_python .py .html PythonHandler mod_python.pubre PythonOption "pubregex" "(?P[\w]+)?(\.(?P [\w]+))?(/(?P[^/]+))?(\?$)?" I know that not all grammars will work with the current version attached (due to some code being still dependent on the conservative url structure /path/file.ext), eventually though, I hope I can get this solved and allow any regex grammar to work. Anyway, please share your comments and feedback to make sure I am headed in the right direction by keeping in mind that my first goal is to be able to publish using a defined regex url grammar a callable class within a module. I believe that once this first step is accomplished the real design of the web framework can begin. Can you provide an English description with examples of how your extension is used in practice so we don't have to reverse engineer it to work out what it does? Thanks. Graham
Regex based publisher proposal
Hi,
I have been following with passion mod_python development for quite a
while now, and in the light of a few emails over the past few months
discussing web frameworks in mod_python, I decided I would attempt to
contribute to the project in order to move towards a fast, flexible
MVC mod_python only based web framework.
I have written 2 or 3 different ones along the past couple of years,
but nothing worthy of sharing by any mean. They have helped me
however to define what would be the "dream" web framework for
mod_python, but more importantly to identify the needed plumbing
improvements to mod_python.
One of the first needed improvements, in my opinion, is the capacity
to route web requests in a more flexible manner than via the current
publisher module. So, I would like to propose the following module
(pubre.py). It is basically a copy of the mod_python.publisher module
to the exception that a lot of the core handler code has been
modified to use regex in order to route a web request to the
appropriate module/function.
I have been developing against mod_python/trunk and I attached the
file for whoever wants to review it and give it a try. Keep in mind
though it is still probably rough around the edges and not any solid
testing has been performed yet. I only performed some trivial
benchmarking/stress testing to make sure that performance wise it was
on par with the current mod_python.publisher.
The default behavior is suppose to be 100% compatible with the way
mod_python.publisher behaves. Eventually though you would be able to
pass as a PythonOption the grammar of the urls in your web
application, by simply declaring something like:
AddHandler mod_python .py .html
PythonHandler mod_python.pubre
PythonOption "pubregex" "(?P[\w]+)?(\.(?P[\w]
+))?(/(?P[^/]+))?(\?$)?"
I know that not all grammars will work with the current version
attached (due to some code being still dependent on the conservative
url structure /path/file.ext), eventually though, I hope I can get
this solved and allow any regex grammar to work.
Anyway, please share your comments and feedback to make sure I am
headed in the right direction by keeping in mind that my first goal
is to be able to publish using a defined regex url grammar a callable
class within a module. I believe that once this first step is
accomplished the real design of the web framework can begin.
Cheers!
Sébastien
#
# Copyright 2004 Apache Software Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you
# may not use this file except in compliance with the License. You
# may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing
# permissions and limitations under the License.
#
# Originally developed by Gregory Trubetskoy.
#
# $Id: publisher.py 384754 2006-03-10 10:20:06Z grahamd $
"""
This handler is conceptually similar to Zope's ZPublisher, except
that it:
1. Is written specifically for mod_python and is therefore much faster
2. Does not require objects to have a documentation string
3. Passes all arguments as simply string
4. Does not try to match Python errors to HTTP errors
5. Does not give special meaning to '.' and '..'.
"""
import apache
import util
import sys
import os
from os.path import exists, isabs, normpath, split, isfile, join, dirname
import imp
import re
import base64
import new
import types
from types import *
imp_suffixes = " ".join([x[0][1:] for x in imp.get_suffixes()])
### The published page cache ##
from cache import ModuleCache, NOT_INITIALIZED
class PageCache(ModuleCache):
""" This is the cache for page objects. Handles the automatic reloading of pages. """
def key(self, req):
""" Extracts the normalized filename from the request """
return req.filename
def check(self, key, req, entry):
config = req.get_config()
autoreload=int(config.get("PythonAutoReload", 1))
if autoreload==0 and entry._value is not NOT_INITIALIZED:
# if we don't want to reload and we have a value,
# then we consider it fresh
return None
else:
return ModuleCache.check(self, key, req, entry)
def build(self, key, req, opened, entry):
config = req.get_config()
log=int(config.get("PythonDebug", 0))
if log:
if entry._value is NOT_INITIALIZED:
req.log_error('Publisher loading page %s'%req.filename, apache.APLOG_NOTICE)
else:
req.log_error('Publisher reloading page %s'%req.filena
