Re: sys.path and htaccess change?
Jorey Bump writes: Graham Dumpleton wrote: The whole point of the changes which were made was to draw a well defined line between the code modules used in the web application and which reside in the document tree, or other specially specified areas by way of mod_python module importer path, and the standard Python sys.path locations. I think it's important to emphasize this point. I don't have a problem with making a distinction, either. But I do have a problem with behaviour like this changing (handler directory no longer in sys.path) between minor upgrades. I probably have users downloading mpservlets, launching the tutorial, saying to themselves %#@% piece of crap and moving on. That there was no distinction before is what caused all the issues with same named modules, loading of incorrect modules, reloading of modules which shouldn't be reloaded etc etc. And it's also important to point out that discussion of these problems has become virtually nonexistent, whereas they used to dominate the list. It was never clear to me that the problem was mod_python, but rather publisher. The novices coming to mod_python would go to the tutorial which pointed them to publisher and so in the mind of the novice: mod_python == publisher is True. As they used publisher and pushed its boundaries, importing issues became a major problem. I think it worthy to note that the swarm of importing questions rarely (ever?) involved psp or mpservlets, which had their own importers. I remember when I first brought up this idea of 'import' working in magic ways so things would just work as one expected, that there were at least one person who thought it was a silly idea. That could well have been me. I viewed the issues as limitations, not necessarily weaknesses, and knew what to expect once I refined my approach. I was worried that changing the module importer could introduce more issues and force users to include proprietary mod_python code in their modules (excluding handlers, which are a special case). I don't have a problem with mod_python tweaking import. It's well known by python programmers (well, those who've been around awhile) that this is a tool at their disposal. I would just like to see it more **boldly** stated that it does. This knowledge may not make a difference to those who are using mod_python indirectly (those coming to it to use publisher, psp, mpservlets, vampire, etc.), but to those who write and maintain these handlers, it's HUGE. Since I keep nearly all of my code outside of the document tree, I'm not necessarily the target beneficiary of the changes made to the module importer. But it has eliminated namespace collisions and reloading issues in my frontends, so I benefit from the improvements and agree that things are much better. 3.3 is easily my favorite version of mod_python. I have found it a very powerful tool to be able to co-locate mps files with regular python modules. In fact, I find it odd to separate them: why put part of my applications code in one place and install the rest in another? Mpservlets does not allow files beginning with an underscore to be requested (generates a 403 response), so it's great to be able to place modules (eg, named _utilities.py) in the same directory as the front-end mps files. The developer doesn't have to worry about installing pieces of their package in site-packages or some such place, so packaging is much easier. A prime example is the mpservlets tutorial: everything is in one directory, very self-contained and easy to install. All I can suggest is you try it by starting with the changes to mpservlets as I described and then simply see if your existing applications which use mpservlets still work. Since the release we have only had a few people even be slightly affected by the differences, plus all the complaints we used to get about problems with the old importer have pretty well vanished. I'm lucky enough that none of my applications are publicly distributed... Yes, you are. :-) Daniel's in a rough spot, though, because he has to support all of the versions of mod_python that are out there (within reason). While we may find it trivial, the average user is unlikely to upgrade from a version included in the platform distribution (which could be dangerous if other packages were tweaked to rely on the state of mod_python when it was packaged). It sounds like he will still need to include his old caching system and disable it when it's not needed. Hopefully, this won't be too hard or require a fork. I don't think there's really a problem with my importer co-existing with 3.3 (although it would make sense for me to piggyback off mod_python's, just like I do for sessions, cookies, etc., so I *do* want to use the 3.3 importer at some point). I just need to figure out how to fix the python module/mps file co-location problem in an elegant way. Dan
Re: sys.path and htaccess change?
Graham Dumpleton writes: Read: http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html Especially the area which starts just before: PythonOption mod_python.importer.path ['~'] Your particular issue is mentioned just after this example. Doesn't work, either in .htaccess or in Directory directives. This is what I have in my apache config: Directory /home/dpopowich/public_html/py SetHandler mod_python PythonHandler mod_python.servlet PythonOption mod_python.importer.path ['~'] PythonDebug on /Directory When I go to http://localhost/~dpopowich/py/syspath, a simple mpservlet that writes out the current value of sys.path, I do NOT see /home/dpopowich/public_html/py in the path. And when I try to import a python module in that directory, I get import errors. However, if I put this in my apache global context: PythonOption mod_python.legacy.importer * this fixes my immediate problem. Is this the way to completely turn this subsystem OFF? Dan PS As a side note: I've been away from mod_python for a while and now am getting back into things and thought one project would be to update mpservlets to use the new importer I've heard so much about instead of the one I wrote for mpservlets. I must confess I'm a bit taken aback that it takes over six pages to document this one function and a quick read has left me scratching my head and wondering if this is the way I want to go. I must confess I find it less than intuitive.
Re: sys.path and htaccess change?
On 18/05/07, Daniel J. Popowich [EMAIL PROTECTED] wrote:
Graham Dumpleton writes:
Read:
http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
Especially the area which starts just before:
PythonOption mod_python.importer.path ['~']
Your particular issue is mentioned just after this example.
Doesn't work, either in .htaccess or in Directory directives. This
is what I have in my apache config:
Directory /home/dpopowich/public_html/py
SetHandler mod_python
PythonHandler mod_python.servlet
PythonOption mod_python.importer.path ['~']
PythonDebug on
/Directory
When I go to http://localhost/~dpopowich/py/syspath, a simple
mpservlet that writes out the current value of sys.path, I do NOT see
/home/dpopowich/public_html/py in the path.
You will not see the path listed in sys.path. The mod_python module
importer path and the Python sys.path are deliberately kept separate
because of all the problems caused by each looking in the same
location.
And when I try to import
a python module in that directory, I get import errors.
If you are still using the old module importer in mpservlets to import
your .mps file then that would probably still occur.
However, if I put this in my apache global context:
PythonOption mod_python.legacy.importer *
this fixes my immediate problem.
Is this the way to completely turn this subsystem OFF?
Yes, but the old importer will disappear at some point in the future.
As a side note: I've been away from mod_python for a while and now am
getting back into things and thought one project would be to update
mpservlets to use the new importer I've heard so much about instead of
the one I wrote for mpservlets. I must confess I'm a bit taken aback
that it takes over six pages to document this one function
Considering the list of problems with the old importer came to about
12 pages, I thought it wasn't too bad. :-)
http://www.dscpl.com.au/wiki/ModPython/Articles/ModuleImportingIsBroken
and a quick
read has left me scratching my head and wondering if this is the way I
want to go. I must confess I find it less than intuitive.
Just change this in your mpservlets package. The new importer can
handle arbitrary extensions so can load .mps files no problem. With
just this change your tutorial example works with no problems and it
uses 'import' to get stuff from handler directory.
kundalini:/usr/local/src/mpservlets-1.1.6 grahamd$ diff
servlet.py.dist servlet.py
1269,1270c1269,1271
code = {}
execfile(fname, code)
---
#code = {}
#execfile(fname, code)
module = apache.import_module(fname)
1276c1277,1278
klass = code[basename]
---
#klass = code[basename]
klass = getattr(module, basename)
Graham
Re: sys.path and htaccess change?
I worked it out. As my change didn't disable the mpservlets caching
system, it was never triggering the module importer again to perform
its own checks. Thus, if I add:
klass, cached_mtime, servlets = self.class_cache.get(fname,
(None,None, []))
# XXX Pretend we didn't find it so mod_python importer used
klass, cached_mtime, servlets = (None,None, [])
# XXX
if not klass or mtime cached_mtime:
In other words, a bit more work is required in mpservlets as with the
new module importer as it shouldn't really be supplying its own
caching system as that then defeats the mod_python importers own
caching and reloading system.
BTW, if you have program that can display DOT graphs, the graph data
was produced using:
PythonLogHandler ./_loghandler.py
and the following as the log handler.
from mod_python import apache
def loghandler(req):
try:
output = file('/tmp/request.dot', 'w')
output.write(apache.modules_graph(1))
finally:
output.close()
return apache.OK
Seeing the relationships between your modules and the data presented
there can be helpful sometimes in seeing what is happening and working
out how the module importer works.
Graham
On 18/05/07, Graham Dumpleton [EMAIL PROTECTED] wrote:
On 18/05/07, Graham Dumpleton [EMAIL PROTECTED] wrote:
Just change this in your mpservlets package. The new importer can
handle arbitrary extensions so can load .mps files no problem. With
just this change your tutorial example works with no problems and it
uses 'import' to get stuff from handler directory.
kundalini:/usr/local/src/mpservlets-1.1.6 grahamd$ diff
servlet.py.dist servlet.py
1269,1270c1269,1271
code = {}
execfile(fname, code)
---
#code = {}
#execfile(fname, code)
module = apache.import_module(fname)
1276c1277,1278
klass = code[basename]
---
#klass = code[basename]
klass = getattr(module, basename)
Hmmm, although web pages are being displayed okay, there is something
strange about the scenario in which apache.import_module() is being
used which is meaning that depth searches for changes to child
resulting in reloading from root down isn't working. Thus, if changing
a module which isn't a root module, then I am still needing to restart
Apache which I shouldn't have to with the new module importer. :-(
For example, if I change _SitePage.py in attached graph, then the two
.mps files and _SitePage.py should be reloaded on next request to
either of the .mps files, but it isn't. Looks like I'll have to track
down why as must be missing something very subtle, or obvious, which
is causing the module importer not to behave as expected.
Graham
