Re: TLP Name
+1 Quetzalcoatl
Re: TLP Name
Gregory (Grisha) Trubetskoy wrote: PythonScript Pythonidae PyPache pythonalia Quetzalcoatl Asphyxia Scales Pythonistas PigeonPy Pungi I liked Pungi, but there is unfortunately another project out there with that name (https://hosted.fedoraproject.org/projects/pungi). Quetzalcoatl is a great Apache project name in that it's kind of related in a cultural memory kind of way to the project ideas, but obscure enough to be completely forgettable :) I guess Scales also falls into that category, and it gets my vote because it's easier to spell. Nick
Re: Constructing of a URL for location redirect.
Nicolas Lehuen wrote: BTW, did we ever considered using SWIG to map the Apache API ? I know it can be quite tricky to use, but it could be a real time saver. That's essentially what mod_snake did, and why I liked it so much. Though I don't remember if it used swig or pyrex. Nick
Re: Constructing of a URL for location redirect.
Gregory (Grisha) Trubetskoy wrote: SWIG in my opinion is good when you want some kind of an API made available to you quickly, but in a static environment where can put some thought into functionality, usability, Pythonic-ness of every approach, write documentation with good examples and a meaningful test case SWIG does not buy much. If someone just wanted to map the APR to Python - they can always use SWIG, but that's not what mod_python is about. All true; mod_python is a layer up from the apache API, so there's no way the SWIGed API could come close to replacing it. I just found it generally more convenient to work in Python rather than C, even if it's only a thin layer above that, and I think there is some merit to using it in some places, carefully wrapped in a Python class or module. That said, I don't really see mod_python changing so radically at this point in the game. It's fun to think about, though. :) In a masochistic kind of way. Nick
Re: mod_python 3.2.7 available for testing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 +1 Ubuntu 5.10 Breezy (amd64), Apache 2.0.54-worker, Python 2.4.2 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFD5/mTv4zJ7LQ+i84RAofQAKCb4ptmhPQa5QKRV/2sga60Xz4oAACcDygf IB8UDE0zlcUr+I16DWbQ09U= =WrUY -END PGP SIGNATURE-
Re: please set up a mod_python core group
Jim Gallacher wrote: Jorey Bump wrote: +1 here, but since the build process and typical MPM differs among platforms, could we see a list that this group represents? I'm most interested in default nonvirtualized environments used in production or for principal development. This information will be useful when reviewing release candidates, to make sure we haven't overlooked any key platforms. Your point on making sure we don't overlook any key platforms in our testing is a good one. Should we (python-dev people) put together a list of key platforms as a future guide? It's likely a good idea, even at the risk of a flamewar. ;) I thought I'd put together a summary of 3.2.6 test results in the next few days anyway, which should be a good starting point for the key list. As a non-x86 user (amd64 here), I second the notion that we need some non-Linux non-x86 platform testing out there, if people were willing to commit to be available to build and test when that time comes around (I think it's been pretty good, about every 2 months it's been on average?). I know there are people on PPC OSX, FreeBSD, AIX, Tru64, Solaris, and I just think it's a good idea to have a general concensus that a build will work on at least some of these platforms that both apache and Python are also supported and has worked for in the past. I'm not sure which of these you can identify as key, but I would say that *BSD, OSX and Solaris should top the list. I also suggest Linux x86_64 of some kind, since it's becoming more and more widely used; I know we've got 2 or 3 people that normally respond to release tests that do. Nick
Re: mod_python 3.2.6b available for testing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 +1 Linux (amd64) Ubuntu Breezy (5.10), apache 2.0.54 (mpm-worker), python 2.4 Jim Gallacher wrote: | A new mod_python 3.2.6 beta tarball is now available for testing. | Nicolas has built windows versions for Python 2.4 and Python 2.3 which | should also be available at www.modpython.org/dist shortly. | | This release is similar to 3.2.5b but fixes a couple of issues - | MODPYTHON-95, 96, 97, 98, 99, 105, 106. | | I think if we get enough +1 votes for this version on python-dev that we | should jump right to a 3.2.6 final release rather than go to a wider | beta release. | | So far we have: | | +1 Linux Debian sid, apache 2.0.55 (mpm-prefork), python 2.3 | +1 Linux Debian sarge, apache 2.0.54 (mpm-worker), python 2.3 | +1 Windows XP, python 2.4 | +1 Windows 2000, python 2.3 | +1 Mac OS X 10.3, Apache 2.0.55, python 2.3 | | Here are the rules: | | In order for a file to be officially announced, it has to be tested by | developers on the dev list. Anyone subscribed to this list can (and | should feel obligated to :-) ) test it, and provide feedback *to _this_ | list*! (Not the [EMAIL PROTECTED] list, and preferably not me | personally). | | The files are (temporarily) available here: | | http://www.modpython.org/dist/ | | Please download it, then do the usual | | $ ./configure --with-apxs=/wherever/it/is | $ make | $ (su) | # make install | | Then (as non-root user!) | | $ cd test | $ python test.py | | And see if any tests fail. If they pass, send a +1 to the list, if they | fail, send the details (the versions of OS, Python and Apache, the test | output, and suggestions, if any). | | Thank you, | Jim Ga -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDy83Fv4zJ7LQ+i84RAibhAKCylMw8FP9uvL4WyGQ6VMTyqCh03gCeM3AI vE+fdd6204+kXz2z0rHx1gY= =UjE3 -END PGP SIGNATURE-
Re: [jira] Commented: (MODPYTHON-93) Improve util.FieldStorage efficiency
Jim Gallacher wrote: Nick wrote: Just one comment. It seems like it would be better just to make add_method inline, since everything else in __init__ is, and it never gets called from anywhere else. add_method? Haha, thanks, I haven't had coffee yet. The add_item method, that is. :) I also like properties, but doesn't that cause a problem if someone chooses to subclass FieldStorage? It could if you didn't realize it was a property. But you can always override a property with another property. Nick
Re: [jira] Commented: (MODPYTHON-93) Improve util.FieldStorage efficiency
If you provide say FieldStorage.make_dict that returns a dictionary, then I
don't see why the order of the keys is important when the original list is
still available.
Nick
Nicolas Lehuen wrote:
Hi,
Speaking of ordered dictionary :
http://www.voidspace.org.uk/python/weblog/arch_d7_2005_11_19.shtml#e140
Why is the ordering so important ? I do understand we need to support
multiple values per field name, but I don't see why ordering is
needed.
Regards,
Nicolas
2005/11/28, David Fraser [EMAIL PROTECTED]:
Gregory (Grisha) Trubetskoy wrote:
Having looked at the FieldStorage code, I'm guessing the idea was that
you parse fields as they come in and append them to a list. This
preserves the original order of fields, in case it is needed.
I'm not sure that maintaining a dictionary alongside the list is the
right thing to do. It might be, but there are some difficult questions
to answer -e.g. how costly is a sequential search, and is the code
complexity (and fieldstorage code is no picnic to read as it is) worth
the speedup?
Also while it would speed up retrieval, it will slow down the write
operation - when a field is added to fieldstorage you now need to
append it to the list, AND check whether it exists in the dictionary,
then add it there as well.
How often do developers access form fields via __getitem__? I noticed
the publisher does not use it - it iterates the list, so nothing would
be gained there.
We do it a lot but we copy it into a different dictionary first to get
exactly the setup we want. But dictionary-style access is a very
obvious, pythonic way to do it.
I have a simple 70-line ordereddict implementation which is derived from
dict and remembers the keys in the order that they were assigned when
iterating through the list, this may be a way to go for this. It just
uses a list of keys internally to remember the order, and otherwise is a
dictionary...
Also, something else to consider - is there a simple programatic
solution that could be documented, e.g. something like
my_fs = util.FieldStorage(req)
dict_fs = {}
dict_fs.update(my_fs)
[have no idea whether this will work :-)]
It may work but still has the potential performance problem since it
loops through the keys and then does a getitem on each key which loops
through them again. Not likely to cause problems for a small number of
arguments but not ideal :-)
and voila - you've got a dictionary based fieldstorage?
Anyway, just a few cents from me.
Grisha
Re: 3.2b5 : mod_python doesn't handle properly signals likeKILL,SEGV...
for myself, I have not had problems on ubuntu 5.10 amd64. Nick Jim Gallacher wrote: Michel Jouvin wrote: Jim, I am not totally surprised... I am afraid this is a platform specific issue as we are running mod_python on Tru64. Something like a 64 bits issue. Does it sound a reasonnable possibility ? I have no idea what may be going on, but that seems as likely as anything else. How to progress in troubleshooting ? Again, no clue. :(. Hopefully some of the bigger brains that hang out around here will chime in. I know Indrek Järve tested 3.2.2b on SuSE Linux 9.2 (x86-64). Perhaps he or someone else with a 64-bit platform could try and reproduce the problem. That would tell us if it's 64-bit related or Tru64 related. I've attached my test script if anyone wants to mess with it. I'm sure I don't need to tell you to *not* run it on a production machine. ;) You'll likely want to change the PAUSE variable to something less than 30 seconds, which is the time between the kill calls. I was testing using qemu, and it needs lots of time for things to happen. usage: ./killchildren # number of loops Jim Michel --On jeudi 24 novembre 2005 17:41 -0500 Jim Gallacher [EMAIL PROTECTED] wrote: Michel, I can't reproduce the problem on debian i386. I put together a script that continually greps a apache child pid and kills it. After killing 200 processes there is no change in the total number of apache processes, and nothing in the apache log other an entry for each process killed: [Thu Nov 24 17:03:44 2005] [error] cgid daemon process died, restarting ... Regards, Jim Michel Jouvin wrote: I don't know If really need to write a script, this is so simple. asa/root % ps -e -opid,ppid,cmd | grep http 15601381048577 /www/Web/servers/apache/2.0.54/bin/httpd -k start 15601631560138 /www/Web/servers/apache/2.0.54/bin/httpd -k start 10863961086105 grep http From this output, you see that 1560163 is the child. Kill it with : kill -KILL 1560163 If you enter again 'ps -e|grep http', you'll see (I am seeing...) the number of httpd processes increasing until the max number (determined by MaxClient and ThreadPerChild). When this max number is reached you get the error message in main Apache error log. Michel --On mercredi 23 novembre 2005 19:30 -0500 Jim Gallacher [EMAIL PROTECTED] wrote: Michel Jouvin wrote: Graham, I played a little bit with worker MPM parameters. In particular I tested your suggestion to increase to 2 StartServers. This has no effect on the problem. I also tried to raise MaxSpareThread to MaxClient and suppressed child recycling (MaxRequestPerChild=0) to suppress restart of child as it seems to trig the problem with mod_pyton. No effect. I also checked the load during all these tests. Almost no request. So the heavy load syndroma you described doesn't seem to apply in this case. Again, one month ago I tested during 2 or 3 days an Apache configuration with mod_python loaded and without any url to trig its usages. And the problem was already the same. So it seems this is not related to mod_python usage (it happens even if you didn't execute any Python code) but rather to mod_python interaction with other Apache components. Michel Michel, I'm not able to reproduce the behaviour on debian stable (i386) with apache 2.0.54, but I'm not sure if I'm testing this correctly. Could you create a test script (bash or python) that will produce the error? That way I can know for sure that I'm testing in the same way. Jim * * Michel Jouvin Email : [EMAIL PROTECTED] * * LAL / CNRSTel : +33 1 64468932* * B.P. 34 Fax : +33 1 69079404* * 91898 Orsay Cedex * * France* * * * Michel Jouvin Email : [EMAIL PROTECTED] * * LAL / CNRSTel : +33 1 64468932* * B.P. 34 Fax : +33 1 69079404* * 91898 Orsay Cedex * * France* *
Re: mod_python 3.2.3b available for testing
Jim Gallacher wrote: So this is an inconsistency within Python. Should mod_python attempt to correct it, or just claim a Python bug? I think we should correct it. I'm sure users don't care that we implement this with TemporaryFile. That being said, I wonder how many applications on Windows we may break by fixing this? Version 3.1.4 also used TemporaryFile, so this is not a new bug. Yeah, I never noticed it either until someone pointed it out to me. I appreciated the change to TemporaryFile, but being primarily a Linux user I never noticed that this broke my code in Windows. In any case, I'm still gonna have to implement a workaround in my own code to catch people using the different versions of mod_python out there, so I can live with whatever decision you guys make. But here's +1 for making the interface consistent at least for mod_python users. As for code breakage, I would consider this a bug introduced in 3.1.4, which was the last official release of mod_python, which will be corrected in release 3.3. Nick
Re: mod_python 3.2.3b available for testing
Jim Gallacher wrote: You may have misunderstood. I was not suggesting that tempfile.TemporaryFile was introduced in 3.1.4, only that it existed there. Looking at the svn repository I see it's used in 3.0.0-beta and 2.7.9, so this bug has been lurking for a while. ;) Yes, although the fact that the implementation of TemporaryFile changed in Python 2.3 may have something to do with it. I honestly don't remember what the previous behavior was, but it worked OK for me at one time :) Nick
Re: glue between apache and python logging
Jim Gallacher wrote: Beyond that it still segfaults for me. The other problem is that you are not removing the handler instance from the logging instance so you still have a memory leak. 100k requests would result in 100k handler instances. Oh, and there might be a bit of a performance issue when emit() gets fired 100k times. ;) Or should I assume that the user will still be required to include req.register_cleanup(log.removeHandler, hdlr) in their code? Not sure if this will help, but in my implementation I registered my hander on the root logger as global code in the handler module, not for each request. In that case I used a threading.local to keep track of req, which I had to register and free for each request of course. I couldn't get around the minimal bookkeeping required of registering the req object on each request, though like Nic's code, I registered a cleanup in a closure to handle the freeing. Alternatively, you can register the server and use apache.log_error with a server object, which should not leak. Also, if you don't care about logging to a particular server, you can, of course, just call apache.log_error without a server argument. Nick
Re: glue between apache and python logging
Nic Ferrier wrote: Nick [EMAIL PROTECTED] writes: Nic Ferrier wrote: I just joined this list (at the suggestion of Graham Dumpleton) to try and get you guys to consider adding some glue to connect python 2.2 logging to Apache's logging. I have done this before, and although I haven't checked this code, it is as trivial to do as Nic makes it appear (my code was a bit different, but it did essentially the same thing). On the other hand, I'm not sure including this code fits with Grisha's philosophy of mod_python being what it is -- glue between Python and apache -- especially when this is such a simple exercise to anyone who has read the logging docs. But that's my point Nick, it's so simple as to be annoying when you have to do it (it certainly annoyed me when I came to mod_python). I agree, there are several bits of code that are like that. I really think the argument about mod_python being minimal does not apply here. Nick said it - mod_python is glue between Python and apache - well, making Python logging work directly to Apache seems like important glue to me. And again I renew my argument that there needs to be some kind of contrib archive that is probably separate from mod_python and unsupported in the core distribution. Maybe a wiki or code repository or something to support all the contributions like this. That way all these neat things can be available to people who are using mod_python, but not burden the development process of mod_python itself with issues concerning the constributions. Nick
Re: glue between apache and python logging
Nic Ferrier wrote: But it's difficult to change your mind if you say prove that logging is the most widely used logging available and then I'll think about it but I don't use it anyway because I wrote my own. Well, that's not what I said. What I said was: 1. I'm not convinced that a logger handler should be included in mod_python just because it's a standard module. A lot of packages out there that have logging facilities don't necessarily use logger. Even though it's part of the Python distribtion, that doesn't make it uniquitous in its use. mod_python exposes apache's logging facility in using interface that apache provides. I think that's the goal of mod_python, to (1) provide access to apache's API and (2) provide dispatching for handlers. 2. My counterargument to the logging interface as being essential is that I neither use the logging module nor want to use it. I tried to use it, but the implementation required a lot of overhead (mostly due to threading issues and access to the right req object). YMMV, but my experience led me to create a thinner wrapper around apache.log_error that was simply easier to use and maintain. That really has nothing to do with whether or not In terms of changing my mind, that has to do with whether or not I believe it should be part of the core mod_python distribution. And I still think it should not. I still think it falls under the umbrella of contributed code, just as if it was an implementation that utilized cgitb or SimpleXMLRPCServer, both of which are standard modules and arguably pretty darn useful in mod_python development. Please note that I also disagree with including the publisher and psp handlers as part of the core distribution, but, well, we can't turn back the clock on that ;-) And besides, I'm probably one of the few here who don't use them, making my opinion on that matter a minority view. Unfortunately I have to argue from the position that logging is officially part of the python language. Respectfully, I don't think that has anything to do with it. There's a lot that can be done with only the official parts of the Python language that aren't included in mod_python but left as an option for the user to implement. Recently a bid to include a WSGI implementation for mod_python was shot down, which is a Python PEP and something I thought would have been pretty neat to include. Regardless of Grisha's personal feelings on that (and I have a pretty good idea of what they are ;-)), I think it was ultimately the right decision for mod_python. Nick
Re: SQLite and other things [was Re: svn commit: r290569]
Gregory (Grisha) Trubetskoy wrote: I think that (and we can discuss this - I don't set laws, I just have opinions that may not always beright, so feel free to comment) mod_python should do fewer things but do them exceptionally well. I would agree with that totally. As a long time user of mod_python, I would add that I personally disagree with including Sessions, psp, publisher, etc. in the base distribution, since these things seem to have held up releases. I don't use any of those things as all with mod_python, and I don't see how they're necessary to use mod_python for what it's for: creating apache extension modules in Python. The thing we need to address is what to do with nifty things we create but that don't qualify for inclusion. The idea of a 'contrib' directory has been floated around for a while, I for one am against it for the same reasons above - it should either be 100% supported or not included at all IMO. If that's the case, there definitely needs to be a central place for contributed code like this. I think a contrib directory is convenient, but I also know the issues of who decides what to put in there, how much do put include, how do you gently waive off support requests for its contents. But there needs to be something. What do you see on the list all the time? People asking how do I implement database pooling, how to I use sessions, here's my database implementation for sessions. I say put the publisher there as well, since it doesn't have anything to do with the philosophy of mod_python, even by Grisha's definition, and solves some release issues with mod_python itself (can anyone say imports?). Nick
Re: flex [was mod_python 3.2.0-BETA available for testing]
Jim, I don't think it's too verbose, but maybe you could delay it to the end of the configure script so you don't have to either interrupt with control-C or scroll back to see what went wrong. Here's another idea: Fail the flex test fairly silently (e.g. just no), but fall back to a script that generates a nice, verbose error message explaining the situation. That way, when the user tries to call make after modifying the .l file, the fake flex alternative script gets called, displays the message, and exits with status 1. Nick Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: OK, here is the flex scoop - as the the docs point out, anything before 2.5.31 is not reentrant and I think even uses a slightly different interface so older flex won't even process the psp_parser.l file correctly. Looking at Fedora Core 4, it still has flex 2.5.4a. (Note that 2.5.31 2.5.4 because 31 4 - I for a while had trouble seeing that for some reason), so the new flex is still not commonplace. So until reentrant flex becomes commonplace, the solution was to include a pre-parsed psp_parser.c so that you woudln't need flex at all to compile mod_python. Looks like this still should be the case. The ./configure should just print a warning that if flex is not found or too old, should you need to rebuild psp_parser.c you will need to get the right version of flex. I've made the changes to configure.in, but before I commit I wanted to get some feedback. Are the following configure messages unclear or too verbose? For the case of the missing flex, ./configure will generate: ... checking for --with-flex... no checking for flex... no configure: WARNING: flex not found You can generally ignore this warning unless you need to regenerate psp_parser.c from psp_parse.l. If you do need regenerate psp_parser.c, use --with-flex to specify the location of flex. See the README for more information. ... For the case of the wrong flex version, ./configure will generate: ... checking for --with-flex... no checking for flex... /usr/local/sbin/flex found /usr/local/sbin/flex, we'll use this. Use --with-flex to specify another. checking flex version... configure: WARNING: Flex version 2.5.4 found. Version 2.5.31 or greater is required. You can generally ignore this warning unless you need to regenerate psp_parser.c from psp_parse.l. If you do need regenerate psp_parser.c, use --with-flex to specify the location of the correct flex version. See the README for more information. ... Any comments? Jim
Re: Session Benchmarks
Jim Gallacher wrote: Using bsddb3 would introduce new dependency for mod_python, so I don't know if it's a good idea to use transaction handling by default for DbmSession. Maybe we could offer a subclass? Starting with Python 2.3 this module is included in the standard python distribution as its bsddb module. Nick
Re: PythonSessionOption - a new apache directive for session configuration
How about an explicit None value to completely disable it? If you don't
want users on your site using it.
Nick
Jim Gallacher wrote:
So, any further thoughts / comments / objections to PythonSessionOption,
or shall I just check in the code?
Regards
Jim
Jim Gallacher wrote:
I've created a new apache directive called PythonSessionOption. This
would be used to configure session handling in the apache config file.
This data is accessed with a new request method,
req.get_session_options().
Although we could use the PythonOption directive instead of creating a
new one, I believe it's better to keep the session config data
separate so we don't need to worry about collisions with current user
code or configuration.
Typical Usage
-
In a test script mptest.py
def handler(req)
opts = req.get_session_options()
for k in sess_conf:
req.write('%s: %s' % (k,opts[k])
In Session.FileSession:
__init__(self,req,sid):
opts = req.get_session_options()
timeout = int(opts.get('timeout', DFT_TIMEOUT))
In an Apache config file:
VirtualHost 192.168.1.12:80
ServerAdmin [EMAIL PROTECTED]
ServerName example.com
DocumentRoot /var/www/
PythonSessionOption session FileSession
PythonSessionOption session_directory /var/lib/mod_python/sess
PythonSessionOption timeout 14400
PythonSessionOption lock 1
...
/VirtualHost
If there are no objections I'll commit the code. I have not refactored
Sessions.py to use the new configuration scheme just yet.
Regards,
Jim
