Quetzalcoatl
Just so that many of you do not wonder We are a TLP - where is it??? Everything is well. :) It's just turned out that selecting python.apache.org as the main destination for it is not something that everyone is in agreement on and quite a bit of discussion had to take place, and we're still in the process, but it will all hopefully be resolved soon now. Grisha
Re: TLP Name
Quetzalcoatl is the winner. Grisha On Tue, 22 May 2007, Rob Sanderson wrote: +1 Quetzalcoatl And ditto re pyFoo and kFoo. :) Rob On Tue, 2007-05-22 at 12:08 -0400, Jim Gallacher wrote: +1 Quetzalcoatl I think I voted 3 times for different things, so this is my *final* official vote. ;) I'm not a fan of pypache for the same reasons that others have stated. Projects that stick py, q, k or gnu on the front of the name have always bugged me. Jim Gregory (Grisha) Trubetskoy wrote: So far PyPache seems to be the winner. Terranium is second, Quetzalcoatl thrid. Let's say this vote closes on Wednesday, and please send in +1's for one name - statements like I like both blank and blank are hard to account for. On Fri, 18 May 2007, Clodoaldo wrote: +1 pypache 2007/5/17, Daniel J. Popowich [EMAIL PROTECTED]: +1 on pypache -- Clodoaldo Pinto Neto
Re: TLP Name
On Sun, 20 May 2007, Justin Erenkrantz wrote: FWIW, the full name of the TLP will be: Apache foo - so Apache PyPache doesn't roll off the tongue very well... Yes, Apache PyPache does sound a bit silly. My vote is then: +1 Quetzalcoatl Those who have not voted yet (there are hundreds of you, I know) PLEASE keep on sending in your votes. Grisha
Re: TLP Name
So far PyPache seems to be the winner. Terranium is second, Quetzalcoatl thrid. Let's say this vote closes on Wednesday, and please send in +1's for one name - statements like I like both blank and blank are hard to account for. On Fri, 18 May 2007, Clodoaldo wrote: +1 pypache 2007/5/17, Daniel J. Popowich [EMAIL PROTECTED]: +1 on pypache -- Clodoaldo Pinto Neto
Re: TLP Name
I like Quetzalcoatl and PyPache. Terrarium seemed nice at first, but then the ophidiophobia in me took over. (Nobody suggested ophis, btw..) On Wed, 16 May 2007, Gregory (Grisha) Trubetskoy wrote: So I think we've got (in no particular order): PythonScript Pythonidae PyPache pythonalia Quetzalcoatl Asphyxia Scales Pythonistas PigeonPy Pungi Would people (ANYONE here on the list, yes, that includes *YOU*) please respond with the one they like most and I will compile the results? Thanks! Grisha
TLP Name
So I think we've got (in no particular order): PythonScript Pythonidae PyPache pythonalia Quetzalcoatl Asphyxia Scales Pythonistas PigeonPy Pungi Would people (ANYONE here on the list, yes, that includes *YOU*) please respond with the one they like most and I will compile the results? Thanks! Grisha
Re: Core vote for 3.3.1 stable release
I just got back from Nicolas's country where my 'net access was a bit spotty, but everything else was great (but expensive) as always... :-) core +1 from me as soon as i get over the jetlag and catch up with things, i'll do the rest (we have all the core votes, right?) grisha On Thu, 1 Feb 2007, Nicolas Lehuen wrote: Following my tests on Windows, and knowing that 3.3.1 = 3.3.0b + a version number change, I give my +1 on the release. Regards, Nicolas 2007/2/1, Jim Gallacher [EMAIL PROTECTED]: I think we have sufficiently tested 3.3.1 and it is time for the core group to vote on a release. This vote is only open to the mod_python core group (Grisha, Nicolas, Graham and myself) and is binding. We need at least three +1 votes for the release to proceed. A -1 from any of the four voters will veto the release. And here is my vote: +1 Release 3.3.1 for General Availability Jim
ANNOUNCE: Mod_python 3.3.0b (Beta)
The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the 3.3.0b (Beta) release of mod_python. Version 3.3.0b of mod_python features several new functions and attributes providing better access to apache internals, as well as many bug fixes and various performance and security improvements. A detailed description of the changes is available in Appendix A of the mod_python manual, also available here http://www.modpython.org/live/mod_python-3.3.0b/doc-html/app-changes-from-3.2.10.html Beta releases are NOT considered stable and usually contain bugs. This release is intended to solicit widespread testing of the code. We strongly recommend that you try out your existing applications and experiment with new features in a non-production environment using this version and report any problems you may encounter so that they can be addressed before the final release. Preferred method of reporting problems is the mod_python user list [EMAIL PROTECTED] Mod_python 3.3.0b is available for download from: http://httpd.apache.org/modules/python-download.cgi For more information about mod_python visit http://www.modpython.org/ Regards, The Apache mod_python team.
Re: Core vote [Re: mod_python 3.3.0 beta available for testing]
Sorry it took a little long, 3.3.0b is out there, and the web page has been updated. The proposed announcement follows, I'll be sending it out some time over X-Mas, please respond before then if you think something needs to be added/changed: Subject: ANNOUNCE: Mod_python 3.3.0 Beta The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the 3.3.0 Beta release of mod_python. Version 3.3.0b of mod_python features several new functions and attributes providing better access to apache internals, as well as many bug fixes and various performance and security improvements. A detailed description of the changes is available in Appendix A of the mod_python manual, also available here http://www.modpython.org/live/mod_python-3.3.0b/doc-html/app-changes-from-3.2.10.html Beta releases are NOT considered stable and usually contain bugs. This release is intended to solicit widespread testing of the code. We strongly recommend that you try out your existing applications and experiment with new features in a non-production environment using this version and report any problems you may encounter so that they can be addressed before the final release. Preferred method of reporting problems is the mod_python user list [EMAIL PROTECTED] Mod_python 3.3.0b is available for download from: http://httpd.apache.org/modules/python-download.cgi For more information about mod_python visit http://www.modpython.org/ Regards, The Apache mod_python team. On Fri, 15 Dec 2006, Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: I concur - my +1 was for a beta +1 for 3.3.0 beta Jim grisha On Wed, 13 Dec 2006, David Fraser wrote: I'm not core but I think its good practice to officially release this as a beta to the wider community before making it an actual release. I didn't test it because I was waiting for the core vote :-) David Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: core +1 on releasing it into the wild grisha I'm not sure what we're voting on here, and I'm not sure what I meant by the next level either. :) Is this a vote to give 3.3.0 beta a wider release (apache mirrors and so on), or a vote to go right to the 3.3.0 final release? I'm +1 either way. As I recall we didn't get much additional testing as a result of uploading the betas to the apache mirrors in the past. The people most likely to chip in with testing are already here on python-dev. Having the 3.2.x stable branch certainly facilitated releasing quick fixes for 3.2. There is no reason why we can't do the same with 3.3, so my inclination is to do an official 3.3.0 final release now rather than later. Jim On Mon, 11 Dec 2006, Jim Gallacher wrote: Test results so far, FYI. How long shall we wait until we kick it up to the next level? - jim +1 FreeBSD 6.1, Apache 2.2.3 (mpm-prefork), Python 2.4.3,1 +1 Linux Debian 3.1 Sarge, Apache 2.0.54 (mpm-prefork), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.4.4 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.5 +1 Linux Fedora Core 5 (i386), Apache 2.2.2 (mpm-prefork), Python 2.4.3 +1 Linux Fedora Core 5 i386, Apache 2.2.2 (mpm-prefork), Python 2.4.3 +1 Linux Fedora Core 6, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Fedora Core 6 i386, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Fedora Core 6 x86_64, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Slackware 10.2, Apache 2.2.3 (mpm-prefork), Python 2.4.1 +1 Linux Ubuntu 6.0.6, Apache 2.0.55 (mpm-worker), Python 2.4.3 +1 Linux Ubuntu 6.10, Apache 2.0.55 (mpm-prefork), Python 2.4.4c1 +1 Mac OS X (Darwin 8.8.1), Apache 2.2.3 (mpm-prefork), Python 2.4.2 +1 Mac OS X (PowerPC), Apache 2.2.1 (mpm-worker), Python 2.3.5 (OS Supplied Version) +1 Mac OS X (PowerPC), Apache 2.0.55 (mpm-worker), Python 2.3.5 (OS Supplied Version) +1 Microsoft Windows XP, Apache 2.0.58 (mpm_winnt), Python 2.5 +1 Microsoft Windows XP, Apache 2.0.58 (mpm_winnt), Python 2.4 +1 Microsoft Windows XP, Apache 2.2.2 (mpm_winnt), Python 2.5 +1 Microsoft Windows XP, Apache 2.2.2 (mpm_winnt), Python 2.4
Re: Core vote [Re: mod_python 3.3.0 beta available for testing]
I concur - my +1 was for a beta grisha On Wed, 13 Dec 2006, David Fraser wrote: I'm not core but I think its good practice to officially release this as a beta to the wider community before making it an actual release. I didn't test it because I was waiting for the core vote :-) David Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: core +1 on releasing it into the wild grisha I'm not sure what we're voting on here, and I'm not sure what I meant by the next level either. :) Is this a vote to give 3.3.0 beta a wider release (apache mirrors and so on), or a vote to go right to the 3.3.0 final release? I'm +1 either way. As I recall we didn't get much additional testing as a result of uploading the betas to the apache mirrors in the past. The people most likely to chip in with testing are already here on python-dev. Having the 3.2.x stable branch certainly facilitated releasing quick fixes for 3.2. There is no reason why we can't do the same with 3.3, so my inclination is to do an official 3.3.0 final release now rather than later. Jim On Mon, 11 Dec 2006, Jim Gallacher wrote: Test results so far, FYI. How long shall we wait until we kick it up to the next level? - jim +1 FreeBSD 6.1, Apache 2.2.3 (mpm-prefork), Python 2.4.3,1 +1 Linux Debian 3.1 Sarge, Apache 2.0.54 (mpm-prefork), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.4.4 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.5 +1 Linux Fedora Core 5 (i386), Apache 2.2.2 (mpm-prefork), Python 2.4.3 +1 Linux Fedora Core 5 i386, Apache 2.2.2 (mpm-prefork), Python 2.4.3 +1 Linux Fedora Core 6, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Fedora Core 6 i386, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Fedora Core 6 x86_64, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Slackware 10.2, Apache 2.2.3 (mpm-prefork), Python 2.4.1 +1 Linux Ubuntu 6.0.6, Apache 2.0.55 (mpm-worker), Python 2.4.3 +1 Linux Ubuntu 6.10, Apache 2.0.55 (mpm-prefork), Python 2.4.4c1 +1 Mac OS X (Darwin 8.8.1), Apache 2.2.3 (mpm-prefork), Python 2.4.2 +1 Mac OS X (PowerPC), Apache 2.2.1 (mpm-worker), Python 2.3.5 (OS Supplied Version) +1 Mac OS X (PowerPC), Apache 2.0.55 (mpm-worker), Python 2.3.5 (OS Supplied Version) +1 Microsoft Windows XP, Apache 2.0.58 (mpm_winnt), Python 2.5 +1 Microsoft Windows XP, Apache 2.0.58 (mpm_winnt), Python 2.4 +1 Microsoft Windows XP, Apache 2.2.2 (mpm_winnt), Python 2.5 +1 Microsoft Windows XP, Apache 2.2.2 (mpm_winnt), Python 2.4
Core vote [Re: mod_python 3.3.0 beta available for testing]
core +1 on releasing it into the wild grisha On Mon, 11 Dec 2006, Jim Gallacher wrote: Test results so far, FYI. How long shall we wait until we kick it up to the next level? - jim +1 FreeBSD 6.1, Apache 2.2.3 (mpm-prefork), Python 2.4.3,1 +1 Linux Debian 3.1 Sarge, Apache 2.0.54 (mpm-prefork), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.4.4 +1 Linux Debian Sid, Apache 2.2.3 (mpm-worker), Python 2.5 +1 Linux Fedora Core 5 (i386), Apache 2.2.2 (mpm-prefork), Python 2.4.3 +1 Linux Fedora Core 5 i386, Apache 2.2.2 (mpm-prefork), Python 2.4.3 +1 Linux Fedora Core 6, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Fedora Core 6 i386, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Fedora Core 6 x86_64, Apache 2.2.3 (mpm-prefork), Python 2.4.4 +1 Linux Slackware 10.2, Apache 2.2.3 (mpm-prefork), Python 2.4.1 +1 Linux Ubuntu 6.0.6, Apache 2.0.55 (mpm-worker), Python 2.4.3 +1 Linux Ubuntu 6.10, Apache 2.0.55 (mpm-prefork), Python 2.4.4c1 +1 Mac OS X (Darwin 8.8.1), Apache 2.2.3 (mpm-prefork), Python 2.4.2 +1 Mac OS X (PowerPC), Apache 2.2.1 (mpm-worker), Python 2.3.5 (OS Supplied Version) +1 Mac OS X (PowerPC), Apache 2.0.55 (mpm-worker), Python 2.3.5 (OS Supplied Version) +1 Microsoft Windows XP, Apache 2.0.58 (mpm_winnt), Python 2.5 +1 Microsoft Windows XP, Apache 2.0.58 (mpm_winnt), Python 2.4 +1 Microsoft Windows XP, Apache 2.2.2 (mpm_winnt), Python 2.5 +1 Microsoft Windows XP, Apache 2.2.2 (mpm_winnt), Python 2.4
ANNOUNCE: Mod_python 3.2.10
The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the 3.2.10 release of mod_python. Mod_python 3.2.10 is considered a stable release, suitable for production use. Mod_python is an Apache HTTP Server module that embeds the Python language interpreter within the server. With mod_python you can write web-based applications in Python that will run many times faster than traditional CGI and will have access to advanced features such as ability to maintain objects between requests, access to httpd internals, content filters and connection handlers. The 3.2.10 release has many new features, feature enhancements, fixed bugs and other improvements over the previous version. 3.2.10 now works with Apache HTTP Server 2.2. See Appendix A of mod_python documentation for a complete list. Mod_python 3.2.10 is released under Apache License version 2.0. Mod_python 3.2.10 is available for download from: http://httpd.apache.org/modules/python-download.cgi More information about mod_python is available at: http://httpd.apache.org/modules/ Many thanks to Jim Gallacher, Graham Dumpleton, Nicolas Lehuen and everyone else who contributed to and helped test this release, without your help it would not be possible! Regards, Grisha Trubetskoy
Re: mod_python 3.2.10 core vote
The site's been update (both mopython.org and the download page), so 3.2.10 is out. I'll work on the announcement next. Grisha On Mon, 31 Jul 2006, Gregory (Grisha) Trubetskoy wrote: Core +1 from me. I will take care of the signing, etc, some time tomorrow. P.S. In order for you to be able to sign you need to meet in person someone (or probably more than one person) from ASF. ApacheCon is the best place, and members do not have to pay the conference fee (at least I think that it is still true) ;-) Grisha On Thu, 27 Jul 2006, Nicolas Lehuen wrote: Just to make sure I've reinstalled my Python 2.3 test environment... So even if I've already voted, I've got an additional +1 Windows 2000 Server SP4, Apache 2.0.58 (mpm-winnt), Python 2.3.5 Regards, Nicolas 2006/7/26, Nicolas Lehuen [EMAIL PROTECTED]: +1 too. 2006/7/26, Jim Gallacher [EMAIL PROTECTED]: I think it's time for a core vote on the 3.2.10 release, as no more test results have appeared since Saturday. This vote is for the mod_python core only (Jim, Graham, Grisha and Nicolas). I am: +1 release now Jim Test summary: +1 Fedora Core 5, Apache 2.2.0 (mpm-prefork), Python 2.4.3 +1 FreeBSD 6.1-RELEASE-p2 (i386), Apache 2.2.2(mpm-prefork), python-2.4.3 +1 Gentoo 2006.0 (x86_64), Apache 2.2.2 (mpm-prefork), python-2.4.3 +1 Linux Slackware 10.1, Apache 2.0.55 (mpm-prefork), Python 2.4.1 +1 Linux Debian Sid, Apache 2.0.55 (mpm-prefork), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.0 (mpm-worker), Python 2.4.2 +1 Linux Ubuntu 6.06 Dapper Drake, Apache 2.0.55 (mpm-worker), Python 2.4.3 +1 MacOSX 10.4.7 Intel, Apache 2.0.55 (mpm-prefork), Python 2.4.3 +1 MacOSX 10.4.7 PPC, Apache 2.2.1 (mpm-prefork), Python 2.3.5 +1 MacOSX 10.4.7 PPC, Apache 2.2.1 (mpm-worker), Python 2.3.5 +1 Windows XP SP2, Apache 2.0.58 (mpm-winnt), Python 2.4.3
Re: mod_python 3.2.10 core vote
Core +1 from me. I will take care of the signing, etc, some time tomorrow. P.S. In order for you to be able to sign you need to meet in person someone (or probably more than one person) from ASF. ApacheCon is the best place, and members do not have to pay the conference fee (at least I think that it is still true) ;-) Grisha On Thu, 27 Jul 2006, Nicolas Lehuen wrote: Just to make sure I've reinstalled my Python 2.3 test environment... So even if I've already voted, I've got an additional +1 Windows 2000 Server SP4, Apache 2.0.58 (mpm-winnt), Python 2.3.5 Regards, Nicolas 2006/7/26, Nicolas Lehuen [EMAIL PROTECTED]: +1 too. 2006/7/26, Jim Gallacher [EMAIL PROTECTED]: I think it's time for a core vote on the 3.2.10 release, as no more test results have appeared since Saturday. This vote is for the mod_python core only (Jim, Graham, Grisha and Nicolas). I am: +1 release now Jim Test summary: +1 Fedora Core 5, Apache 2.2.0 (mpm-prefork), Python 2.4.3 +1 FreeBSD 6.1-RELEASE-p2 (i386), Apache 2.2.2(mpm-prefork), python-2.4.3 +1 Gentoo 2006.0 (x86_64), Apache 2.2.2 (mpm-prefork), python-2.4.3 +1 Linux Slackware 10.1, Apache 2.0.55 (mpm-prefork), Python 2.4.1 +1 Linux Debian Sid, Apache 2.0.55 (mpm-prefork), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.0 (mpm-worker), Python 2.4.2 +1 Linux Ubuntu 6.06 Dapper Drake, Apache 2.0.55 (mpm-worker), Python 2.4.3 +1 MacOSX 10.4.7 Intel, Apache 2.0.55 (mpm-prefork), Python 2.4.3 +1 MacOSX 10.4.7 PPC, Apache 2.2.1 (mpm-prefork), Python 2.3.5 +1 MacOSX 10.4.7 PPC, Apache 2.2.1 (mpm-worker), Python 2.3.5 +1 Windows XP SP2, Apache 2.0.58 (mpm-winnt), Python 2.4.3
Core Vote (Re: mod_python 3.2.9 available for testing)
Sorry for the late response - I was trying to have a vacation - that's when you are geographically in a different place with slow internet access and read only some of your e-mail ;-) +1 for core vote (with the note about the 2.2.2 XP SP2 issue). Grisha On Sat, 8 Jul 2006, Graham Dumpleton wrote: On 08/07/2006, at 4:26 AM, Jim Gallacher wrote: Hi Grisha, Here is the tally: +1 FreeBSD 6.1-RELEASE-p2, Apache 2.2 (mpm-prefork), Python 2.4.3 +1 Linux Debian Sid, Apache 2.0.55 (mpm-worker), Python 2.3.5 +1 Linux Debian Sid, Apache 2.2.0 (mpm-worker), Python 2.4.2 +1 Linux Fedora Core 5 (i386), Apache 2.2.0 (mpm-prefork), Python 2.4.3 +1 Linux Slackware 10.2, Apache 2.0.55 (mpm-prefork), Python 2.4.1 +1 Linux Ubuntu 6.06, Apache 2.0.55 (mpm-worker), Python 2.4.3 +1 MacOSX 10.4.6 PPC, Apache-2.0.55 (mpm/worker), Python-2.3.5 +1 MacOSX 10.4.6 PPC, Apache-2.2.1 (mpm/worker), Python-2.3.5 +1 MacOSX 10.4.7 Intel, Apache-2.0.55 (mpm/prefork), Python-2.4.2 +1 Windows XP SP2, Apache 2.0.58 (mpm_winnt), Python 2.4.3 -1 Windows XP SP2, Apache 2.2.2 (mpm_winnt), Python 2.4.3 The -1 was from Nicolas, with the following comment: Only two tests fail but with a segfault, it's test_srv_register_cleanup and test_apache_register_cleanup. This is not really surprising... I think we should go ahead and release the 3.2.9 version, while filing a known bug regarding the fact that we drop the support for those two functions. If we accept this, then it's a +1. The issue with server cleanups failing and why is covered by: http://issues.apache.org/jira/browse/MODPYTHON-109 The last test results were submitted July 1, so I think we may as well have a core vote. Jim Gregory (Grisha) Trubetskoy wrote: I'm barely keeping my head above water right now with work, so not really following the list - if someone could please ping me when/if you think we're ready for the core group vote and we have a tally. Thanks! Grisha -- Forwarded message -- Date: Sat, 01 Jul 2006 23:18:05 -0400 From: Jorey Bump [EMAIL PROTECTED] To: python-dev@httpd.apache.org Subject: Re: mod_python 3.2.9 available for testing +1 Linux Slackware 10.2, Apache 2.0.55 (mpm-prefork), Python 2.4.1 Jim Gallacher wrote: The mod_python 3.2.9 tarball is available for testing. This tarball is unchanged from 3.2.9-rc3, but should be retested anyway - just in case something went pair-shaped in the process of tagging and packaging. Here are the rules: In order for a file to be officially announced, it has to be tested by developers on the dev list. Anyone subscribed to this list can (and should feel obligated to :-) ) test it, and provide feedback *to _this_ list*! (Not the [EMAIL PROTECTED] list, and preferably not me personally). The files are (temporarily) available here: http://people.apache.org/~jgallacher/mod_python/dist/ http://people.apache.org/~jgallacher/mod_python/dist/ mod_python-3.2.9.tgz http://people.apache.org/~jgallacher/mod_python/dist/ mod_python-3.2.9.tgz.md5 Please download it, then do the usual $ ./configure --with-apxs=/wherever/it/is $ make $ (su) # make install Then (as non-root user!) $ cd test $ python test.py And see if any tests fail. If they pass, send a +1 to the list, if they fail, send the details (the versions of OS, Apache, Apache-mpm, Python, the test output, and suggestions, if any). Please present your test results in the following format: +1 OS version, Apache version (apache mpm), Python Version For example: +1 Linux Debian Sid, Apache 2.0.55 (mpm-worker), Python 2.3.5 Presenting your information in a consistent format will help in tabulating the results. You can include additional information in each section, just don't use extra commas. There is no need to include the mod_python version in this string as that information is available in the email subject. Who knows, one day I may actually write a script to extract this information automatically. :) Thank you for your assistance, Jim Gallacher
Re: release 3.2.10?
I'm +1 on going for 3.2.10. You in Canada probably have it easier - I think we hit 96F/35C at some point today or yesterday (I wouldn't know I'm in the office which has AC sunrise to sunset, I just listen to the news), and unfortunately (or not) due to work pressures I have no time for anything - no frolicking for me this summer :-( Grisha On Sun, 16 Jul 2006, Jim Gallacher wrote: Graham Dumpleton wrote: Jim Gallacher wrote .. Shall we proceed with a 3.2.10 release with the current memory leak fixes, or keep digging for more leaks? Seeing as it's summer for most of us (except for Graham), I get the feeling people don't have a lot of free time to spend on mod_python right now. Just because it is winter down here and I'm freezing my butt off doesn't mean I am any less busy. :-( Oh, I wasn't suggesting you were any less busy - just that folks in the northern hemisphere may well choose to frolic in the summer sun rather that hack away at mod_python in their spare time. :) Jim
Re: release 3.2.10?
On Tue, 18 Jul 2006, Jim Gallacher wrote: For 3.2.9 I called for 2 rounds of testing: one for the release candidate and one for the final tarball. Do folks here feel that is necessary for 3.2.10 or should I just jump right to the 3.2.10 final? That tarball would still be subject to a vote on this list before an official release. Cutting out the first step will save a few days. I think what you term it doesn't matter - if it's voted down, it will be voted down, be it 3.2.10 RC or 3.2.10 Final (we'll just have to make a 3.2.11 then). So just go for it :-) Grisha
new mod_python faq (fwd)
This was sent to me directly, anyone willing to act on it? (I don't have
the CPU cycles right now).
-- Forwarded message --
Date: Mon, 17 Jul 2006 18:12:07 +0200
To: [EMAIL PROTECTED]
Subject: new mod_python faq
I think may be useful to add this question:
2.30. Why do I get No space left on device: Failed to create global
mutex ?
Because mod_python has get all the available semaphores of your os.
To avoid this error on GNU/Linux do
#ipcs (to see all locked semaphores)
#for i in `ipcs | grep www-data | awk '{print $2}'`; do ipcrm -s $i;
done
www-data is the user which runs apache[12] (in Debian)
In this way you release all resources locked by apache[12]
If you want to increase the number of available semaphores append
kernel.sem = 512 32000 100 512
to your /etc/sysctl.conf
and do
#sysctl -p
--
Some of the content has benne derived from
http://www.modpython.org/pipermail/mod_python/2005-April/017858.html
You can change anything you want of course :)
Thank you for the great work
(even if configuring mod_python in debian is painfully... )
Re: Auto updating of req.finfo when req.filename changed.
On Sun, 26 Mar 2006, Graham Dumpleton wrote: One use for it that I already have is to get around the DirectoryIndex problems in mod_python caused by Apache's use of the ap_internal_fast_redirect() function to implement that feature. The specifics of this particular issue are documented under: http://issues.apache.org/jira/browse/MODPYTHON-146 Could we zoom in this a little bit. I've read the description, but not quite sure I understand it quite yet. Is the problem that if I set req.notes['foo'] = 'bar' in a phase prior to fixup, by the time we get to the content handler, it will be gone because notes would be overwritten by mod_dir? Grisha
Re: Auto updating of req.finfo when req.filename changed.
I'm -1 on updating req.finfo when req.filename is updated. I don't have the time to explain it in detail, but I think it flows from Graham's explanation below. Basically, httpd does a stat() for you, and its result is stored in finfo. Why make it any more complicated and magical? If you alter req.filename - should that imply another stat()? I don't think so, because the stat() is the most expensive operations in the whole request service process and unnecessary stats should be avoided. I also don't see any good use case for it. If you need a stat - Python provides way for you to do it, there is no need to get httpd involved in this process. I, of course, may be missing something obvious here, but this is my initial reaction. Let httpd/apr behave the way they behave - there is no need to pythonize them, especially when the Python standard library provides the same functionality in a pythonic way. Grisha On Sun, 26 Mar 2006, Jim Gallacher wrote: Hi Graham, +1 auto update req.finfo when req.filename changed. Is there a use case where the user might change filename but not want finfo to change? I can't think of one, so let's save the user some work and make their code more robust to boot. A point I'd like to address is your concern about mod_python differing from the Apache C api in implementing certain features. Personally I think our general concern about this is somewhat misguided. I suspect that the majority of our users are more concerned about the python-ness of mod_python rather than its apache-ness, and most would never notice if we deviate slightly from the apache C api. Adding a method or attribute to the request object for example, if it's useful to *our* users, shouldn't be rejected out of hand just because it does not exist in the underlying api. Jim Graham Dumpleton wrote: Now the mailing list is a bit quiet, I would like to see if I can get some explicit feedback on some issues related to the inability to update the req.finfo attribute. Grisha, would be nice if you could respond on this issue and give some guidance else I fear I'll never be able to progress a solution to this issue. :-( As explained in: http://issues.apache.org/jira/browse/MODPYTHON-128 although it is possible to assign a new value to req.filename, there is no way to update req.finfo to the file stats associated with that new value of req.filename. If one had access to the low level C API this would normally be achieved using: apr_stat(r-finfo, r-filename, APR_FINFO_MIN, r-pool); In mod_python though, there is no way to access the function and affect that outcome. In mod_perl 1.0 they implemented the behaviour whereby the finfo attribute was automatically updated when the filename attribute was updated by a handler. In mod_perl 2.0 they dropped this though, as they wanted to preserve the idea that in mod_perl everything behaved exactly like the C API they were trying to provide a 1 to 1 mapping for. Thus in mod_perl 2.0 you need to write: use Apache2::RequestRec (); use APR::Finfo (); use APR::Const -compile = qw(FINFO_NORM); $r-filename($newfile); $r-finfo(APR::Finfo::stat($newfile, APR::Const::FINFO_NORM, $r-pool)); As mod_python isn't attempting to provide a strict 1 to 1 mapping, it might be argued that it could do what mod_perl 1.0 did and automatically updated the finfo attribute when filename is updated. The only other alternative is to add a new method to the Python request object for which there isn't strictly speaking a direct equivalent to in the Apache C API. That is, a method that calls apr_stat() but which only performs it in relation to the filename and finfo attributes in the request object itself and is not a generic routine. Since it isn't likely that mod_python will ever provide a lower level API for use of finfo related structures and functions and even if it did they most likely would be distinct to the request object, the name of the function added to the request object could still be called stat(). Thus mod_python equivalent to what mod_perl 2.0 does would be: req.filename = newfile req.stat() This though doesn't really convey a sense of what it occurring. Thus a more descriptive name would probably be more appropriate. For example: req.filename = newfile req.update_finfo() There is no ap_update_finfo() function now, but if they did later implement one, this would shadow it and prevent it being added to the request object if it was pertinent for that be done. The next problem is that apr_stat() actually takes an argument indicating what fields in the finfo attribute should be updated. In mod_perl 1.0 the value used when the automatic update was done was APR_FINFO_MIN which results in type, mtime, ctime, atime, size being updated. In the documentation for mod_perl 2.0 it suggests use of APR_FINFO_NORM instead which is described as intended to be used when an atomic unix apr_stat() is required whatever that means. Important
Cross-platform query: _FILE_OFFSET_BITS in python and httpd
Could folks with access to different OS's try the following: Compare output of apxs -q CPPFLAGS with the value of _FILE_OFFSET_BITS in pyconfig.h. For example, on my Fedora Core 4 i386 system (stock httpd and python): $ /usr/sbin/apxs -q CPPFLAGS -DSSL_EXPERIMENTAL_ENGINE [note - no mention of _FILE_OFFSET_BITS above] $ grep _FILE_OFFSET_BITS /usr/include/python2.4/pyconfig.h #define _FILE_OFFSET_BITS 64 In case you're wondering, this is in relation to https://issues.apache.org/jira/browse/MODPYTHON-138 and to some degree https://issues.apache.org/jira/browse/MODPYTHON-20 and probably a few other unexplained issues. What the output on Fedora Core 4 means is that essentially Python and APR/httpd are compiled in an incomatible way - in APR the size of an inode (ino_t) is 32 bits and in Python it is 64 bits (this is what _FILE_OFFSET_BITS 64 does). This issue goes unnoticed when Python.h is included after http.h, but becomes very obvious if you put Python.h before http.h - httpd will segfault on the first request because the request_rec (which includes finfo, which includes ino_t inode) becomes incompatible between httpd and mod_python and anything past finfo in request_rec structure is junk (off by 4 bytes). I wanted to see how widespread this problem is. I think the right solution is for configure to catch this (exactly how to best detect this I'm not yet sure) and stop cold. Thanks, Grisha
Re: get_session(), req.session, req.form and MODPYTHON-38
On Mon, 13 Mar 2006, Graham Dumpleton wrote: Thus I want a documented convention that if a handler is going to use util.FieldStorage, that it should before doing so, first check whether an existing instance resides as req.form and use that instead. I'm not sure if this is a good example - req.form is something specific to the publisher. Rather than perhaps documenting it as you suggest, util.FieldStorage can take it upon itself to create a req.form so that subsequent attempts to instantiate it just return req.form. (This is just an example, I'm not 100% sure that I having FS do this makes sense - seems like a good idea). Similarly, if a handler is going to create a Session object, that it look for an existing instance as req.session and again use that instead. OR, the Session module would know to look for a req.session, in which case the handlers wouldn't need to worry about it. (One thing to watch out for would be that mutliple concurrent sessions in the same request is a possibility) Grisha
Re: get_session(), req.session, req.form and MODPYTHON-38
On Mon, 13 Mar 2006, Jim Gallacher wrote: The idea of something like req.get_session() is to give users an obvious way to grab a session object without the deadlock concerns. How many times have we seen this kind of problem-code on the mailing list? def index(req): sess = Session.Session(req) do_stuff(req) ... def do_stuff(req): sess = Session.Session(req) ... do other stuff. Having the session constructor check for the existence of req.session is of no use here. We need a way to make sure only *one* session instance is created per request. (Bonus marks for making it work with internal redirect). [sorry, i only read the beginning of the message, so i might be not fully understanding] Session.Session is not a constructor, just a function. But also, if it were, I think this can be solved with the new object's __new__() ? Grisha
Re: [jira] Assigned: (MODPYTHON-59) Add get_session() method to request object
I'm -1 on get_session() too. The request object is supposed to be a representation of Apache's request, and get_session() just does not belong there. Grisha On Sun, 12 Mar 2006, Jim Gallacher wrote: I handn't really intended to start working on an implementation. I just don't like seeing all those issues in JIRA marked as unassigned. :) Since I created it I figured I should take some responsibility for it. Plus, it's a gentle reminder when I list my assigned issues - resolve it one way or another. I still think we need some sort of solution to the problem of people trying to create 2 session instances in the same request, but I agree that the original concept of req.get_session() was not quite right. Jim Graham Dumpleton wrote: I would rather we not go ahead with adding req.get_session() at this time. At least not how it was envisaged to be done previously. I'll come back with a bit of analysis after I review where we were up to previously. Graham On 12/03/2006, at 8:47 AM, Jim Gallacher (JIRA) wrote: [ http://issues.apache.org/jira/browse/MODPYTHON-59?page=all ] Jim Gallacher reassigned MODPYTHON-59: -- Assign To: Jim Gallacher Add get_session() method to request object -- Key: MODPYTHON-59 URL: http://issues.apache.org/jira/browse/MODPYTHON-59 Project: mod_python Type: New Feature Components: session Versions: 3.1.4, 3.1.3, 3.2.7 Environment: All Reporter: Jim Gallacher Assignee: Jim Gallacher Fix For: 3.3 Attachments: Session.py.diff.txt Users will get session instances by calling req.get_session(). If a session already exists it will be returned, otherwise a new session instance will be created. Session configuration will be handled using apache directives rather than within their code. Using this scheme means only one session instance will be created per request, which will eliminate the deadlock problems many people experience. Also, using this scheme makes it possible for sessions to be properly handled within psp pages and across req.internal_redirect() calls. Code will be commited to svn shortly. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Vote on whether to integrate server side include (SSI) support.
I don't understand this enough to have an opinion on it, seems like
another way to skin a cat, but to really form an opinion would require
some thinking on my part at least (may be I'm getting thick with age).
I think it'd be great if those who send in +1's (or -1's) would explain
why they think this is good, and even if it's not so useful, then is it
worth being supported and maintained in the future.
Grisha
On Fri, 10 Mar 2006, Jim Gallacher wrote:
+1
Graham Dumpleton wrote:
I have had patches for adding server side include support into
mod_python ready for a while now. See:
https://issues.apache.org/jira/browse/MODPYTHON-104
In short, it would add the ability to add Python code into files
being served up through the INCLUDES output filter. More
commonly this is known as server side include (SSI). For example:
!--#python exec= from mod_python import apache import cgi import sys
parts = apache.import_module('parts') def _escape(object): return
cgi.escape(str(object)) -- html body pre !--#python
eval=_escape(str(globals().keys()))-- !--#python
eval=_escape(str(locals().keys()))-- !--#python exec= print filter
for key in filter.req.subprocess_env: print filter, _escape((key,
filter.req.subprocess_env[key])) -- !--#python
eval=parts.content()-- /pre /body /html
One could say that there is an overlap between this and the
mod_python.psp handler, but the SSI feature is a builtin feature of
Apache and it make sense to support it. Using SSI, if one was mad
enough, you could even have Python and Perl code appearing in the one
file.
Anyway, the point of this email is to get a decision on whether this
major new feature should or should not be added into mod_python.
Core developer votes obviously matter the most, but others are more
than welcome to voice an opinion.
So, is it a Yes or a No?
Graham
[ANNOUNCE] Mod_python 3.2.8 (security)
The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the release of version 3.2.8 of mod_python. This release addresses a vulnerability in mod_python's FileSession object whereby a carefully crafted session cookie could potentially permit an attacker to execute code on the server. FileSession was introduced in mod_python 3.2.7 released on February 15 2006 and is not enabled by default, therefore only a very small number of installations, if any, are likely to be affected by this issue. There are no other changes or improvements from the previous version in this release. Mod_python is available for download from: http://httpd.apache.org/modules/python-download.cgi For more information about mod_python visit http://www.modpython.org/ Regards, Gregory Trubetskoy
[DRAFT] [ANNOUNCE] Mod_python 3.2.8 (security)
If you see any problems with this text, let me know. -- Forwarded message -- Date: Sat, 12 Feb 2005 22:00:56 -0500 (EST) From: Gregory (Grisha) Trubetskoy [EMAIL PROTECTED] To: announce@httpd.apache.org, [EMAIL PROTECTED] Cc: python-dev@httpd.apache.org Subject: [ANNOUNCE] Mod_python 3.2.8 (security) The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the release of versions 3.2.8 of mod_python. This release addresses a vulnerability in mod_python's FileSession object whereby a carefully crafted session cookie could potentially permit an attacker to execute code on the server. FileSession was introduced in mod_python 3.2.7 released on February 15 2006 and is not enabled by default, therefore only a very small number of installations, if any, are likely to be affected by this issue. There are no other changes or improvements from the previous version in this release. Mod_python is available for download from: http://httpd.apache.org/modules/python-download.cgi For more information about mod_python visit http://www.modpython.org/ Regards, Gregory Trubetskoy
Re: 3.2.8 summary / core group vote
they're out there: http://www.apache.org/dist/httpd/modpython/win/3.2.8/ On Tue, 21 Feb 2006, Nicolas Lehuen wrote: Hi Grisha, Could you also put the Win32 binaries and make sure they are referenced on the download page ? We regularly have questions about where those binaries are, and I end up serving the files from my personal hosting solution, which is not really tailored for that. You can find the binaries here : http://nicolas.lehuen.com/download/mod_python TIA and regards, Nicolas 2006/2/21, Gregory (Grisha) Trubetskoy [EMAIL PROTECTED]: Resolved, the files have been placed on www.apache.org/dist, allowing time for mirror sync, then the web page update, then an announcement. Grisha On Mon, 20 Feb 2006, Graham Dumpleton wrote: +1 core vote Nicolas Lehuen wrote .. +1 core vote 2006/2/20, Jim Gallacher [EMAIL PROTECTED]: +1 core vote Jim Gregory (Grisha) Trubetskoy wrote: Based on summary below, +1 from for putting it out there. Grisha Graham Dumpleton [EMAIL PROTECTED] +1 Mac OS X 10.4, apache 2.0.55 mpm-worker, python 2.3.5 (3.2.x SVN trunk) Nicolas Lehuen [EMAIL PROTECTED] +1 mod_python 3.2.8 + Apache/2.0.55 + Python/2.2.3 + Windows 2000 SP4 +1 mod_python 3.2.8 + Apache/2.0.55 + ActivePython/2.3.5 + Windows 2000 SP4 +1 mod_python 3.2.8 + Apache/2.0.55 + ActivePython/2.4.2 + Windows XP SP2 Barry Pederson [EMAIL PROTECTED] +1 FreeBSD 6.0, apache 2.0.55 mpm-prefork, python 2.4.2 Jim Gallacher [EMAIL PROTECTED] +1 Debian sid, apache 2.0.55 mpm-prefork, python 2.3.5
Re: How mod_python treats apache.OK/apache.DECLINED response fromhandlers.
If I understand this correctly, then +1. ...though I'm wondering if anyone will actually try to do something as arcane as dynamicaly registering non-content handers? :-) Grisha On Tue, 21 Feb 2006, Jim Gallacher wrote: Nice summary. +1 for the change. Jim Graham Dumpleton wrote: Jim Gallacher wrote .. I don't have alot more to say on these last 2 emails. I think you are on the right path here. Okay, I think I have a good plan now. To summarise the whole issue, the way Apache treats multiple handlers in a single phase for non content handler phases is as follows: PostReadRequestHandler RUN_ALL TransHandler RUN_FIRST MapToStorageHandler RUN_FIRST InitHandler RUN_ALL HeaderParserHandler RUN_ALL AccessHandlerRUN_ALL AuthenHandlerRUN_FIRST AuthzHandler RUN_FIRST TypeHandler RUN_FIRST FixupHandler RUN_ALL LogHandler RUN_ALL RUN_ALL means run all handlers until one returns something other than OK or DECLINED. Thus, handler needs to return DONE or an error to have it stop processing for that phase. RUN_FIRST means run all handlers while they return DECLINED. Thus, needs handler to return OK, DONE or error to have it stop processing for that phase. Where multiple handlers are registered within mod_python for a single phase it doesn't behave like either of these. In mod_python it will keep running the handlers only while OK is returned. Returning DECLINED causes it to stop. This existing behaviour can be described (like mod_perl) as stacked handlers. Having non content handler phases behave differently to how Apache does it causes problems. For example things like a string of authentication handlers which only say OK when they handle the authentication type, can't be implemented properly. In Apache, it should stop the first time one returns OK, but in mod_python it will keep running the handlers in that phase. In summary, it needs to behave more like Apache for the non content handler phases. In respect of the content handler phase itself, in practice only one handler module is supposed to implement it. At the Apache level there is no concept of different Apache modules having goes at the content handler phase and returning DECLINED if they don't want to handle it. This is reflected in how in the type handler phase, selection of the module to deliver content is usually done by setting the single valued req.handler string. Although, when using mod_python this is done implicitly by setting the SetHandler/AddHandler directives and mod_negotiation then in turn setting req.handler to be mod_python for you. Because mod_python when executed for the content handler phase is the only thing generating the content, the existing mechanism of stacked handlers and how the status is handled is fine within just the content handler phase. Can thus keep that as is and no chance of stuffing up existing systems. Where another phase calls req.add_handler() to add a handler or multiple handlers for the PythonHandler (content) phase, these will be added in a stacked manner within that phase. This also is the same as it works now. There would be non need to have a new function to add stacked handlers as that behaviour would be dictated by phase being PythonHandler. For all the non content handler phases though, the current stacked handlers algorithm used by mod_python would be replaced with how Apache does it. That is, within multiple handlers registered with mod_python for non content handler phase, it would use RUN_FIRST or RUN_ALL algorithm as appropriate for the phase. For those which use RUN_ALL, this wouldn't be much different than what mod_python does now except that returning DECLINED would cause it to go to next mod_python handler in that phase instead of stopping. It is highly unlikely that this change would have an impact as returning DECLINED in RUN_ALL phases for how mod_python currently implements it, tends not to be useful and can't see that anyone would have been using it. For those which use RUN_FIRST, the difference would be significant as reurning OK will now cause it to stop instead of going to next mod_python handler in the phase. Personally I don't think this would be a drama as not many people would be using these phases and highly unlikely that someone would have listed multiple handlers for such phases. If they had and knew what they were doing, they should have long ago realised that the current behaviour was a bit broken and it even probably stopped them from doing what they wanted unless they fudged it. As to use of req.add_handler() for non content handler phases, each call would create a distinct handler, ie., no stacking of handlers at all. No separate function is required though, as slight change in behaviour determine form phase specified. To sum up, I think these changes would have minimal if no impact as where changes are
[SECURITY] A Security Issue with FileSession in 3.2.7
If you are using the recently released mod_python 3.2.7 please beware that a security issue was discovered in the FileSession code. You are vulnerable only if you are using mod_python 3.2.7 AND you are using FileSession to keep sessions. FileSession is new in 3.2.7 and is not enabled by default, therefore if you are using mod_python Session in its default configuration you are not vulnerable. The extent of this vulnerability is limited. Only a user who already has an account (or some ability to write to the filesystem) on the system running httpd could exploit it, and to the best of our knowledge such a user could potentially cause httpd to execute arbitrary code. We are working on a security release of the next version of mod_python and expect it to be out shortly. Until then, please do not use FileSession. Regards, Your mod_python team.
Re: mutex dir?
On Tue, 14 Feb 2006, Jim Gallacher wrote: I wonder if we should generalize this, so rather than PythonMutexDir, we have PythonModuleConfig. Usage might look like: PythonModuleConfig mutex_dir /path/to/mutexs PythonModuleConfig max_mutex_locks 8 I may be wrong, but I think the reason this was never configurable was because the mutex is created before the apache config is read. Grisha
Re: Getting Started on mod_python 3.3.
On Tue, 14 Feb 2006, Nicolas Lehuen wrote: 2006/2/14, Graham Dumpleton [EMAIL PROTECTED]: [...] If we want to go down the path of having interim 3.2 bug rollup releases while 3.3 is being developed, might I suggest that we target the following for such a release in the near future. MODPYTHON-77 The Simplified GIL Aquisition patches. If this is the one where you get Restriction Execution errors upon launching a thread, then I'm kinda keen on seeing this fixed sooner than later. Just my $0.02. :-) Grisha
ANNOUNCE: Mod_python 3.2.7
The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the 3.2.7 release of mod_python. Mod_python 3.2.7 is considered a stable release, suitable for production use. Mod_python is an Apache HTTP Server module that embeds the Python language interpreter within the server. With mod_python you can write web-based applications in Python that will run many times faster than traditional CGI and will have access to advanced features such as ability to maintain objects between requests, access to httpd internals, content filters and connection handlers. The 3.2.7 release has many new features, feature enhancements, fixed bugs and other improvements over the previous version. See Appendix A of mod_python documentation for more details. Mod_python 3.2.7 is released under Apache License version 2.0. Mod_python 3.2.7 is available for download from: http://httpd.apache.org/modules/python-download.cgi More information about mod_python is available at: http://httpd.apache.org/modules/ Many thanks to Jim Gallacher, Graham Dumpleton, Nicolas Lehuen and everyone else who contributed to and helped test this release, without your help it would not be possible! Regards, Grisha Trubetskoy
Re: [DRAFT] ANNOUNCE: Mod_python 3.2.7
On Thu, 9 Feb 2006, Jim Gallacher wrote: Depending on Grisha's preference, I think we should either put the content in svn and have a cron job on modpython.org do a nightly update, or start using the Apache infrastructure for the website. See http://www.apache.org/dev/project-site.html for more info. We've been talking about moving modpython.org to apache infrastructure for years, it's just always ended up being a lower priority thing that never got done. modpython.org isn't much of a site at this point, but if someone would be willing to step up and dedicate some serious time to web design and content, perhaps it would make it worth it to actually do the work of moving it... Grisha
3.2.x branch
On Thu, 9 Feb 2006, Jim Gallacher wrote: Looks good. As soon as you make the official announcement I'll create branches/3.2.x in svn and we can start on 3.3. I don't think we need to wait for the announcement - 3.2.7 is already out if you look on the download page, the announcement is more about marketing if you will. Grisha
Re: mod_python 3.2.7 available for testing
On Tue, 7 Feb 2006, Jim Gallacher wrote: I assumed we would have a separate thread for the core vote, but what the heck. :) +1 is my core vote. +1 from me as well. Grisha
Re: 3.2.6 test period - how long do we wait?
On Tue, 31 Jan 2006, Jim Gallacher wrote: I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? Unfortunately, the answer is yes... As much as we'd like to release it sooner, I think it's important to not loose the perspective that quality over speed is what we favor. In other words - if it takes a little longer to get the quality we want, then that's just fine. :-) Grisha
Re: Segfaults in ConnectionHander
This may be a good question to post to dev@httpd.apache.org
Grisha
On Mon, 30 Jan 2006, Graham Dumpleton wrote:
Getting a bit closer now, have next part of puzzle worked out.
Graham Dumpleton wrote ..
This is starting to look really ugly.
In _conn_read(), it first creates a bucket brigade from the connection
objects pool object. No chance of this being destroyed prematurely
as a result.
bb = apr_brigade_create(c-pool, c-bucket_alloc);
From what I understand, it then makes a call which links the bucket
brigade to the actual source of data.
rc = ap_get_brigade(c-input_filters, bb, mode, APR_BLOCK_READ, bufsize);
Under normal circumstances this would also have the side effect of
performing the first actual read of data off the socket connection which
the client created to Apache.
When ap_get_brigade() is called, it is actually calling through to the
function core_input_filter() in Apache (server/core.c). In that function, it
ultimately hits the code:
e = APR_BRIGADE_FIRST(ctx-b);
rv = apr_bucket_read(e, str, len, block);
if (APR_STATUS_IS_EAGAIN(rv)) {
return APR_SUCCESS;
}
Tracking down into apr_bucket_read() it ends up calling the function
socket_bucket_read() containg the code:
*str = NULL;
*len = APR_BUCKET_BUFF_SIZE;
buf = apr_bucket_alloc(*len, a-list); /* XXX: check for failure? */
rv = apr_socket_recv(p, buf, len);
if (block == APR_NONBLOCK_READ) {
apr_socket_timeout_set(p, timeout);
}
if (rv != APR_SUCCESS rv != APR_EOF) {
apr_bucket_free(buf);
return rv;
}
The apr_socket_recv() is what is doing the initial read of data from the
socket connection. This should block until the first data is received.
What is happening though is that it is returning -1 with errno set to
EAGAIN. Thus it frees the temporary bucket it created and returns
EAGAIN as the result.
If you note the code in the core_input_filter() it has:
if (APR_STATUS_IS_EAGAIN(rv)) {
return APR_SUCCESS;
}
Thus, when EAGAIN is encountered, it simply returns success and does
not do anything else.
Returning back up to _conn_read() in mod_python source code, we have
where core_input_filter() was called ap_get_brigade():
Py_BEGIN_ALLOW_THREADS;
rc = ap_get_brigade(c-input_filters, bb, mode, APR_BLOCK_READ, bufsize);
Py_END_ALLOW_THREADS;
if (! APR_STATUS_IS_SUCCESS(rc)) {
PyErr_SetObject(PyExc_IOError,
PyString_FromString(Connection read error));
return NULL;
}
Since APR_SUCCESS was returned and assigned to rc, no problem is detected.
The code which follows then assumes that the first bucket in the bucket
brigade actually contains valid data, when in fact the first bucket is actually
crap as nothing was done to set up a valid bucket since EAGAIN was returned.
As a consequence it crashes.
Thus in summary, _conn_read() doesn't cater in any way for the possibility
that the initial socket read may have failed because of EAGAIN and thus
the bucket is bogus. The problem is, how is it mean't to know this if the
value APR_SUCCESS is returned by ap_get_brigade().
At this point, seems a bit of research is needed of other examples of
connection handlers for Apache to see how they handle the initial startup
sequence and processing of initial data. What is in mod_python now does
not appear to be reliable in the face of an EAGAIN error occuring.
Graham
Re: 3.2.6 test period - how long do we wait?
On Sun, 29 Jan 2006, Graham Dumpleton wrote: Grisha wrote .. buffer = bufsize; I suspect you mean't: buffer += bufsize; buffer = bufsize should be correct because you move the pointer to the end of where the bufer was. buffer += bufsize would set it further than you need it. Anyway, this change doesn't help with Mac OS X as it doesn't even get invoked. Unlike suggestions by someone else that self seemed to be getting corrupted, it looks fine to me, and code simply crashed down in: apr_bucket_read(b, data, size, APR_BLOCK_READ) on very first call to it. Thus need to start tracking into Apache itself and see what there may be about bucket structures that isn't correct. This is where I got to last time before I gave up, feeling it wasn't worth the effort at the time. I'll try and build a version of Apache with debug so I can get a better stack trace. The first thing I'd check is for validity of b. Buckets use reference counting much like Python, so sometimes it's possible for a bucket to self-distruct. Grisha
Re: 3.2.6 test period - how long do we wait?
On Sun, 29 Jan 2006, Graham Dumpleton wrote: buffer += bufsize; On a second thought - yes, you're right :-) Grisha
Re: 3.2.6 test period - how long do we wait?
On Thu, 26 Jan 2006, Jim Gallacher wrote: It seems like any 3.2.6 testing that is going to be done, has been done. I've been kinda swamped with unrelated things past two weeks, so I wasn't paying much attention. Perhaps an e-mail summarizing the +1's so far and a quick vote of the core group on whether this is enough? Grisha
Re: please set up a mod_python core group
On Thu, 19 Jan 2006, Jorey Bump wrote: +1 here, but since the build process and typical MPM differs among platforms, could we see a list that this group represents? This group would not represent any platforms when acting in _this_ capacity. One of the group's responsibility would be to decide whether sufficient number of platforms were represented by tests done by anyone on this mailing list (including anyone from this group, of course). Grisha
Re: [jira] Updated: (MODPYTHON-106) PythonAutoReload directive can't be turned off in config.
On Mon, 2 Jan 2006, Graham Dumpleton (JIRA) wrote: Should this be fixed in 3.2, given that it probably hasn't work in 3.0 and 3.1? I think it would be a good idea. Hopefully this is the last fix before we roll it out. Grisha
Re: [jira] Created: (MODPYTHON-105) mod_python.publisher should not discard content for HEAD request.
On Mon, 2 Jan 2006, Graham Dumpleton (JIRA) wrote: In addressing MODPYTHON-71, mod_python.publisher code was changed to read: if req.method!='HEAD': req.write(result) This change should not really have been made and it should be changed back to what was there before, ie., just: req.write(result) [...] As an an example of an Apache module that uses output filters to do stuff, there is mod_cache. Luckily in that case, a HEAD request is one of various cases where mod_cache decides it will not use the output. This does not mean though that some other output filter that someone is using might expect content to be there for HEAD. I am not sure I agree with this explanation (while I do agree that the behaviour should be reverted, but for different reason, see below). The RFC is pretty clear on HEAD: The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response., so for a filter (or anything) to expect the content with HEAD is wrong. In summary, one could also say that if a user wants to not output anything for a HEAD request, that is there decision, but mod_python.publisher should not be making that decision for them. May be not for the publisher, but given the MUST NOT it should be OK for either httpd or mod_python to do this. If httpd does not do it, then perhaps it should be made part of req.write()? It may be a good idea to check the httpd-dev archives to see if the issue of HEAD has been discussed in the past. In any event, I think the behaviour should be reverted to ignore HEAD for now simply because it is a half-ass solution given that req.write() gets through, especially because PSP templates rely on req.write() primarily. I think for 3.2 we can just leave it at that, but for 3.3 to seriously consider whether mod_python should chop output for HEAD requests. Also, I think a link to this JIRA issue as a comment somewhere in the code would be great or someone down the road will repeat this whole HEAD thing again. Grisha
Re: [jira] Created: (MODPYTHON-105) mod_python.publisher should not discard content for HEAD request.
On Wed, 4 Jan 2006, Gregory (Grisha) Trubetskoy wrote: It may be a good idea to check the httpd-dev archives to see if the issue of HEAD has been discussed in the past. Here's a relevant bit of info from the httpd-dev archives: * All handlers should always send content down even if r-header_only is set. If not, it means that the HEAD requests don't generate the same headers as a GET which is wrong. Grisha
Re: [jira] Created: (MODPYTHON-105) mod_python.publisher should notdiscard content for HEAD request.
On Wed, 4 Jan 2006, Graham Dumpleton wrote: Either way, we agree that mod_python.publisher should still output content for HEAD. Yep. I would also propose as a change that the req.write() call not cause output to be flushed to allow an output filter like CONTENT_LENGTH to be used. Hmmm... This needs some more research. i.e. I don't quite completely understand why the CONTENT_LENGTH filter can only count bytes when there is no flush() (shortcoming of CONTENT_LENGTH or an impossibility?)... Implicit buffering would be a significant change - if someone used req.write() to generate dynamic content (e.g. output from one of those traceroute sites being sent in real time), they'd be very surprised to not see the output anymore. CGI I think flushes implicitely at every end of line which where this behaviour comes from... On the other hand implicit flush slows things down tremenduosly when you have lots of req.write()s, this was discovered when PSP was added and that's when the no-flush argument was introduced, so if backwards compatibility was of no concern, I'd make implicit buffering (i.e. no-flush) default. I'll add a new JIRA issue for that. Yep, totally. And thanks for all your help BTW :-) Grisha
Re: [jira] Created: (MODPYTHON-107) mod_python.publisher shouldn't flush result when written.
On Wed, 4 Jan 2006, Graham Dumpleton (JIRA) wrote: This makes one wander if there should be a configurable option for mod_python.psp to tell it not to flush output as well so that CONTENT_LENGTH could be used in that case as well. ??? I kinda mentioned this in the previous e-mail - PSP always does not flush output, in fact that 0/1 argument was added to req.write() just for PSP. So for pure PSP pages, content-length _should_ work (haven't tested it). Grisha
3.2.6b
I hope everyone's having a merry Christmas or whatever other holidays you're celebrating :-) I haven't been following the list very closely lately, but it looks like last time 3.2.6b was brought up a bunch of bugs came out of the woodwork. Does it look like we're near being ready to roll 3.2.6b now? Grisha
Re: input/output filters and .htaccess
On Tue, 20 Dec 2005, Graham Dumpleton wrote: Grisha, do you remember why the following warning is present in directive_PythonInputFilter() and directive_PythonOutputFilter() of mod_python.c. /* register the filter NOTE - this only works so long as the directive is only allowed in the main config. For .htaccess we would have to make sure not to duplicate this */ Having input/output filters be able to be specified in .htaccess must have been considered, but there must have been some issue with it that prevented it being done at the time. I think it was because if specified in .htaccess, that code would be executed for every request and something would need to be done to make sure ap_register_input_filter() is only called once. I don't remember the intricate details at this point, and it's possible that this warning is unnecessary either because I didn't understand something or perhaps the way Apache works has changed. In any event, I'd do some careful investigating before disregarding that NOTE :) Grisha
Re: input/output filters and .htaccess
On Wed, 21 Dec 2005, Graham Dumpleton wrote: /* register the filter NOTE - this only works so long as the directive is only allowed in the main config. For .htaccess we would have to make sure not to duplicate this */ Having input/output filters be able to be specified in .htaccess must have been considered, but there must have been some issue with it that prevented it being done at the time. Hmmm, looks like it will not be able to be done after all. This is because registration of a filter by name is done at server scope. Thus, if done from a .htaccess file in one directory, it will then be visible in other scopes. The continual registration of the filter each time a request hits that directory will also cause a growing use of memory from the server pool (I think). Pity. This may be something you could clarify on the httpd-devel or apr-devel lists, may be we're missing something in how it was intended to work. Grisha
Re: input/output filters and .htaccess
This sounds like a good idea, but it's probably better to push this off to 3.3 just to veer on the side of caution. My $0.02 Grisha On Tue, 20 Dec 2005, Graham Dumpleton wrote: Anyone know if there are any technical reasons why input/output filters as they exist at the moment, applying only to body content and not headers, can not be specified in a .htaccess files? Specifically, the SetInputFilter, SetOutputFilter, AddInputFilter and AddOutputFilter directives of Apache can be specified in either main server configuration or .htaccess files, yet the PythonInputFilter and PythonOutputFilter directives are only allowed to be specified in the main server configuration. This is because mod_python.c contains: AP_INIT_TAKE12( PythonInputFilter, directive_PythonInputFilter, NULL, RSRC_CONF|ACCESS_CONF, Python input filter.), AP_INIT_TAKE12( PythonOutputFilter, directive_PythonOutputFilter, NULL, RSRC_CONF|ACCESS_CONF, Python output filter.), If however I change it to: AP_INIT_TAKE12( PythonInputFilter, directive_PythonInputFilter, NULL, OR_ALL, Python input filter.), AP_INIT_TAKE12( PythonOutputFilter, directive_PythonOutputFilter, NULL, OR_ALL, Python output filter.), it is then possible to use the mod_python directives in a .htaccess file. Running the code this way appears to work for both input and output filters when specified in the .htaccess file without problems. So, does anyone know why this might be a bad idea? I can't really think of any reasons why it shouldn't be okay. If there is some confirmation that it all sounds reasonable, I'll create a JIRA issue for it to be a future enhancement. Graham
Re: Does 3.2 still support python 2.2?
On Fri, 9 Dec 2005, Jim Gallacher wrote: Next question. Should we bump the Apache version requirement as well. Currently the docs state that Apache 2.0.40 or later is required. I don't recall seeing anyone testing mod_python 3.2 on anything less than apache 2.0.53. I don't know if there are any changes between 40 and 53 that may have a negative impact, but if we haven't actually tested on the earlier versions are we just asking for trouble? Probably :) I don't remember where the 2.0.40 came from. There may have been a technical reason for it, or may be it was just the current version at the time. I think the most honest thing to would be to say that we've tested it with 2.0.53, but it will most likely work with a few minor versions earlier just as well. Since we're part of HTTP project, we should anyhow encourage folks to use the most recent versions. Grisha
Re: What's in a URL ?
On Sat, 3 Dec 2005, Nicolas Lehuen wrote: 2. I don't know - I did not made this to distribute It's a _very_ nice document, but I think you jumped the gun by checking it into Doc because it's not a .tex file and doesn't fit into the Mod_python manual, which is what the Doc directory is for. I worry we may develop a tendency towards treating the Apache SVN as file sharing mechanism, which is not what it's for. At least whatever is underneath mod_python/ directory in SVN must IMHO be part of the final package, and should probably not include any scratch-pad/temoprary/work-in-progress type stuff. Grisha
Re: Various musings about the request URL / URI / whatever
On Tue, 29 Nov 2005, Nicolas Lehuen wrote:
def current_url(req):
[snip]
# host
current_url.append(req.hostname)
[snip]
This part isn't going to work reliably if you are not using virtual hosts
and just bind to an IP number. Deciphering the URL is an impossible task -
I used to have similar code in my apllications, but lately I realized that
it does not work reliably and it is much simpler to just treat it as a
configuration item...
First question, is there a simpler way to do this ? Ironically, when using
mod_rewrite, you get an environment variable named SCRIPT_URI which is
precisely what I need (SCRIPT_URL, also added by mod_rewrite, is equivalent
to req.uri... Don't ask we why). But relying on it isn't safe since
mod_rewrite isn't always used.
well - here's how it does it.
/*
* create the SCRIPT_URI variable for the env
*/
/* add the canonical URI of this URL */
thisserver = ap_get_server_name(r);
port = ap_get_server_port(r);
if (ap_is_default_port(port, r)) {
thisport = ;
}
else {
apr_snprintf(buf, sizeof(buf), :%u, port);
thisport = buf;
}
thisurl = apr_table_get(r-subprocess_env, ENVVAR_SCRIPT_URL);
/* set the variable */
var = apr_pstrcat(r-pool, ap_http_method(r), ://, thisserver, thisport,
thisurl, NULL);
apr_table_setn(r-subprocess_env, ENVVAR_SCRIPT_URI, var);
/* if filename was not initially set,
* we start with the requested URI
*/
if (r-filename == NULL) {
r-filename = apr_pstrdup(r-pool, r-uri);
rewritelog(r, 2, init rewrite engine with requested uri %s,
r-filename);
}
Second question, if there isn't any simpler way to do this, should we add it
to mod_python ? Either as a function like above in mod_python.util, or as a
member of the request object (named something like url to match the other
member named uri, but that's just teasing).
I don't know... Since the result is going to be half-baked... I think a
more interesting and mod_python-ish thing to do would be to expose all the
API's used in the above code (e.g. ap_get_server_name, ap_is_default_port,
ap_http_method) FIRST, then think about this.
And third question (in pure Spanish inquisition style) : why is
req.parsed_uri returning me a tuple full of Nones except for the uri and
path_info part ?
This is an httpd question most likely...
Ah, fourth question : why are we (mod_python, mod_rewrite and the CGI
environment variables) using the terms URI and URL to distinguish
between a full, absolute resource path and a path relative to the server,
whereas the definition of URLs and URIs is very vague and nothing close to
this (http://www.w3.org/TR/uri-clarification/#contemporary) ? Shouldn't we
save our souls and a lot of saliva by choosing better names ?
No, we (mod_python) should just use the exact same name that httpd uses.
If we come up better names, then it's just going to make it even more
confusing.
OK, OK, fifth question : we made req.filename and other members writable.
But when those attributes are changed, as Graham noted a while ago, the
other dependent ones aren't, leading to inconsitencies (for example, if you
change req.filename, req.canonical_filename isn't changed). Should we try to
solve this
The solutions is to make req.canonical_filename writable too and document
that if you change req.filename, you may consider changing
canonical_filename as well and what will happen if you do not.
and provide clear definition of the various parts of a request
for mod_python 3.3 ?
Yes, that'd be good :)
Grisha
Re: [jira] Commented: (MODPYTHON-93) Improve util.FieldStorage efficiency
On Tue, 29 Nov 2005, Jim Gallacher wrote: I still think the correct place to create the index dictionary is in the __init__ phase. Using the dictionary-on-demand idea only improves performance on the second access to a form field. For the first access you are still iterating through the whole list for each field name. I am still not convinced we need an index. I'd like to see some concrete proof that we're not engaging in overoptimization here - is this really a bottleneck for anyone? If we're concerned (and I'm not at this point) that FieldStorage is too slow, we should just rewrite the whole thing in C :-) Grisha
Re: [SPAM] [mod_python] [SPAM] ANNOUNCE: Mod_python 3.2.5 Beta
I don't know how to do that, and it doesn't bother me that much :-) Grisha On Mon, 28 Nov 2005, David Fraser wrote: Gregory (Grisha) Trubetskoy wrote: The Apache Software Foundation and The Apache HTTP Server Project are pleased to announce the 3.2.5 Beta release mod_python. Can we make sure the final release doesn't come out as SPAM on the announce list? :-) David
Re: mod_python 3.2.5b available for testing
On Thu, 17 Nov 2005, Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: OK, I think we got enough +1's and no show-stopping problems (for a beta at least). I copied it over the apache server, once the mirrors sync I'll update the site and send the big announcement. I was also thinking it was time for a wider release, but was hoping to see a +1 for FreeBSD 4 or 5 first. +1 FreeBSD 4.9-RC apache 2.0.53 Python 2.3 Grisha
Re: mod_python 3.2.5b available for testing
OK, I think we got enough +1's and no show-stopping problems (for a beta at least). I copied it over the apache server, once the mirrors sync I'll update the site and send the big announcement. Grisha On Mon, 14 Nov 2005, Jim Gallacher wrote: A new mod_python 3.2.5 beta tarball is now available for testing. A windows binary should be available shortly. This release is similar to 3.2.4b but fixes a couple of minor issues - MODPYTHON-87 (psp_parser), and MODPYTHON-40 (file upload). Here are the rules: In order for a file to be officially announced, it has to be tested by developers on the dev list. Anyone subscribed to this list can (and should feel obligated to :-) ) test it, and provide feedback *to _this_ list*! (Not the [EMAIL PROTECTED] list, and preferably not me personally). The files are (temporarily) available here: http://www.modpython.org/dist/ Please download it, then do the usual $ ./configure --with-apxs=/wherever/it/is $ make $ (su) # make install Then (as non-root user!) $ cd test $ python test.py And see if any tests fail. If they pass, send a +1 to the list, if they fail, send the details (the versions of OS, Python and Apache, the test output, and suggestions, if any). Thank you, Jim Gallacher
Re: mod_python 3.2.5b available for testing
hehe - sorry about that, should be fixed now Grisha On Tue, 15 Nov 2005, David Fraser wrote: Jim Gallacher wrote: A new mod_python 3.2.5 beta tarball is now available for testing. A windows binary should be available shortly. The windows binary for python 2.3 is borked, and contains its own md5sum, being only 68 bytes long. This did raise the interesting thought of how easy it would be to create a file which contains its own md5sum, but aside from that, I think it should be reuploaded :-) David
Re: board report for HTTP server project
On the mod_python side we got a 3.2.2 beta out and will try to get 3.2.? final done before Apachecon (hopefully). The last release (not counting security fix ones) was 20 months ago, so this is pretty significant. Grisha On Mon, 14 Nov 2005, Roy T. Fielding wrote: Much to my surprise, I apparently have a board report due yesterday for this Wednesday's board meeting. Do we have any ASF issues that need reporting to the board, aside from what is in STATUS*? Any choice commentary? Does anyone else feel like we have too many dev lists for one project? Roy
Re: PythonEnablePdb option.
On Thu, 3 Nov 2005, Graham Dumpleton wrote: With the thought of mod_python perhaps ignoring the PythonEnabledPdb option when not run in single process mode, is there a way using the apache.mpm_query() function or some other function of determining that Apache is running in single process mode? I wonder if simply examining sys.argv would work? Grisha
Re: Gentoo (Was: mod_python 3.2.3b available for testing)
If we don't get an resolution on this Gentoo issue - should we just go ahead and release the file anyway? Hopefully then someone will fix it before the final release? Grisha On Tue, 25 Oct 2005, Gregory (Grisha) Trubetskoy wrote: Hmmm... Looking at /usr/lib/python2.4/httplib.py, sock.readline() gets an EOF upon reading the first byte. Do you see anything in the error logs associated with this, like a segfault? To make it easier to isolate, try editing test.py to comment out all other tests and just leave this one. Look for a line that looks like: perRequestSuite.addTest(PerRequestTestCase(test_req_headers_out)) and comment out every other test in this block but this one, then rerun it. At the end you should have a log file in the logs directory, hopefully it will contain a clue. Thanks! On Tue, 25 Oct 2005, Dominic Wong wrote: -1 for Gentoo Linux 2.6.13-gentoo-r3 Apache 2.0.54 Python 2.4.1 * Testing req.headers_out connect: (127.0.0.1, 32873) send: 'GET / HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: test_req_headers_out:32873\r\n\r\n' reply: '' E == ERROR: test_req_headers_out (__main__.PerRequestTestCase) -- Traceback (most recent call last): File test.py, line 597, in test_req_headers_out response = conn.getresponse() File /usr/lib/python2.4/httplib.py, line 863, in getresponse response.begin() File /usr/lib/python2.4/httplib.py, line 333, in begin version, status, reason = self._read_status() File /usr/lib/python2.4/httplib.py, line 297, in _read_status raise BadStatusLine(line) BadStatusLine --
Re: Gentoo (Was: mod_python 3.2.3b available for testing)
I think the proper thing to do (i forgot about the cookie and x86-64 issues) is to consider 3.2.3b as shut down by pre-release testing, so it's just going to be a version that will never be publicly released. The next step is to apply the fixes you mentioned below and roll a 3.2.4b, then submit it to the list for test/vote for public release. Grisha On Thu, 27 Oct 2005, Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: If we don't get an resolution on this Gentoo issue - should we just go ahead and release the file anyway? Hopefully then someone will fix it before the final release? Since we have not received any additional information on this I think we should proceed. However there is also the issue of Too many cookies using mod_python 3.2.2b from a thread on the mod_python list. I don't know if we should generate another beta or just fix it in 3.2 final. The fix is very simple - remove 2 lines which had been added for the get_session feature, but that feature didn't make it into 3.2. There is also the issue of ./configure failing for x86-64 platforms. Again, it is a very simple fix - 3 lines in configure.in. I'll create JIRA issues and commit the fixes for these real soon. (tonight - maybe). To what extent can a final release differ from the last beta, Grisha? Is it OK to make some small changes from one to the other? When would changes be considered critical enough to require a new beta release round? Regards, Jim Grisha On Tue, 25 Oct 2005, Gregory (Grisha) Trubetskoy wrote: Hmmm... Looking at /usr/lib/python2.4/httplib.py, sock.readline() gets an EOF upon reading the first byte. Do you see anything in the error logs associated with this, like a segfault? To make it easier to isolate, try editing test.py to comment out all other tests and just leave this one. Look for a line that looks like: perRequestSuite.addTest(PerRequestTestCase(test_req_headers_out)) and comment out every other test in this block but this one, then rerun it. At the end you should have a log file in the logs directory, hopefully it will contain a clue. Thanks! On Tue, 25 Oct 2005, Dominic Wong wrote: -1 for Gentoo Linux 2.6.13-gentoo-r3 Apache 2.0.54 Python 2.4.1 * Testing req.headers_out connect: (127.0.0.1, 32873) send: 'GET / HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: test_req_headers_out:32873\r\n\r\n' reply: '' E == ERROR: test_req_headers_out (__main__.PerRequestTestCase) -- Traceback (most recent call last): File test.py, line 597, in test_req_headers_out response = conn.getresponse() File /usr/lib/python2.4/httplib.py, line 863, in getresponse response.begin() File /usr/lib/python2.4/httplib.py, line 333, in begin version, status, reason = self._read_status() File /usr/lib/python2.4/httplib.py, line 297, in _read_status raise BadStatusLine(line) BadStatusLine --
Re: Where do we stand on 3.2.2 final?
On Sat, 22 Oct 2005, Jim Gallacher wrote: Just a note on updating src/include/mpversion.h. Should we be changing MPV_PATCH in svn trunk (and by extension MPV_STRING) to track the beta release patch level? In trunk we are still at MPV_PATCH 0, but in tags/release-3-2-2b we are at MPV_PATCH 2. Hmm... It looks like you've set the version number in the tag. The tags should be completely static (i think there may be certain instances where you modify the tag but that's very rare). The version should have been set in the trunk, _then_ you create the tag. Techincally the right way to solve this now is to merge the changes into the trunk, though I'll admit I've never done a merge with svn (yet). Another way is just to edit mpversion manually in the trunk ;) While on the subject - didn't we have some experimental 3.3 stuff - is that in SVN, and if so, is it in the trunk, or do we have a branch for this? Lastly, I agree, let's roll 3.2.3b ASAP. Grisha
Re: Where do we stand on 3.2.2 final?
On Fri, 21 Oct 2005, Nicolas Lehuen wrote: Hi, http://issues.apache.org/jira/browse/MODPYTHON-82 is resolved and fixed in trunk. Are you sure you didn't interchange the two issues ? Cool, I'm reading my mail serially. Grisha
Re: Minor documentation error in 3.2.2b.
On Fri, 30 Sep 2005, Graham Dumpleton wrote: BTW, what is the timetable for actual release of 3.2? Where I am working is finally upgrading to Apache 2.0 on one of their servers and want to be able to use final 3.2 release on the system. I think we should give a few weeks to give people a chance to report bugs, so given that it was announced on Sep 18th, i'd say some time mid-october? Grisha
Re: svn commit: r290569 - /httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py
Can we have a little discussion on pros/cons of this? Does this make
mod_python dependent on sqlite?
Thanks,
Grisha
On Tue, 20 Sep 2005 [EMAIL PROTECTED] wrote:
Author: nlehuen
Date: Tue Sep 20 14:28:32 2005
New Revision: 290569
URL: http://svn.apache.org/viewcvs?rev=290569view=rev
Log:
A first try at implementing a session storage relying on SQLite. It is slower
than FileSession but could scale better ?
Added:
httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py
Added: httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py
URL:
http://svn.apache.org/viewcvs/httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py?rev=290569view=auto
==
--- httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py (added)
+++ httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py Tue Sep 20
14:28:32 2005
@@ -0,0 +1,150 @@
+ #
+ # Copyright 2004 Apache Software Foundation
+ #
+ # Licensed under the Apache License, Version 2.0 (the License); you
+ # may not use this file except in compliance with the License. You
+ # may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an AS IS BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ # implied. See the License for the specific language governing
+ # permissions and limitations under the License.
+ #
+ # Originally developed by Gregory Trubetskoy.
+ #
+ # $Id: Session.py 231126 2005-08-09 22:26:38Z jgallacher $
+from Session import *
+from time import time
+
+try:
+# If this code is included into Session.py,
+# we don't want to add a dependency to SQLite
+from pysqlite2 import dbapi2 as sqlite
+except:
+pass
+else:
+class SQLiteSession(BaseSession):
+ A session implementation using SQLite to store session data.
+pysqlite2 is required, see http://pysqlite.org/
+
+
+def __init__(self, req, filename=None, sid=0, secret=None, timeout=0,
lock=1):
+# if no filename is given, get it from PythonOption
+if not filename:
+opts = req.get_options()
+if opts.has_key(session_filename):
+filename = opts[session_filename]
+else:
+# or build a session file in a temporary directory
+filename = os.path.join(
+opts.get('session_directory', tempdir),
+'mp_sess.sqlite'
+)
+
+self.filename = filename
+
+# check whether the sessions table exists, and create it if not
+db = sqlite.connect(self.filename)
+try:
+try:
+cur = db.cursor()
+cur.execute(
+select name from sqlite_master
+where name='sessions' and type='table'
+)
+if cur.fetchone() is None:
+cur.execute(
+create table sessions
+(id text,data blob,timeout real)
+)
+cur.execute(
+create unique index idx_session on sessions (id)
+)
+db.commit()
+finally:
+cur.close()
+finally:
+db.close()
+
+BaseSession.__init__(self, req, sid=sid, secret=secret,
+ timeout=timeout, lock=lock)
+
+def count(self):
+db = sqlite.connect(self.filename)
+try:
+try:
+cur = db.cursor()
+cur.execute(select count(*) from sessions)
+return cur.fetchone()[0]
+finally:
+cur.close()
+finally:
+db.close()
+
+def do_cleanup(self):
+db = sqlite.connect(self.filename)
+try:
+try:
+cur = db.cursor()
+cur.execute(
+delete from sessions where timeout?,
+(time(),)
+)
+db.commit()
+finally:
+cur.close()
+finally:
+db.close()
+
+def do_load(self):
+db = sqlite.connect(self.filename)
+try:
+try:
+cur = db.cursor()
+cur.execute(
+select data from sessions where id=?,
+(self._sid,)
+)
+row = cur.fetchone()
+if row is None:
+
Re: svn commit: r290569 - /httpd/mod_python/trunk/lib/python/mod_python/SQLiteSession.py
OK, my next question would be - is MySQL, PostgreSQL, Informix, Oracle, etc next, and is this the path we want to take, or is there something about sqlite that makes it unique? Grisha On Thu, 22 Sep 2005, Robert Sanderson wrote: Can we have a little discussion on pros/cons of this? Does this make mod_python dependent on sqlite? Nope. It'll just silently fail if it can't import dbapi2. Rob +try: +# If this code is included into Session.py, +# we don't want to add a dependency to SQLite +from pysqlite2 import dbapi2 as sqlite +except: +pass +else: +class SQLiteSession(BaseSession):
Re: mod_python 3.2.2b available for testing
We should give it a couple of days to make sure that no -1's appear. Once we have a good number of +1's and sufficient time has passed to be reasonably sure that no -1's are coming, the file will need to placed on www.apache.org, PGP-signed, given about 24 hours for mirrors to pick it up; then the download page needs to be regenerated (some XML files edited and CVS/SVN updated - I'll take care of it and/or give you the details when we get there). Then we send out an official announcements to all the usual places - announce@apache.org, [EMAIL PROTECTED] and comp.lang.python. I'll dig up the earlier announcements. Grisha On Wed, 14 Sep 2005, Jim Gallacher wrote: Grisha, I think you mentioned that we should announce this beta on the mod_python list as well. If so I thought we could wait until we get a +1 from a MacOS X user here on python-dev before proceeding. Regards, Jim Jim Gallacher wrote: A new mod_python 3.2.2 beta tarball is now available for testing. Hopefully this will be the last beta before the official 3.2 release. Here are the rules: In order for a file to be officially announced, it has to be tested by developers on the dev list. Anyone subscribed to this list can (and should feel obligated to :-) ) test it, and provide feedback *to _this_ list*! (Not the [EMAIL PROTECTED] list, and preferably not me personally). The files are (temporarily) available here: http://www.modpython.org/dist/ Please download it, then do the usual $ ./configure --with-apxs=/wherever/it/is $ make $ (su) # make install Then (as non-root user!) $ cd test $ python test.py And see if any tests fail. If they pass, send a +1 to the list, if they fail, send the details (the versions of OS, Python and Apache, the test output, and suggestions, if any). Thank you, Jim Gallacher
Re: FreeBSD compile problem (was Getting ready for 3.2 beta 2)
OK, I found the problem. The LockFile line is commented out in test.py
which causes Apache to try to create the lock in the default location in
/var/run.
http://svn.apache.org/viewcvs.cgi/httpd/mod_python/trunk/test/test.py?rev=125771r1=106619r2=125771
So the question is - can we just put it back in, or does it break
something on Win32?
Cheers
Grisha
On Tue, 13 Sep 2005, Jim Gallacher wrote:
Nicolas Lehuen wrote:
Jim, do you manage to build and test the 3.1.4 version on your setup ?
This looks like a permission problem, not something related to our current
problem.
I haven't tried 3.1.4. And I could also try the tests as root, which would
eliminate any permission problems. I have a busy day ahead of me so this will
have to wait until tonight.
Jim
Regards,
Nicolas
2005/9/13, Jim Gallacher [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]:
-1 for this patch. Actually, the patch itself is fine - it just
doesn't
fix the problem. The unit tests are still failings as per my previous
messages. ie the following is getting logged in test/logs/error_log:
[Mon Sep 12 19:49:33 2005] [emerg] (2)No such file or directory:
Couldn't create accept lock
Jim
Gregory (Grisha) Trubetskoy wrote:
Here's a patch (this is against 3.1.2b). Untested. This replaces
GIL
with with the APR lock.
--- src/mod_python.c.orig Mon Sep 12 16:42:28 2005
+++ src/mod_python.cMon Sep 12 17:32:26 2005
@@ -31,7 +31,9 @@
* (In a Python dictionary) */
static PyObject * interpreters = NULL;
+#ifdef APR_HAS_THREAD
static apr_thread_mutex_t* interpreters_lock = 0;
+#endif
apr_pool_t *child_init_pool = NULL;
@@ -127,9 +129,8 @@
if (! name)
name = MAIN_INTERPRETER;
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
-PyEval_AcquireLock();
#endif
if (!interpreters) {
@@ -156,8 +157,7 @@
idata = (interpreterdata *)PyCObject_AsVoidPtr(p);
}
-#ifdef WITH_THREAD
-PyEval_ReleaseLock();
+#ifdef APR_HAS_THREAD
apr_thread_mutex_unlock(interpreters_lock);
#endif
@@ -513,8 +513,10 @@
/* initialze the interpreter */
Py_Initialize();
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_create(interpreters_lock,APR_THREAD_MUTEX_UNNESTED,p);
+#endif
+#ifdef WITH_THREAD
/* create and acquire the interpreter lock */
PyEval_InitThreads();
#endif
On Mon, 12 Sep 2005, Gregory (Grisha) Trubetskoy wrote:
Yep, this is getting a little hairy, but nothing we couldn't
handle :-)
I did a little more research. Basically, this started with
Graham's
patch that addressed a problem with modules being reimported (or
something). From Graham's message:
The basic problem revolves around the Python dictionary used to
hold
the set of interpreters. The code in mod_python.c is trying to
use
the Python GIL to provide exclusive access to that dictionary
and any
subsequent creation of an interpreter.
The only catch is that in creating a new interpreter, the
Python core
is, in someway I don't understand, swapping thread states at some
point which is allowing other threads to acquire the GIL.
So what Graham's patch does is create an APR lock
(interpreters_lock)
and wrap all the access to the dictionary with calls to
apr_mutex_lock/unlock.
I think the _real_ way to address this issue is to first find
what is
the problem with using the Python GIL to serialize access to the
interpreters dictionary. Is this a Python bug, or are we not
understanding GIL and using it improperly?
BUT, given that the above question may be complicated to answer,
and
that Graham's patch resolves the issue, another thought:
If the APR lock works, what is the point of using the GIL in
addition?
Should we just use the APR-based lock alone? I.e., where we had
(after
Graham's patch):
#ifdef WITH_THREAD
apr_thread_mutex_lock(interpreters_lock);
PyEval_AcquireLock();
#endif
we would use:
#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
#endif
_without_ a call to PyEval_AcquireLock() at all.
It should compile OK, and on platforms where APR has no thread
support, like you said, it's not an issue since no separate
interpreters run in one process at the same time.
Grisha
On Mon, 12 Sep 2005, Nicolas Lehuen wrote
Re: FreeBSD compile problem (was Getting ready for 3.2 beta 2)
I'd keep the patch nonetheless - I think this is how it should be.
Grisha
On Mon, 12 Sep 2005, Jim Gallacher wrote:
-1 for this patch. Actually, the patch itself is fine - it just doesn't fix
the problem. The unit tests are still failings as per my previous messages.
ie the following is getting logged in test/logs/error_log:
[Mon Sep 12 19:49:33 2005] [emerg] (2)No such file or directory: Couldn't
create accept lock
Jim
Gregory (Grisha) Trubetskoy wrote:
Here's a patch (this is against 3.1.2b). Untested. This replaces GIL with
with the APR lock.
--- src/mod_python.c.orig Mon Sep 12 16:42:28 2005
+++ src/mod_python.cMon Sep 12 17:32:26 2005
@@ -31,7 +31,9 @@
* (In a Python dictionary) */
static PyObject * interpreters = NULL;
+#ifdef APR_HAS_THREAD
static apr_thread_mutex_t* interpreters_lock = 0;
+#endif
apr_pool_t *child_init_pool = NULL;
@@ -127,9 +129,8 @@
if (! name)
name = MAIN_INTERPRETER;
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
-PyEval_AcquireLock();
#endif
if (!interpreters) {
@@ -156,8 +157,7 @@
idata = (interpreterdata *)PyCObject_AsVoidPtr(p);
}
-#ifdef WITH_THREAD
-PyEval_ReleaseLock();
+#ifdef APR_HAS_THREAD
apr_thread_mutex_unlock(interpreters_lock);
#endif
@@ -513,8 +513,10 @@
/* initialze the interpreter */
Py_Initialize();
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_create(interpreters_lock,APR_THREAD_MUTEX_UNNESTED,p);
+#endif
+#ifdef WITH_THREAD
/* create and acquire the interpreter lock */
PyEval_InitThreads();
#endif
On Mon, 12 Sep 2005, Gregory (Grisha) Trubetskoy wrote:
Yep, this is getting a little hairy, but nothing we couldn't handle :-)
I did a little more research. Basically, this started with Graham's
patch that addressed a problem with modules being reimported (or
something). From Graham's message:
The basic problem revolves around the Python dictionary used to hold
the set of interpreters. The code in mod_python.c is trying to use the
Python GIL to provide exclusive access to that dictionary and any
subsequent creation of an interpreter.
The only catch is that in creating a new interpreter, the Python core
is, in someway I don't understand, swapping thread states at some
point which is allowing other threads to acquire the GIL.
So what Graham's patch does is create an APR lock (interpreters_lock)
and wrap all the access to the dictionary with calls to
apr_mutex_lock/unlock.
I think the _real_ way to address this issue is to first find what is
the problem with using the Python GIL to serialize access to the
interpreters dictionary. Is this a Python bug, or are we not
understanding GIL and using it improperly?
BUT, given that the above question may be complicated to answer, and
that Graham's patch resolves the issue, another thought:
If the APR lock works, what is the point of using the GIL in addition?
Should we just use the APR-based lock alone? I.e., where we had (after
Graham's patch):
#ifdef WITH_THREAD
apr_thread_mutex_lock(interpreters_lock);
PyEval_AcquireLock();
#endif
we would use:
#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
#endif
_without_ a call to PyEval_AcquireLock() at all.
It should compile OK, and on platforms where APR has no thread support,
like you said, it's not an issue since no separate interpreters run in
one process at the same time.
Grisha
On Mon, 12 Sep 2005, Nicolas Lehuen wrote:
Duh, this is becoming difficult :)
I was thinking that if APR_HAS_THREADS was 0, then Apache was forcibly
ran
in prefork mode, so there was no need for thread safety at all, given
the
fact that mod_python would only run one interpreter thread. So if
WITH_THREAD was not defined, ORAPR_HAS_THREADS was 0, then we would
not need
any thread safety code. Hence the definition of
MOD_PYTHON_WITH_THREAD_SUPPORT.
You're right in writing that a user could launch a new thread in
Python,
provided that WITH_THREAD is defined, even if APR_HAS_THREADS==0.
However,
having a look at the parts of mod_python.c where the thread safety was
put
in, I think we can safely say that those parts are only called by
mod_python
(through python_handler, python_cleanup etc who call get_interpreter).
Those
parts are therefore always called in the same thread (if
APR_HAS_THREADS==0,
that is) and there is no need for thread synchronization to be done
(no
shared data between the main thread and the other user threads, no
need to
release the GIL etc.).
BUT, I could be very, very wrong here, and your idea of reverting to a
conservative shield python threading calls with WITH_THREAD and apr
threading code with APR_HAS_THREADS is way more attractive to my
tired mind
right now. So if you want I can revert all this
MOD_PYTHON_WITH_THREAD_SUPPORT hack.
Regards,
Nicolas
2005/9/12, Gregory (Grisha) Trubetskoy
Re: FreeBSD compile problem (was Getting ready for 3.2 beta 2)
Done. All tests pass on my FreeBSD box. Nicolas - can you test Win32, I'm
not 100% sure if the change to test.py I made will work.
Grisha
On Tue, 13 Sep 2005, Nicolas Lehuen wrote:
Yes, now I remember I had to comment this line out because it broke
something on Win32. Go ahead, uncomment it and I'll add a test to remove it
when running the test on Win32.
Regards,
Nicolas
2005/9/13, Gregory (Grisha) Trubetskoy [EMAIL PROTECTED]:
OK, I found the problem. The LockFile line is commented out in test.py
which causes Apache to try to create the lock in the default location in
/var/run.
http://svn.apache.org/viewcvs.cgi/httpd/mod_python/trunk/test/test.py?rev=125771r1=106619r2=125771
So the question is - can we just put it back in, or does it break
something on Win32?
Cheers
Grisha
On Tue, 13 Sep 2005, Jim Gallacher wrote:
Nicolas Lehuen wrote:
Jim, do you manage to build and test the 3.1.4 version on your setup ?
This looks like a permission problem, not something related to our
current
problem.
I haven't tried 3.1.4. And I could also try the tests as root, which
would
eliminate any permission problems. I have a busy day ahead of me so this
will
have to wait until tonight.
Jim
Regards,
Nicolas
2005/9/13, Jim Gallacher [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]:
-1 for this patch. Actually, the patch itself is fine - it just
doesn't
fix the problem. The unit tests are still failings as per my previous
messages. ie the following is getting logged in test/logs/error_log:
[Mon Sep 12 19:49:33 2005] [emerg] (2)No such file or directory:
Couldn't create accept lock
Jim
Gregory (Grisha) Trubetskoy wrote:
Here's a patch (this is against 3.1.2b). Untested. This replaces
GIL
with with the APR lock.
--- src/mod_python.c.orig Mon Sep 12 16:42:28 2005
+++ src/mod_python.c Mon Sep 12 17:32:26 2005
@@ -31,7 +31,9 @@
* (In a Python dictionary) */
static PyObject * interpreters = NULL;
+#ifdef APR_HAS_THREAD
static apr_thread_mutex_t* interpreters_lock = 0;
+#endif
apr_pool_t *child_init_pool = NULL;
@@ -127,9 +129,8 @@
if (! name)
name = MAIN_INTERPRETER;
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
- PyEval_AcquireLock();
#endif
if (!interpreters) {
@@ -156,8 +157,7 @@
idata = (interpreterdata *)PyCObject_AsVoidPtr(p);
}
-#ifdef WITH_THREAD
- PyEval_ReleaseLock();
+#ifdef APR_HAS_THREAD
apr_thread_mutex_unlock(interpreters_lock);
#endif
@@ -513,8 +513,10 @@
/* initialze the interpreter */
Py_Initialize();
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_create(interpreters_lock,APR_THREAD_MUTEX_UNNESTED,p);
+#endif
+#ifdef WITH_THREAD
/* create and acquire the interpreter lock */
PyEval_InitThreads();
#endif
On Mon, 12 Sep 2005, Gregory (Grisha) Trubetskoy wrote:
Yep, this is getting a little hairy, but nothing we couldn't
handle :-)
I did a little more research. Basically, this started with
Graham's
patch that addressed a problem with modules being reimported (or
something). From Graham's message:
The basic problem revolves around the Python dictionary used to
hold
the set of interpreters. The code in mod_python.c is trying to
use
the Python GIL to provide exclusive access to that dictionary
and any
subsequent creation of an interpreter.
The only catch is that in creating a new interpreter, the
Python core
is, in someway I don't understand, swapping thread states at some
point which is allowing other threads to acquire the GIL.
So what Graham's patch does is create an APR lock
(interpreters_lock)
and wrap all the access to the dictionary with calls to
apr_mutex_lock/unlock.
I think the _real_ way to address this issue is to first find
what is
the problem with using the Python GIL to serialize access to the
interpreters dictionary. Is this a Python bug, or are we not
understanding GIL and using it improperly?
BUT, given that the above question may be complicated to answer,
and
that Graham's patch resolves the issue, another thought:
If the APR lock works, what is the point of using the GIL in
addition?
Should we just use the APR-based lock alone? I.e., where we had
(after
Graham's patch):
#ifdef WITH_THREAD
apr_thread_mutex_lock(interpreters_lock);
PyEval_AcquireLock();
#endif
we would use:
#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
#endif
_without_ a call to PyEval_AcquireLock() at all.
It should compile OK, and on platforms where APR has no thread
support, like you said, it's not an issue since no separate
interpreters run in one process at the same time.
Grisha
On Mon, 12 Sep 2005, Nicolas Lehuen wrote:
Duh, this is becoming difficult :)
I was thinking that if APR_HAS_THREADS was 0, then Apache was
forcibly ran
in prefork mode, so there was no need for thread safety at all,
given
the
fact that mod_python would only run one interpreter thread. So if
WITH_THREAD was not defined, ORAPR_HAS_THREADS was 0, then we
would
not need
any thread safety
Re: FreeBSD compile problem (was Getting ready for 3.2 beta 2)
Alright, cool. So I guess we're ready to roll the next tarfile now.
Grisha
On Tue, 13 Sep 2005, Nicolas Lehuen wrote:
I'm sure it is required, even after fixing the S in APR_HAS_THREADS I tried
with and without the PyEval_AcquireLock code and the latter works while the
former doesn't. I don't know why though... I'll have to review the code once
more to understand.
Regards,
Nicolas
2005/9/13, Gregory (Grisha) Trubetskoy [EMAIL PROTECTED]:
Good job - I've tested it and it works on FreeBSD. Just one last thing I
wanted to confirm - are you sure that PyEval_AcquireLock is required, or
was this a sideffect of the missing 'S'? I just want to make sure we're
not doing double locking where it's not needed.
Grisha
On Tue, 13 Sep 2005, Nicolas Lehuen wrote:
OK I fixed the problem, please check again on FreeBSD. Here is what I've
done (from the subversion comment) :
Fixed : APR_HAS_THREADS ends with an 'S'. Reintroduced the calls to
PyEval_AcquireLock and PyEval_ReleaseLock as they are required if
threading
is enabled. Removed two debugging log entries. Added the conditional
commenting of the LockFile directive in the LockFile class.
Regards,
Nicolas
2005/9/13, Nicolas Lehuen [EMAIL PROTECTED]:
Well, bad news... I have a -1 for now on Win32. I don't know why, but
when
I install and test the 20050911 version, everything is OK. When I
install
and test the latest version, the unit test behave very strangely (with
or
without the changes done by Graham). Basically, the Apache server takes
forever to stop. Weird...
I'm trying to find out what happens here.
Regards,
Nicolas
2005/9/13, Jim Gallacher [EMAIL PROTECTED]:
Gregory (Grisha) Trubetskoy wrote:
Done. All tests pass on my FreeBSD box. Nicolas - can you test Win32,
I'm not 100% sure if the change to test.py I made will work.
Good news. If the changes can be checked in and Nicolas can give a +1
on
the Windows test then I'll be able to generate the next, and hopefully
last, beta tonight.
As an aside, it may be useful to have an option for test.py to
preserve
a copy of the apache logs. It would make troubleshooting the unit test
failures much easier. Currently test.py deletes the logs after each
unit
test.
Regards,
Jim
Grisha
On Tue, 13 Sep 2005, Nicolas Lehuen wrote:
Yes, now I remember I had to comment this line out because it broke
something on Win32. Go ahead, uncomment it and I'll add a test to
remove it
when running the test on Win32.
Regards,
Nicolas
2005/9/13, Gregory (Grisha) Trubetskoy [EMAIL PROTECTED]:
OK, I found the problem. The LockFile line is commented out in
test.py
which causes Apache to try to create the lock in the default
location in
/var/run.
http://svn.apache.org/viewcvs.cgi/httpd/mod_python/trunk/test/test.py?rev=125771r1=106619r2=125771
So the question is - can we just put it back in, or does it break
something on Win32?
Cheers
Grisha
On Tue, 13 Sep 2005, Jim Gallacher wrote:
Nicolas Lehuen wrote:
Jim, do you manage to build and test the 3.1.4 version on your
setup ?
This looks like a permission problem, not something related to
our
current
problem.
I haven't tried 3.1.4. And I could also try the tests as root,
which
would
eliminate any permission problems. I have a busy day ahead of me
so
this
will
have to wait until tonight.
Jim
Regards,
Nicolas
2005/9/13, Jim Gallacher [EMAIL PROTECTED]
mailto: [EMAIL PROTECTED]:
-1 for this patch. Actually, the patch itself is fine - it just
doesn't
fix the problem. The unit tests are still failings as per my
previous
messages. ie the following is getting logged in
test/logs/error_log:
[Mon Sep 12 19:49:33 2005] [emerg] (2)No such file or directory:
Couldn't create accept lock
Jim
Gregory (Grisha) Trubetskoy wrote:
Here's a patch (this is against 3.1.2b). Untested. This replaces
GIL
with with the APR lock.
--- src/mod_python.c.orig Mon Sep 12 16:42:28 2005
+++ src/mod_python.c Mon Sep 12 17:32:26 2005
@@ -31,7 +31,9 @@
* (In a Python dictionary) */
static PyObject * interpreters = NULL;
+#ifdef APR_HAS_THREAD
static apr_thread_mutex_t* interpreters_lock = 0;
+#endif
apr_pool_t *child_init_pool = NULL;
@@ -127,9 +129,8 @@
if (! name)
name = MAIN_INTERPRETER;
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_lock(interpreters_lock);
- PyEval_AcquireLock();
#endif
if (!interpreters) {
@@ -156,8 +157,7 @@
idata = (interpreterdata *)PyCObject_AsVoidPtr(p);
}
-#ifdef WITH_THREAD
- PyEval_ReleaseLock();
+#ifdef APR_HAS_THREAD
apr_thread_mutex_unlock(interpreters_lock);
#endif
@@ -513,8 +513,10 @@
/* initialze the interpreter */
Py_Initialize();
-#ifdef WITH_THREAD
+#ifdef APR_HAS_THREAD
apr_thread_mutex_create(interpreters_lock,APR_THREAD_MUTEX_UNNESTED,p);
+#endif
+#ifdef WITH_THREAD
/* create and acquire the interpreter lock */
PyEval_InitThreads();
#endif
On Mon, 12 Sep 2005, Gregory (Grisha) Trubetskoy wrote:
Yep, this is getting a little hairy
Re: FreeBSD compile problem (was Getting ready for 3.2 beta 2)
I'm not sure I understand this, perhaps someone could write a message to the list explaining what we're doing here so there is a record. Sorry if I'm being slow-headed here. To me it seems that when you use thread-related calls from Python, you wrap those in Python defines (WITH_THREAD) and when you use thread-related calls from APR, you wrap those in APR defines (APR_HAS_THREAD), and that's all? In other words - what does MOD_PYTHON_WITH_THREAD_SUPPORT accomplish that the above does not. Also, given: #if(defined(WITH_THREAD) APR_HAS_THREADS) #define MOD_PYTHON_WITH_THREAD_SUPPORT 1 #else #define MOD_PYTHON_WITH_THREAD_SUPPORT 0 #endif Does this mean that if Python is compiled with thread support and APR is not, MOD_PYTHON_WITH_THREAD_SUPPORT is 0 which means that the thread safety code isn't there, but you still _can_ create threads because Python will let you - isn't this asking for a segfault/deadlock/whatever? Grisha On Mon, 12 Sep 2005, Jim Gallacher wrote: Gregory (Grisha) Trubetskoy wrote: Shouldn't that be PYTHON_WITH_THREAD rather than MOD_PYTHON_WITH_THREAD? I understand it to mean that we want the thread handling code compiled into mod_python. Compiling and testing right now. Jim On Mon, 12 Sep 2005, Nicolas Lehuen wrote: I've checked in a changeset wherein I define MOD_PYTHON_WITH_THREAD_SUPPORT and use it everywhere WITH_THREAD was previously used. This should do the trick ! Now if someone (like Jim) can give us his +1, that would be great. Regards, Nicolas 2005/9/12, Gregory (Grisha) Trubetskoy [EMAIL PROTECTED]: Just wanted to add to this message that if Jim's version runs and tests with the trick below (envvars is executed prior to apache start, but I don't think the tests use it, so you'll probably just have to set this var in the shell in which the tests are run), then this would be a solution for all FreeBSD issues and we could roll a beta 3 which will have a great change of being publicly released. Grisha On Mon, 12 Sep 2005, Gregory (Grisha) Trubetskoy wrote: OK, found it. This should work on FreeBSD where Python is threaded and Apache is not. [snip] And, if you built apache without thread support, you may need to add the following lines to $PREFIX/sbin/envvars: LD_PRELOAD=/usr/lib/libc_r.so export LD_PRELOAD [snip] On Mon, 12 Sep 2005, Gregory (Grisha) Trubetskoy wrote: On Mon, 12 Sep 2005, Jim Gallacher wrote: *** Warning: Linking the shared library mod_python.la against the *** static library /usr/local/lib/python2.4/config/libpython2.4.a is not portable! I think this was always there and its pretty harmless. On qemu: Syntax error on line 44 of /usr/home/jim/tmp/mod_python/test/conf/test.conf: Cannot load /usr/home/jim/tmp/mod_python/src/mod_python.so into server: /usr/home/jim/tmp/mod_python/src/mod_python.so: Undefined symbol pthread_attr_init This is because FreeBSD's libc comes in two versions - threaded and non-threaded. If Python is linked against the threaded ones and Apache against the non-thrreaded, then you get this problem. There is a simple fix for this - you just cause Apache to start with threaded libs, but I can't find any references to it right now and have to run off to a meeting. Grisha It is quite possible I don't have things configured correctly on the QEMU version and hence the different undefined symbol but it doesn't really matter since it fails either way. I don't have time to investigate further right now. I'll revisit this tonight. Regards, Jim Regards, Nicolas #if APR_HAS_THREADS defined(WITH_THREAD) 2005/9/11, Jim Gallacher [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: FYI, I found the following note in the INSTALL file in the apache source: * If you are building on FreeBSD, be aware that threads will be disabled and the prefork MPM will be used by default, as threads do not work well with Apache on FreeBSD. If you wish to try a threaded Apache on FreeBSD anyway, use ./configure --enable-threads. I'm also setting up FreeBSD under QEMU... so far so good, but installing anything using ports is really slow. QEMU's performance here is just killing me. I guess I should have read the manual first and used the binary packages for the software I wanted to install. :-( Regards, Jim Jim Gallacher wrote: Nicolas Lehuen wrote: OK, I've checked in a version that compiles both on at least Win32 and FreeBSD. I'm just testing if APR_HAS_THREAD is defined and only include the apr_thread_mutex_lock and unlock calls if it is defined. Compiles a passes unit tests on Linux Debian sid with mpm-prefork. Now, on minotaur, APR_HAS_THREAD is defined as 0. Does this mean that Apache is not configured for threading ? Can we assume that we are in the prefork model if APR_HAS_THREAD==0, so that we can skip all the locking code ? Because that's what we do right now. On Debian sid with apache2.0.54 mpm-prefork
Re: FeeBSD build (was mod_python 3.2.1b available for testing)
On Thu, 8 Sep 2005, Jim Gallacher wrote: I don't have FreeBSD, or any experience with any BSD, but I won't let that stop me from commenting. :) I don't see apr-0 listed in your includes in the above output. APR_THREAD_MUTEX_UNNESTED is defined in apr_thread_mutex.h, which on debian is in /usr/include/apr-0/. class ModPyExtension in dist/setup.py.in has been modified somewhat since the last release. This either broke the FreeBSD build or something is indeed misconfigured on your system. 'cept that ModPyExtension doesn't do anything on non-Win32... all those args come from apxs. I'll see if I can make some time to dig deeper. I'm hoping someone else with FreeBSD access can override my outcome, else this would be a -1... Grisha
Re: mod_python 3.2.1b available for testing
On Wed, 7 Sep 2005, Jorey Bump wrote: -1 Slackware Linux 10.1 Python 2.4.1 Apache 2.1.6 Alpha I don't think we mean to support 2.1.6 alpha, so this doesn't count. :-) Grisha
Re: mod_python 3.2.1b available for testing
Anybody got FreeBSD? I'm getting this. This is an old and possibly misconfigured system, so the problem could be on my end. FreeBSD 4.9 apache 2.0.53 (from ports) python 2.3.3 $ make Compiling for DSO. /usr/local/sbin/apxs -I/home/grisha/src/tmp/mod_python-3.2.1b/src/include -I/usr/local/include/apache2 -I/usr/local/include/python2.3 -c mod_python.c _apachemodule.c requestobject.c tableobject.c util.c serverobject.c connobject.c filterobject.c hlist.c hlistobject.c -Wl,--export-dynamic -pthread -lm /usr/local/lib/python2.3/config/libpython2.3.a-lutil -lm /usr/local/share/apache2/build/libtool --silent --mode=compile cc -prefer-pic -O -pipe -DAP_HAVE_DESIGNATED_INITIALIZER -D_REENTRANT -D_THREAD_SAFE -I/usr/local/include/apache2 -I/usr/local/include/apache2 -I/usr/local/include/apache2 -I/usr/local/include -I/home/grisha/src/tmp/mod_python-3.2.1b/src/include -I/usr/local/include/apache2 -I/usr/local/include/python2.3 -c -o mod_python.lo mod_python.c touch mod_python.slo mod_python.c:34: syntax error before `*' mod_python.c:34: warning: data definition has no type or storage class mod_python.c: In function `python_cleanup': mod_python.c:238: warning: passing arg 1 of `free' discards qualifiers from pointer target type mod_python.c: In function `python_init': mod_python.c:517: `APR_THREAD_MUTEX_UNNESTED' undeclared (first use in this function) mod_python.c:517: (Each undeclared identifier is reported only once mod_python.c:517: for each function it appears in.) apxs:Error: Command failed with rc=65536 . *** Error code 1 Stop in /usr/home/grisha/src/tmp/mod_python-3.2.1b/src. *** Error code 1 Stop in /usr/home/grisha/src/tmp/mod_python-3.2.1b.
Re: Getting ready for 3.2 beta 2
I've been away this weekend - just got back, but I'm too busy to try to read all the multiple-interpreter related comments. I guess my question is - can someone provide a quick summary of how far we are from 3.2.1b test tarbal? Thanks! Grisha On Thu, 1 Sep 2005, Graham Dumpleton wrote: Nicolas Lehuen wrote .. Well I for one am happy woth MODPYTHON-73, I've integrated Graham's patch and made unit test to check if everything was OK. Graham should be happy too :). As troublesome as I am, even I am happy at this point. :-) Unfortunately, probably will not be able to do any last build checks on MacOSX. I think I'll get killed tonight if I start working on the computer tonight since I fly off quite early tomorrow morning. If I am lucky I'll get just enough time to sync from the svn repository and then I'll play with it on the plane. I'll then be off the Internet for about 4-5 days so if there are any problems you will not hear about them in time anyway. Graham
Re: Getting ready for 3.2 beta 2
On Tue, 6 Sep 2005, Jim Gallacher wrote: As Graham stated on the weekend, the use of thread states can be very tricky. I think we should proceed with the 3.2.1b without the fix. That way we can take the time to make sure we understand the issues and fix it in 3.3. If that seems reasonable, I'll make the tarball today. +1 Grisha
Re: Getting ready for 3.2 beta 2
Or speaking in diff (not tested):
--- setup.py.in.orig2005-09-01 11:42:09.082202944 -0400
+++ setup.py.in 2005-09-01 11:44:35.969872624 -0400
@@ -140,18 +140,24 @@
# this is a hack to prevent build_ext from trying to append
initmod_python to the export symbols
self.export_symbols = finallist(self.export_symbols)
-ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(),
getapache_includedir()], [getapache_libdir()])
if winbuild:
+
+# build mod_python.so
+ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(),
getapache_includedir()], [getapache_libdir()])
+
scripts = [win32_postinstall.py]
# put the mod_python.so file in the Python root ...
# win32_postinstall.py will pick it up from there...
# data_files = [(, [(os.path.join(getmp_srcdir(), 'Release',
'mod_python.so'))])]
data_files = []
+ext_modules = [ModPyModule, PSPModule]
+
else:
-# mpso = ../src/mod_python.so
+
scripts = []
data_files = []
+ext_modules = [PSPModule]
import string
from distutils import sysconfig
@@ -174,7 +180,7 @@
package_dir={'mod_python': os.path.join(getmp_rootdir(), 'lib',
'python', 'mod_python')},
scripts=scripts,
data_files=data_files,
- ext_modules=[ModPyModule, PSPModule])
+ ext_modules=ext_modules)
# makes emacs go into python mode
### Local Variables:
On Thu, 1 Sep 2005, Gregory (Grisha) Trubetskoy wrote:
On Wed, 31 Aug 2005, Jim Gallacher wrote:
3. Eliminate creation of mod_python_so.so in non-windows environments.
Fix is ready to commit.
Not Done. I decided to defer this for reasons I won't go into just now.
It is not a show stopper anyway.
Isn't the fix basically just placing the ModPyModule and setup() with
ModPyModule inside the if winbuild block and then having another set()
without the ModPyModule in the else clause?
Unless there is some good reason for it, I think it is a show stopper because
it makes the build process look a bit on the bizzare side on Unix.
Grisha
Re: Getting ready for 3.2 beta 2
On Thu, 25 Aug 2005, Jim Gallacher wrote: I think we should aim for the second beta release in the next couple of days. I have a few questions and a list of outstanding issues. Name of tarball: mod_python-3.2.1b.tgz? yep, 3.2.1b Also, I assume a new branch called tags/release-3.2.1-BETA will be created in subversion, correct? yup Regards, Jim Thanks Jim! Grisha
Re: flex [was mod_python 3.2.0-BETA available for testing]
On Fri, 26 Aug 2005, Nick wrote: Here's another idea: Fail the flex test fairly silently (e.g. just no), but fall back to a script that generates a nice, verbose error message explaining the situation. That way, when the user tries to call make after modifying the .l file, the fake flex alternative script gets called, displays the message, and exits with status 1. Yep, you probably don't need a separate script - it can all fit in the Makefile somewhere. That is how it worked originally, sort of, 'cept there was no message - it just failed if flex wasn't there ;-) But I'm also perfectly fine with what Jim already has in configure.in, if we don't have time for these refinements, I'd leave it as it is, no biggie. Grisha
Re: 3.2 beta release today?
On Tue, 16 Aug 2005, Jim Gallacher wrote: What is the correct name for the tarball? mod_python-3.2.0-b.tgz, mod_python-3.2.0-BETA.tgz or something else? I like mod_python-3.2.0b and the tarball would be mod_python-3.2.0b.tgz Grisha
Re: 3.2 beta release today?
I think the best thing to do is just to go ahead and tag and a create a tarball. Whether everyone was ready will become apparent during the testing/voting. Grisha On Mon, 15 Aug 2005, Jim Gallacher wrote: Are we on track to release a 3.2.0beta tarball today? Regards, Jim
Re: [jira] Commented: (MODPYTHON-70) Add configure --with-max-locks option to set MAX_LOCKS.
On Tue, 9 Aug 2005, Jim Gallacher wrote: My question is : should we keep on with ./configure ; make ; make install or try to do everything in setup.py ? As long as we can put setup.py in a Makefile. ;) Seriously though, ./configure --help; ./configure; make; make install; is just such second nature to me I never even thought of the alternative. Yep, and I think setup.py isn't smart enough to deal with the Apache side of mod_python which is C-centric. But in any event, I think ./configure is the way to go. Grisha
Re: Release schedule
On Mon, 8 Aug 2005, Jim Gallacher wrote: I had to double escape the r, ie \\\release to make it work. Grisha, did you generate the docs on BSD before, and if so is the BSD sed different? Yes, it was always done on FreeBSD before - it is quite likely that the sed is different More likely it's make - the BSD make is _definitely_ different from GNU make. Grisha
Re: 3.2
Just thought I'd ask if we're making any progress towards a 3.2 tarball to test. No pressure, just curious :-) Grisha
Re: Potential deadlock in psp.py
Yeah, we've got to be inline with the HTTP Project - prefork is the default on unix systems, so we have to abide by it... So I guess the solution is that we need to reserve two locks instead of just one? Grisha On Thu, 23 Jun 2005, Jim Gallacher wrote: Nicolas Lehuen wrote: Hi Jim, Until now, we suspected that the way global locks are handled could be deadlock prone. You have just proved it. I know that global locks are expensive on some systems, especially if we want to use them in a multiprocess (forked) environment. That's why we are forced to have such a small pool of global locks. On the other hand, in a multithreaded environment, locks which are valid for the whole process are not so expensive ; indeed, we can create a whole bunch without bringing down the system (think about Java where all object potentially have a monitor which is equivalent to a lock). I think this is a strong incentive to abandon the forked model and go for the multi-threaded model (or the mono-threaded, asynchronous one). For concurrency control, the forked model does not scale, apparently. It would make life easier I suppose, but we are stuck with the fact that apache provides several different mpm models that need to be supported. Jim
Re: Session Benchmarks
On Fri, 17 Jun 2005, Nicolas Lehuen wrote: As for the MySQL implementation I'd stay away from anything vendor-specific in mod_python, because then the question becomes why not a postresql, why not oracle, etc. Grisha
New committer: Jim Gallacher
Hi folks - We have a new committer - Jim Gallacher. Thanks Jim for all your contributions to mod_python and for accepting the invitation to become a committer! Cheers, Grisha
