Re: 3.2.6 test period - how long do we wait?
+1 Release what's fixed already, and then keep going afterwards. But can somebody address MODPYTHON-53 please. (updating modpython.org website). -- Deron Meranda
Re: mod_python as a mod_dav backend
Actually it seems that this is yet another case of trying to get mod_python to hook into more places in the Apache framework; specifically to hook into other modules. We've already been discussing specific-module hooks for mod_ssl - http://issues.apache.org/jira/browse/MODPYTHON-94 mod_include - http://issues.apache.org/jira/browse/MODPYTHON-104 In fact there's a bit of potentially useful discussion in the notes for those JIRA issues which is valid for any module-hooking in general (along with this mailing list archives). I'd say it might be worthwhile to at least create a new feature-request issue for mod_dav hooks though; just so it's on the record. BTW, even though it's not what you asked, it should be possible to totally implement DAV using only the features already present in mod_python (without mod_dav); but of course you're doing a lot of the work yourself (but on the other hand you get to use Python rather than C). -- Deron Meranda
Re: mod_python as a mod_dav backend
On 02/02/2006, at 6:52 AM, Deron Meranda wrote: Actually it seems that this is yet another case of trying to get mod_python to hook into more places in the Apache framework; specifically to hook into other modules. We've already been discussing specific-module hooks for mod_ssl - http://issues.apache.org/jira/browse/MODPYTHON-94 mod_include - http://issues.apache.org/jira/browse/MODPYTHON-104 In fact there's a bit of potentially useful discussion in the notes for those JIRA issues which is valid for any module-hooking in general (along with this mailing list archives). I'd say it might be worthwhile to at least create a new feature-request issue for mod_dav hooks though; just so it's on the record. Adding in support for mod_ssl and mod_include was reasonably trivial and a natural fit for the basic functionality of mod_python. Support for mod_dav development using Python would be a considerable amount of work as it involves construction of a range of new Python wrapper classes for C structures and call in infrastructure for Python which would be quite tailored for that purpose. Having mod_dav support goes far beyond the basic charter of what mod_python is about and all this extra work would be an undue burden on core mod_python developers and cause even more delays in any releases. Thus why I see that you would be better of having a separate project for a mod_dav_python module. There would still be changes required in mod_python to allow the separate module to hook into the basic infrastructure, but this would be a more practical way of going about it and would also allow other distinct modules besides mod_dav_python to be developed which could have bindings for Python. Graham
Re: Python 2.2 support
Again this is a resend. I post one message via my secure SMTP and it vanishes. Post one via normal SMTP and it goes to list straight away. This sort of confirms what I suspected which is that my ISPs secure SMTP is busted somehow in that randomly drops email. :-( Sorry for the duplicate if first one does appear eventually. On 01/02/2006, at 9:10 PM, Nicolas Lehuen wrote: Hi, I've just checked in some changes to the Python source code in order to support Python 2.2. Now the test suite runs successfully on Python 2.2.3 on Windows 2000. I've checked that no regressions were introduced in later Python versions, too. The changes are pretty simple : each Python module now features a from python22 import *. The mod_python.python22 module just reimplements new builtins from Python 2.3. It turns out that the only missing builtin for now is enumerate(). The tests module, containing a few tests for generators, has to sport a from __future__ import generators line. Does that mean that 3.2.6 is now no more if you are checking stuff in and we can't avoid a 3.2.7b? I understood the only Python 2.3 dependency to be the use of enumerate() in mod_python.publisher. If that was true, why couldn't the one place be changed rather than modifying all modules with this import? I also had to change mod_python.cache which used time.strptime so that it uses rfc822.parsedate, now. Two things to change then. I've did this because the guys from Nokia use Python 2.2 and mod_python 3.1.3. They spotted memory leaks which are likely to be fixed in mod_python 3.2.X, but they could not upgrade if Python 2.2 was not supported. They are only one customer. I know they might be seen to be of high interest because of the cool factor of what they have done, but should we have really deferred doing a release because of it. Sorry to be the release Nazi, but this really is starting to drag on. The Python 2.2 backward compatibility changes could have been documented in JIRA and people referred to that if they really wanted to make 3.2.6 work with that version of Python. :-( Graham
Re: Python 2.2 support
Graham Dumpleton wrote: On 01/02/2006, at 9:10 PM, Nicolas Lehuen wrote: Hi, I've just checked in some changes to the Python source code in order to support Python 2.2. Now the test suite runs successfully on Python 2.2.3 on Windows 2000. I've checked that no regressions were introduced in later Python versions, too. The changes are pretty simple : each Python module now features a from python22 import *. The mod_python.python22 module just reimplements new builtins from Python 2.3. It turns out that the only missing builtin for now is enumerate(). The tests module, containing a few tests for generators, has to sport a from __future__ import generators line. Does that mean that 3.2.6 is now no more if you are checking stuff in and we can't avoid a 3.2.7b? Not necessarily. If 3.2.6 is good then it doesn't matter what is in trunk. There is a 3.2.6 snapshop in tags/. I think it's time to give 3.2 it's own svn branch so we *can* start checking stuff into trunk again. If we do need to do a 3.2.7 release in the near future it can come from the 3.2.x branch. So, I propose creating a new branches/3.2.x svn branch today. Any objections? I understood the only Python 2.3 dependency to be the use of enumerate() in mod_python.publisher. If that was true, why couldn't the one place be changed rather than modifying all modules with this import? I also had to change mod_python.cache which used time.strptime so that it uses rfc822.parsedate, now. Two things to change then. I've did this because the guys from Nokia use Python 2.2 and mod_python 3.1.3. They spotted memory leaks which are likely to be fixed in mod_python 3.2.X, but they could not upgrade if Python 2.2 was not supported. They are only one customer. I know they might be seen to be of high interest because of the cool factor of what they have done, but should we have really deferred doing a release because of it. Sorry to be the release Nazi, but this really is starting to drag on. The Python 2.2 backward compatibility changes could have been documented in JIRA and people referred to that if they really wanted to make 3.2.6 work with that version of Python. :-( I think it may have been a mistake to drop python 2.2 support just yet. Aren't some RHAT enterprise systems still stuck at that python version? I suspect we won't see this issue raised until *after* mp 3.2 is released. Nicolas' patch python 2.2 patch can be backported to 3.2.x branch for a bugfix release. Jim
Re: Python 2.2 support
Nicolas Lehuen writes: I've just checked in some changes to the Python source code in order to support Python 2.2. Now the test suite runs successfully on Python 2.2.3 on Windows 2000. I've checked that no regressions were introduced in later Python versions, too. The changes are pretty simple : each Python module now features a from python22 import *. The mod_python.python22 module just reimplements new builtins from Python 2.3. It turns out that the only missing builtin for now is enumerate(). The tests module, containing a few tests for generators, has to sport a from __future__ import generators line. I also had to change mod_python.cache which used time.strptime so that it uses rfc822.parsedate, now. I've did this because the guys from Nokia use Python 2.2 and mod_python 3.1.3. They spotted memory leaks which are likely to be fixed in mod_python 3.2.X, but they could not upgrade if Python 2.2 was not supported. But does this scale over the life of a project? Every new python module has to include 'from python22 import *'?? While never shying away from a decent hack myself, this is a hack that affects everyone, not just those that need the hack, which, for me, screams volumes. Certainly, which version of python mod_python is based on should not be taken lightly. Was it formally decided to make mod_python dependent on 2.3+? If not, then perhaps uses of enumerate, etc. should not be used. If a formal decision was made, then it's a done deal, right? If not and uses of 2.3 have slipped in then perhaps it's a done deal anyway because no one can stomach the thought of taking out the 2.3-isms at this late date. Regardless, I do not think it is within the scope of mod_python developers to keep users forward-compatible with the underlying python version. Sorry, but IMHO, this is not scalable software engineering. That said, I don't have a problem helping 2.2 users write their own hacks. For example, let's take enumerate...another way of solving this problem without touching any mod_python code would be a suggestion in the FAQ to write a module, say, python22hacks.py: # python22hacks.py # # This is unsupported software offered to mod_python users who are # stuck using python 2.2. # # Install by placing this module in the same directory with other # mod_python modules and add the following to your apache config: # #PythonImport python22hacks INTERPRETER_NAME # # More disclaimers, blah, blah... import __builtin__ as hack hack.enumerate = lambda s: zip(xrange(len(s)), s) This way we don't have to touch every module of source code and the onus is on the few who need it rather than the many who don't or those who have to maintain it. Cheers, Daniel Popowich --- http://home.comcast.net/~d.popowich/mpservlets/
Worrying code in mod_python.publisher module importer.
I was looking at the new module importer used by mod_python.publisher in 3.2.6 to see whether it reloaded a module if file was replaced with an older file and have come across some code that worries me a bit. Can someone else (not just Nicolas) check this code and how it is used in the context of the module importer and reassure me that the code isn't going to cause a problem? The code in question is in the FileCache class of mod_python.cache. The ModuleCache class used by mod_python.publisher derives from FileCache. The actual code is: def check(self, key, name, entry): opened = file(key, self.mode) timestamp = fstat(opened.fileno())[-2] if entry._value is NOT_INITIALIZED: entry._timestamp = timestamp return opened else: if entry._timestamp != timestamp: entry._timestamp = timestamp return opened else: opened.close() return None This code I am assuming is what is determining if a file has changed on disk and thus would be the trigger for a module being reloaded. First off, it worries me that it actually opens the file on disk and then does a fstat() on it. If the file hasn't changed, it would close the file just opened and use whatever is cached in the process instead. Given that 99% of the time, the file will not have changed on disk, this seems to be an inefficient way of doing this. I would have thought that a stat() call would have less overhead and only if file has deemed to have changed then would then the actual file on disk be opened. Anyway, this issue is not what really worries me. What I see as a much bigger problem is that the whole result of fstat() is what is cached as the timestamp and what is used for comparing later to determine if the file has changed and should be reloaded. Now the result of calling os.fstat() is something like: (33188, 3157895L, 234881033L, 1, 501, 501, 32367L, 1138847501, 1138846987, 1138847194) Python documentation describes it as: The return value is an object whose attributes correspond to the members of the stat structure, namely: st_mode (protection bits), st_ino (inode number), st_dev (device), st_nlink (number of hard links), st_uid (user ID of owner), st_gid (group ID of owner), st_size (size of file, in bytes), st_atime (time of most recent access), st_mtime (time of most recent content modification), st_ctime (platform dependent; time of most recent metadata change on Unix, or the time of creation on Windows). Problem is that this contains lots of stuff besides the modification time of the file. This means that a module reload can occur in circumstances where the content of the file has not even changed. For example, the owner or group of the file is changed, or the number of hard links to the file has changed. The worst of all though is that the time of most recent access is in that tuple of information. This is the st_atime field which is documented as: st_atime Time when file data last accessed. Changed by the mknod(2), utimes(2) and read(2) system calls. Thus simply reading from the file or other operations on the file, will cause a module reload. This may not happen immediately, as it seems some operating systems don't necessarily flush out changes to the access time to disk immediately when no file changes are made, instead waiting until in memory file system cache information needs to be discarded. It will happen though at some point. Can someone check over my reasoning here and see if I am correct? Note that up until now I hadn't even looked over how this new module importer was implemented. I knew it wasn't going to solve various of the existing module importer problems and I knew it was actually going to introduce some new issues that would have to be worked around, but now that I have started to document these new issues for inclusion in my module importer issues list and when I see other possible problems like the above, I am really starting to wander if it is really a good idea letting this interim solution to module importing problems be released. Comments? Graham
Re: Worrying code in mod_python.publisher module importer.
Okay, false alarm (I think). Have got myself worked up over nothing. I missed something very important: timestamp = fstat(opened.fileno())[-2] That is the '[-2]' in the above. I feel like a goose now. I still though question why file/fstat is done and not stat/file though. Ie., why open the file to stat it? Graham Graham Dumpleton wrote .. I was looking at the new module importer used by mod_python.publisher in 3.2.6 to see whether it reloaded a module if file was replaced with an older file and have come across some code that worries me a bit. Can someone else (not just Nicolas) check this code and how it is used in the context of the module importer and reassure me that the code isn't going to cause a problem? The code in question is in the FileCache class of mod_python.cache. The ModuleCache class used by mod_python.publisher derives from FileCache. The actual code is: def check(self, key, name, entry): opened = file(key, self.mode) timestamp = fstat(opened.fileno())[-2] if entry._value is NOT_INITIALIZED: entry._timestamp = timestamp return opened else: if entry._timestamp != timestamp: entry._timestamp = timestamp return opened else: opened.close() return None This code I am assuming is what is determining if a file has changed on disk and thus would be the trigger for a module being reloaded. First off, it worries me that it actually opens the file on disk and then does a fstat() on it. If the file hasn't changed, it would close the file just opened and use whatever is cached in the process instead. Given that 99% of the time, the file will not have changed on disk, this seems to be an inefficient way of doing this. I would have thought that a stat() call would have less overhead and only if file has deemed to have changed then would then the actual file on disk be opened. Anyway, this issue is not what really worries me. What I see as a much bigger problem is that the whole result of fstat() is what is cached as the timestamp and what is used for comparing later to determine if the file has changed and should be reloaded. Now the result of calling os.fstat() is something like: (33188, 3157895L, 234881033L, 1, 501, 501, 32367L, 1138847501, 1138846987, 1138847194) Python documentation describes it as: The return value is an object whose attributes correspond to the members of the stat structure, namely: st_mode (protection bits), st_ino (inode number), st_dev (device), st_nlink (number of hard links), st_uid (user ID of owner), st_gid (group ID of owner), st_size (size of file, in bytes), st_atime (time of most recent access), st_mtime (time of most recent content modification), st_ctime (platform dependent; time of most recent metadata change on Unix, or the time of creation on Windows). Problem is that this contains lots of stuff besides the modification time of the file. This means that a module reload can occur in circumstances where the content of the file has not even changed. For example, the owner or group of the file is changed, or the number of hard links to the file has changed. The worst of all though is that the time of most recent access is in that tuple of information. This is the st_atime field which is documented as: st_atime Time when file data last accessed. Changed by the mknod(2), utimes(2) and read(2) system calls. Thus simply reading from the file or other operations on the file, will cause a module reload. This may not happen immediately, as it seems some operating systems don't necessarily flush out changes to the access time to disk immediately when no file changes are made, instead waiting until in memory file system cache information needs to be discarded. It will happen though at some point. Can someone check over my reasoning here and see if I am correct? Note that up until now I hadn't even looked over how this new module importer was implemented. I knew it wasn't going to solve various of the existing module importer problems and I knew it was actually going to introduce some new issues that would have to be worked around, but now that I have started to document these new issues for inclusion in my module importer issues list and when I see other possible problems like the above, I am really starting to wander if it is really a good idea letting this interim solution to module importing problems be released. Comments? Graham
Re: Worrying code in mod_python.publisher module importer.
2006/2/2, Graham Dumpleton [EMAIL PROTECTED]: Okay, false alarm (I think). Have got myself worked up over nothing. I missed something very important: timestamp = fstat(opened.fileno())[-2] That is the '[-2]' in the above. I feel like a goose now. I still though question why file/fstat is done and not stat/file though. Ie., why open the file to stat it? Graham Well, I thought that if the file was modified, we needed to open it anyway, but you're right, that's optimising for a minority case. We might as well use stat and open the file only if it has changed. I've wrote an alternative publisher a few months ago that overloaded this behaviour in the module cache to use req.finfo[apache.FINFO_MTIME] as the file modification time, thus saving us a call to fstat or stat entirely. I've stopped using this publisher because I thought that using the standard publisher was a better way to see how we could improve it, but anyway, I could back port this trick. If I don't get burned down by the flak I'm currently getting on the Python 2.2 issue, that is ;). Regards, Nicolas
Re: Worrying code in mod_python.publisher module importer.
2006/2/2, Graham Dumpleton [EMAIL PROTECTED]: Note that up until now I hadn't even looked over how this new module importer was implemented. I knew it wasn't going to solve various of the existing module importer problems and I knew it was actually going to introduce some new issues that would have to be worked around, but now that I have started to document these new issues for inclusion in my module importer issues list and when I see other possible problems like the above, I am really starting to wander if it is really a good idea letting this interim solution to module importing problems be released. Comments? Graham You know, Graham, I'm very frustrated about this because we decided not to go any further on the module importer issue until we reach 3.3. Hence, I have stopped any development on this level and kept the code as is (i.e. in a working state), hoping that the 3.2 release would come soon and that we would be able to move on quickly. More than six months later we're still at the same point and now you're beginning to ask questions about the interim solution. Well, indeed, it's an interim solution, but it works and fixes a lot of bugs. It's not perfect, we'll surely have some people asking us how to import one published module from another one (I had wrote some code to support that, but it was refused), but it was never supposed to last long. Anyway, I'd like to point out that I've been using this publisher in various professional projects for months now without having any problems. It's not like we are releasing something flaky. The only problem is that apache.import_module is still as crappy as ever and that we don't have any grand unified theory of module importing that would support both handlers and published modules. Regards, Nicolas
Re: Worrying code in mod_python.publisher module importer.
Graham Dumpleton wrote .. Okay, false alarm (I think). Have got myself worked up over nothing. I missed something very important: timestamp = fstat(opened.fileno())[-2] That is the '[-2]' in the above. I feel like a goose now. Now for some explaination of why my brain turned off and I didn't see the '[-2]'. My excuse is that personally I wouldn't use negative indexing from the end of the stat structure. It therefore didn't look like what I would use and I got confused. I don't use '[-2]' as I am not sure you can. Quoting the Python documentation, it suggests that the tuple returns may actually contain more than 10 values. For backward compatibility, the return value of stat() is also accessible as a tuple of at least 10 integers giving the most important (and portable) members of the stat structure, in the order st_mode, st_ino, st_dev, st_nlink, st_uid, st_gid, st_size, st_atime, st_mtime, st_ctime. More items may be added at the end by some implementations. The standard module stat defines functions and constants that are useful for extracting information from a stat structure. (On Windows, some items are filled with dummy values.) Thus, I would always use: timestamp = fstat(opened.fileno())[stat.ST_MTIME] The use of '[-2]' might not actually be portable and thus stat.ST_MTIME should probably be used. I still though question why file/fstat is done and not stat/file though. Ie., why open the file to stat it? Should clarify that latter would be stat/file/fstat to be totally accurate. In other words, use stat() to determine if it has changed, then open file and then fstat() to make sure have the true modification time in case it just changed. Even though there is a double stat, this only occurs when file changes. In balance, I would have thought this would still be better than opening the file every time. Yes/No? Graham
Re: Python 2.2 support
Quick response, because it's 4:36 AM here, I just woke up to feed my daughter and took all this flak, but I need to sleep :). I guess that's the problem of having a round-the-planet development team, between those in America, Europe, Asia and Australia (nobody from Antarctica yet ?) Graham : Considering that Grisha wanted your fix for the connection handler included in the 3.2 release, I thought that the 3.2.6 release was DOA, so I didn't feel bad about checkin this in. Anyway, Jim has the good idea there : We've got a clean 3.2.6 tag, so we can release it and either branch on it or stay on the trunk. We don't have to stop everything just because we want to release something, Subversion is here to help. Daniel : I think that from python22 import * is no more troublesome as the from __future__ import generators that you have to put in any source code that feature generators and needs to run from Python 2.2 and upward. But that's a matter of taste. Maybe we could do something like you wrote, for example in a mod_python.python22support module that Python 2.2 users would import using PythonImport ? The only problem is that the current solution passes all the test suite from Python 2.2 to Python 2.4 (I've yet to test it on Python 2.5), whereas a solution using PythonImport or any custom tweaking of the configuration would require a special case in the test suite. Now I'm going back to sleep. Regards, Nicolas 2006/2/2, Daniel J. Popowich [EMAIL PROTECTED]: Nicolas Lehuen writes: I've just checked in some changes to the Python source code in order to support Python 2.2. Now the test suite runs successfully on Python 2.2.3 on Windows 2000. I've checked that no regressions were introduced in later Python versions, too. The changes are pretty simple : each Python module now features a from python22 import *. The mod_python.python22 module just reimplements new builtins from Python 2.3. It turns out that the only missing builtin for now is enumerate(). The tests module, containing a few tests for generators, has to sport a from __future__ import generators line. I also had to change mod_python.cache which used time.strptime so that it uses rfc822.parsedate, now. I've did this because the guys from Nokia use Python 2.2 and mod_python 3.1.3. They spotted memory leaks which are likely to be fixed in mod_python 3.2.X, but they could not upgrade if Python 2.2 was not supported. But does this scale over the life of a project? Every new python module has to include 'from python22 import *'?? While never shying away from a decent hack myself, this is a hack that affects everyone, not just those that need the hack, which, for me, screams volumes. Certainly, which version of python mod_python is based on should not be taken lightly. Was it formally decided to make mod_python dependent on 2.3+? If not, then perhaps uses of enumerate, etc. should not be used. If a formal decision was made, then it's a done deal, right? If not and uses of 2.3 have slipped in then perhaps it's a done deal anyway because no one can stomach the thought of taking out the 2.3-isms at this late date. Regardless, I do not think it is within the scope of mod_python developers to keep users forward-compatible with the underlying python version. Sorry, but IMHO, this is not scalable software engineering. That said, I don't have a problem helping 2.2 users write their own hacks. For example, let's take enumerate...another way of solving this problem without touching any mod_python code would be a suggestion in the FAQ to write a module, say, python22hacks.py: # python22hacks.py # # This is unsupported software offered to mod_python users who are # stuck using python 2.2. # # Install by placing this module in the same directory with other # mod_python modules and add the following to your apache config: # #PythonImport python22hacks INTERPRETER_NAME # # More disclaimers, blah, blah... import __builtin__ as hack hack.enumerate = lambda s: zip(xrange(len(s)), s) This way we don't have to touch every module of source code and the onus is on the few who need it rather than the many who don't or those who have to maintain it. Cheers, Daniel Popowich --- http://home.comcast.net/~d.popowich/mpservlets/
Re: Worrying code in mod_python.publisher module importer.
Nicolas Lehuen wrote ..
Well, I thought that if the file was modified, we needed to open it
anyway, but you're right, that's optimising for a minority case. We
might as well use stat and open the file only if it has changed.
I've wrote an alternative publisher a few months ago that overloaded
this behaviour in the module cache to use
req.finfo[apache.FINFO_MTIME] as the file modification time, thus
saving us a call to fstat or stat entirely. I've stopped using this
publisher because I thought that using the standard publisher was a
better way to see how we could improve it, but anyway, I could back
port this trick. If I don't get burned down by the flak I'm currently
getting on the Python 2.2 issue, that is ;).
I'd rather you not use:
req.finfo[apache.FINFO_MTIME]
for the same reasons I got you not to use it last time.
This is because now that req.filename/req.path_info are both writable,
it is possible for someone in their own handler to modify these values,
thus overriding Apache's own decisions about what matched the URL
and then explicitly invoke mod_python.publisher to service it. Because
req.finfo is not updatable by a handler, it isn't possible for a handler
to override it to make it consistent with changes to req.filename.
Thus in code I have:
from lamia import handlers
from mod_python import publisher
__all__ = [ 'MapLocationToPublisher' ]
class MapLocationToPublisher:
def __init__(self,**kwargs):
self.__call__ = handlers.MapLocationToView(
directory = kwargs.pop('directory'),
resource_extension = kwargs.pop('resource_extension','.py'),
script_extension = '.py',
handler = publisher.handler,
)
The MapLocationToView handler object is doing stuff similar to what
Apache does as far as mapping URLs to files, but is actually applying
it against a configurable directory which doesn't even have to be in
the document tree. Once it maps to file and recalculates req.filename
and req.path_info appropriately, it calls mod_python.publisher.handler()
explicitly.
Graham
Re: Python 2.2 support
Daniel J. Popowich wrote: Nicolas Lehuen writes: I've just checked in some changes to the Python source code in order to support Python 2.2. Now the test suite runs successfully on Python 2.2.3 on Windows 2000. I've checked that no regressions were introduced in later Python versions, too. The changes are pretty simple : each Python module now features a from python22 import *. The mod_python.python22 module just reimplements new builtins from Python 2.3. It turns out that the only missing builtin for now is enumerate(). The tests module, containing a few tests for generators, has to sport a from __future__ import generators line. I also had to change mod_python.cache which used time.strptime so that it uses rfc822.parsedate, now. I've did this because the guys from Nokia use Python 2.2 and mod_python 3.1.3. They spotted memory leaks which are likely to be fixed in mod_python 3.2.X, but they could not upgrade if Python 2.2 was not supported. But does this scale over the life of a project? Every new python module has to include 'from python22 import *'?? While never shying away from a decent hack myself, this is a hack that affects everyone, not just those that need the hack, which, for me, screams volumes. Hmm, good points. Certainly, which version of python mod_python is based on should not be taken lightly. Was it formally decided to make mod_python dependent on 2.3+? If not, then perhaps uses of enumerate, etc. should not be used. If a formal decision was made, then it's a done deal, right? If not and uses of 2.3 have slipped in then perhaps it's a done deal anyway because no one can stomach the thought of taking out the 2.3-isms at this late date. My impression is that there was never really a discussion on this issue. Some 2.3-isms got used, so it was decided that 2.2 was not supported. There likely should have been a more formal discussion on whether this is right time to drop 2.2 support. Certainly we haven't done any testing using python 2.2 so even with the hack I don't think we can comfortably claim that that version is supported. Regardless, I do not think it is within the scope of mod_python developers to keep users forward-compatible with the underlying python version. Sorry, but IMHO, this is not scalable software engineering. I'll re-read this paragraph after a good sleep, but right now I'm not sure if you mean we shouldn't be using features only available in newer python versions, or we shouldn't worry about compatibilty with with older python versions? That said, I don't have a problem helping 2.2 users write their own hacks. For example, let's take enumerate...another way of solving this problem without touching any mod_python code would be a suggestion in the FAQ to write a module, say, python22hacks.py: # python22hacks.py # # This is unsupported software offered to mod_python users who are # stuck using python 2.2. # # Install by placing this module in the same directory with other # mod_python modules and add the following to your apache config: # #PythonImport python22hacks INTERPRETER_NAME # # More disclaimers, blah, blah... import __builtin__ as hack hack.enumerate = lambda s: zip(xrange(len(s)), s) This way we don't have to touch every module of source code and the onus is on the few who need it rather than the many who don't or those who have to maintain it. I like this solution. Jim
Re: Python 2.2 support
2006/2/2, Jim Gallacher [EMAIL PROTECTED]: If a formal decision was made, then it's a done deal, right? If not and uses of 2.3 have slipped in then perhaps it's a done deal anyway because no one can stomach the thought of taking out the 2.3-isms at this late date. My impression is that there was never really a discussion on this issue. Some 2.3-isms got used, so it was decided that 2.2 was not supported. There likely should have been a more formal discussion on whether this is right time to drop 2.2 support. Certainly we haven't done any testing using python 2.2 so even with the hack I don't think we can comfortably claim that that version is supported. Note that I have ran successfully the unit tests on Python 2.2, 2.3 and 2.4 before checking this hack in. Granted, this is not a guarantee that Python 2.2 is supported, but given our tests coverage, this is pretty good. Regards, Nicolas
Re: Worrying code in mod_python.publisher module importer.
Nicolas Lehuen wrote .. 2006/2/2, Graham Dumpleton [EMAIL PROTECTED]: Note that up until now I hadn't even looked over how this new module importer was implemented. I knew it wasn't going to solve various of the existing module importer problems and I knew it was actually going to introduce some new issues that would have to be worked around, but now that I have started to document these new issues for inclusion in my module importer issues list and when I see other possible problems like the above, I am really starting to wander if it is really a good idea letting this interim solution to module importing problems be released. Comments? Graham You know, Graham, I'm very frustrated about this because we decided not to go any further on the module importer issue until we reach 3.3. Hence, I have stopped any development on this level and kept the code as is (i.e. in a working state), hoping that the 3.2 release would come soon and that we would be able to move on quickly. More than six months later we're still at the same point and now you're beginning to ask questions about the interim solution. Actually, I raised the issues before a couple of times, so in my email I should really have said I am really starting to wander AGAIN if it is really a good idea. One of the previous emails on this can be seen: http://article.gmane.org/gmane.comp.apache.mod-python.devel/1269 Back then I felt I had no real power to influence anything, or at least I was not part of any final decision making process. Bar your concession of dropping get_page(), there was no other comment by anyone else so to me it seemed there was no point me pursuing it further even though I still had the reservations I mentioned in the email at the time. Anyway, I'd like to point out that I've been using this publisher in various professional projects for months now without having any problems. It's not like we are releasing something flaky. Which is why I hadn't bothered to look at the module importer part of it. I trusted you and that the code you added would work okay. Anyway, we are letting my comments get blow out of proportion now given that I have already acknowledged that my premise for my revived misgivings was bogus because I simply didn't read the code correctly. The only problem is that apache.import_module is still as crappy as ever and that we don't have any grand unified theory of module importing that would support both handlers and published modules. Actually I do believe I have a grand unified theory and I have working code for it was well. I simply gave up pushing it some time back because the initial discussions I started about it on the mailing list degenerated into a multitude of threads with everyone giving their own viewpoints, with many not really understanding what I was proposing and others not understanding why it had to be changed anyway. Thus I gave up and reverted to just documenting the issues I find on my web site. In time I will also add documentation for how my proposed changes and code actually address or don't address the issues I have documented. Until that is done, making any code available isn't going to help. Graham
Re: Worrying code in mod_python.publisher module importer.
OK, I've changed cache.py so that it uses stat() then open() the file if it needs to be reloaded. I've also added a unit test that makes sure the module cache is behaving as expected. Graham, I don't think the stat() / open() / fstat() sequence is required. How would that improve accuracy ? Regards, Nicolas
Re: Python 2.2 support
OK, I've reverted my changes. I left python22.py in place, because I still hope to be able to use it with PythonImport. The only problem is being able to define it in the unit tests. Regards, Nicolas 2006/2/2, Nicolas Lehuen [EMAIL PROTECTED]: 2006/2/2, Jim Gallacher [EMAIL PROTECTED]: If a formal decision was made, then it's a done deal, right? If not and uses of 2.3 have slipped in then perhaps it's a done deal anyway because no one can stomach the thought of taking out the 2.3-isms at this late date. My impression is that there was never really a discussion on this issue. Some 2.3-isms got used, so it was decided that 2.2 was not supported. There likely should have been a more formal discussion on whether this is right time to drop 2.2 support. Certainly we haven't done any testing using python 2.2 so even with the hack I don't think we can comfortably claim that that version is supported. Note that I have ran successfully the unit tests on Python 2.2, 2.3 and 2.4 before checking this hack in. Granted, this is not a guarantee that Python 2.2 is supported, but given our tests coverage, this is pretty good. Regards, Nicolas
Re: Python 2.2 support
2006/2/2, Graham Dumpleton [EMAIL PROTECTED]: Nicolas Lehuen wrote .. OK, I've reverted my changes. I left python22.py in place, because I still hope to be able to use it with PythonImport. The only problem is being able to define it in the unit tests. I plead dumb. What is the connection to PythonImport? My only guess at the moment is that it does something like I do in my new importer, which is to use PythonImport to import a module which goes in and fiddles with the contents of mod_python.apache/publisher to patch in my new code before any request handlers get a chance to be called. In this way I don't have to be patching the actual mod_python source code. Graham That's it ! People with Python 2.2 could use PythonImport mod_python.python22 INTERPRETER_NAME in their configuration file to make sure mod_python supports Python 2.2. The only problem is the need to provide an interpreter name, which complicates things a little bit in the case of the test suite. Then again, the only thing which prevents Python 2.2 support right now is the use of enumerate(), so we could just check whether we could do without enumerate() and support Python 2.2 out of the box. Regards, Nicolas
