Re: 3.2.6 or not
On Thu, Feb 02, 2006 at 10:54:27PM -0500, Jim Gallacher wrote: Graham Dumpleton wrote: To confirm Jim's arithmetic anyway, I say -1 on 3.2.6 as it stands. As to 3.2.7, I say +1, subject to removal of problematic test case as already raised and with us at least confirming tests run OK for version out of SVN prior to packaging. Oh, so *that's* where you said we should disable the test. ;) I've disabled the publisher_cache test and the connection handler fix has been checked in. We are good to go for 3.2.7. As Graham suggests perhaps we could get some people to confirm the current version in SVN trunk prior to packaging. A couple of thumbs up from some FreeBSD'ers would be especially nice since the connection handler problem seemed to be most prevalent on that platform. If everything looks ok I'll create the 3.2.7 tarball sometime Friday. I've tested trunk (rev 374588): FreeBSD 4.9, Apache 2.0.55 (prefork), Python 2.4.2 FreeBSD 4.9, Apache 2.0.50 (prefork), Python 2.3.4 All tests passed successfully.
Re: 3.2.6 or not
Jim Gallacher writes: Graham Dumpleton wrote: To confirm Jim's arithmetic anyway, I say -1 on 3.2.6 as it stands. As to 3.2.7, I say +1, subject to removal of problematic test case as already raised and with us at least confirming tests run OK for version out of SVN prior to packaging. Oh, so *that's* where you said we should disable the test. ;) I've disabled the publisher_cache test and the connection handler fix has been checked in. We are good to go for 3.2.7. As Graham suggests perhaps we could get some people to confirm the current version in SVN trunk prior to packaging. A couple of thumbs up from some FreeBSD'ers would be especially nice since the connection handler problem seemed to be most prevalent on that platform. If everything looks ok I'll create the 3.2.7 tarball sometime Friday. Just checked out the trunk... +1 trunk + Apache/2.0.55 + Python/2.3.5 + Debian/testing(etch)
Re: 3.2.6 or not
+1 trunk rev 374588 + Apache/2.0.55 + Python/2.2.3 + Windows 2000 SP4 +1 trunk rev 374588 + Apache/2.0.55 + ActivePython/2.3.5 + Windows 2000 SP4 +1 trunk rev 374588 + Apache/2.0.55 + ActivePython/2.4.2 + Windows XP SP2 All three installers for win32 are available at http://nicolas.lehuen.com/download/mod_python Regards, Nicolas 2006/2/3, Daniel J. Popowich [EMAIL PROTECTED]: Jim Gallacher writes: Graham Dumpleton wrote: To confirm Jim's arithmetic anyway, I say -1 on 3.2.6 as it stands. As to 3.2.7, I say +1, subject to removal of problematic test case as already raised and with us at least confirming tests run OK for version out of SVN prior to packaging. Oh, so *that's* where you said we should disable the test. ;) I've disabled the publisher_cache test and the connection handler fix has been checked in. We are good to go for 3.2.7. As Graham suggests perhaps we could get some people to confirm the current version in SVN trunk prior to packaging. A couple of thumbs up from some FreeBSD'ers would be especially nice since the connection handler problem seemed to be most prevalent on that platform. If everything looks ok I'll create the 3.2.7 tarball sometime Friday. Just checked out the trunk... +1 trunk + Apache/2.0.55 + Python/2.3.5 + Debian/testing(etch)
Re: 3.2.6 or not
Jim Gallacher wrote: +1 trunk rev 374588 Debian (sid), Apache 2.0.55-prefork, Python 2.3.5 +1 trunk rev 374588 Debian (sarge), Apache 2.0.54-worker, Python 2.3.5 +1 trunk rev 374588 Debian (sarge), Apache 2.0.54-prefork, Python 2.3.5 If I can get just one more test from FreeBSD 5 or 6, I'll produce the 3.2.7 tarball. +1 trunk rev 374709 FreeBSD 6.0 Apache 2.0.55-prefork, Python 2.4.2 This is a machine that always had trouble with that connectionhandler test before. Ran the entire unittest 5 times in a row with no trouble. Barry
Re: 3.2.6 or not
My official vote is eventually -1 for 3.2.6, see the previous discussion for why I've changed my mind. However I'm +1 on releasing 3.2.7 without a restrained testing period, not a long one like for 3.2.6. Regards, Nicolas 2006/2/2, Jim Gallacher [EMAIL PROTECTED]: I know you said no discussion Grisha, but can I have 2 ballots? ;) -1 If Graham thinks his conn handler fix is good, let's do 3.2.7 today. +1 If Graham is not sure, we release 3.2.6 now as is, and do a 3.2.7 bugfix in the next 4 to 6 weeks after digging into _conn_read issue further. So, I guess that makes my official vote a +0. Over to you Graham. No pressure though. :) Jim (Dang, it makes me feel dirty to waffle on my first offical vote that way). Gregory (Grisha) Trubetskoy wrote: OK, I know we've had some votes on this before, but I'd like to put this in a separate thread where it's not intermixed with all kinds of other things. This is a vote for the core group. We can release the 3.2.6 tarball as is or fix the connection handler bugs (there are two of them - the buffer pointer and eagain condition Graham tracked down) and release a 3.2.7 (or 3.2.6.1). The rationale for disregarding those known issues is that the connection handler is hardly used by anyone. The rationale for NOT disregarding is that we claim this to be a stable release, and given our slow release cycle, I imagine 3.2.6 will be around for a while. Anyhow - *the core group* (you know who you are), if you think 3.2.6 should be released as is, send in your +1. Let's keep this thread strictly a vote, without it turning into a discussion (we can discuss things in other threads). My official vote is +0. (To see what this means read http://httpd.apache.org/dev/guidelines.html) Grisha
Re: 3.2.6 test period - how long do we wait?
+1 Release what's fixed already, and then keep going afterwards. But can somebody address MODPYTHON-53 please. (updating modpython.org website). -- Deron Meranda
Re: 3.2.6 test period - how long do we wait?
I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? If that is the case I wonder if we should roll in the changes to support apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The original macro is: #define APR_STATUS_IS_SUCCESS(s) ((s) == APR_SUCCESS \ || (s) == APR_OS_START_SYSERR + NO_ERROR) The discussion on httpd-dev suggested that this macro should be substituted with a simple test such as if (rc != APR_SUCCESS), and the '||' condition was not likely used. So that we are making the fewest possible changes to our current 3.2 codebase, I'd suggest reimplenting APR_STATUS_IS_SUCCESS in our code, and then removing it 3.3. This will give us lot's of time as we work on 3.3 to discover if there are any problems droping the APR_OS_START_SYSERR part of the test. Jim
Re: 3.2.6 test period - how long do we wait?
OK, so shall we schedule the 3.2.x release for 2007, then ? As for the Apache 2.2 version, what if we roll in your suggested patch, Jim, then discover a bunch of problem related to it during the beta tests ? Will we wait until they are all fixed to release the 3.2 version ? Apache 2.2 is quite new so we'll likely to have to squish bugs, due for example to new interaction between Apache filters and mod_python. That's a wild guess but filters have been modified in Apache 2.2 so I'm sure something evil lurks there. bitterOr we could simply forget about making the release one day and tell every user to use the latest snapshot from subversion. Sorry to be like that, but we have users out there that would be perfectly happy with the current state of the 3.2.6 version, and a lot of our answers on the mailing list are yup, we know this bug, it's already been fixed one year ago, but don't worry, you'll get the bugfix soon enough./bitter Once again, it seems that no regression have been introduced in 3.2.6 vs 3.1.4, so we should release it ASAP and try to keep a steady release rythm afterwards. When we'll get momentum we'll solve a bunch of problem pretty fast, but it's been a year now that we are paralysed by perfectionism. What could be worse than leaving our users out there with the current 3.1.4 version ? Regards, Nicolas 2006/1/31, Jim Gallacher [EMAIL PROTECTED]: I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? If that is the case I wonder if we should roll in the changes to support apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The original macro is: #define APR_STATUS_IS_SUCCESS(s) ((s) == APR_SUCCESS \ || (s) == APR_OS_START_SYSERR + NO_ERROR) The discussion on httpd-dev suggested that this macro should be substituted with a simple test such as if (rc != APR_SUCCESS), and the '||' condition was not likely used. So that we are making the fewest possible changes to our current 3.2 codebase, I'd suggest reimplenting APR_STATUS_IS_SUCCESS in our code, and then removing it 3.3. This will give us lot's of time as we work on 3.3 to discover if there are any problems droping the APR_OS_START_SYSERR part of the test. Jim
Re: 3.2.6 test period - how long do we wait?
On Tue, 2006-01-31 at 17:59 +0100, Nicolas Lehuen wrote: Once again, it seems that no regression have been introduced in 3.2.6 vs 3.1.4, so we should release it ASAP and try to keep a steady release rythm afterwards. When we'll get momentum we'll solve a bunch of problem pretty fast, but it's been a year now that we are paralysed by perfectionism. What could be worse than leaving our users out there with the current 3.1.4 version ? +1 :) Rob -- Dr Robert Sanderson Dept of Computer Science, University of Liverpool Home: http://www.csc.liv.ac.uk/~azaroth/ Cheshire: http://www.cheshire3.org/
Re: 3.2.6 test period - how long do we wait?
On 1/31/06, Jim Gallacher [EMAIL PROTECTED] wrote: apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The apr_sockaddr_port_get() call was introduced by me to support IPv6 in modpython with MODPYTHON-64. Looking at the APR 1.0 CHANGES file (accompaning httpd 2.2): *) The following deprecated interfaces have been removed: apr_sockaddr_port_get- (access directly) I'm trying to review the APR source to see exactly what that means and the conditions under which it's safe. We don't want to accidentally pull the wrong field and have endianness problems. BTW, I want to get this one fixed before release rather than reverting to the pre-MODPYTHON-64 code...or we'll break under IPv6 again. -- Deron Meranda
Re: 3.2.6 test period - how long do we wait?
On 1/31/06, Deron Meranda [EMAIL PROTECTED] wrote: On 1/31/06, Jim Gallacher [EMAIL PROTECTED] wrote: apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The apr_sockaddr_port_get() call was introduced by me to support IPv6 in modpython with MODPYTHON-64. Okay, the correct fix for apr_sockaddr_port_get() which works under both Apache 2.0 and 2.2 is to replace the single call in connobject.c, apr_sockaddr_port_get(port, addr); with port = addr-port; I've added this information to MODPYTHON-78 notes (httpd 2.2 port). Of course that fix only *needs* to be made when 2.2 is completely addressed, but it can be safely rolled in before then too. -- Deron Meranda
Re: 3.2.6 test period - how long do we wait?
On Tue, 31 Jan 2006, Jim Gallacher wrote: I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? Unfortunately, the answer is yes... As much as we'd like to release it sooner, I think it's important to not loose the perspective that quality over speed is what we favor. In other words - if it takes a little longer to get the quality we want, then that's just fine. :-) Grisha
Re: 3.2.6 test period - how long do we wait?
Nicolas Lehuen wrote: OK, so shall we schedule the 3.2.x release for 2007, then ? As for the Apache 2.2 version, what if we roll in your suggested patch, Jim, then discover a bunch of problem related to it during the beta tests ? Will we wait until they are all fixed to release the 3.2 version ? Apache 2.2 is quite new so we'll likely to have to squish bugs, due for example to new interaction between Apache filters and mod_python. That's a wild guess but filters have been modified in Apache 2.2 so I'm sure something evil lurks there. I'm totally happy to defer changes for Apache 2.2 as I don't personally plan on migrating any time soon. The only reason I brought it up is that some of the code which seems to be causing issues for Apache 2.2 happens to reside in connobject.c, which is an area of interest for the _conn_read issue. Even if we were to roll fixes for apache 2.2 into mp 3.2, I'm not suggesting we advertise apache 2.2 compatibility just yet. bitterOr we could simply forget about making the release one day and tell every user to use the latest snapshot from subversion. Sorry to be like that, but we have users out there that would be perfectly happy with the current state of the 3.2.6 version, and a lot of our answers on the mailing list are yup, we know this bug, it's already been fixed one year ago, but don't worry, you'll get the bugfix soon enough./bitter Point taken, and it is a very good point indeed. Once again, it seems that no regression have been introduced in 3.2.6 vs 3.1.4, so we should release it ASAP and try to keep a steady release rythm afterwards. When we'll get momentum we'll solve a bunch of problem pretty fast, but it's been a year now that we are paralysed by perfectionism. What could be worse than leaving our users out there with the current 3.1.4 version ? I *still* wonder why the whole ConnectionHandler issue was not seen on FreeBSD with mp 3.1.4 though. I'll need to check the svn logs again but I'm pretty sure this is not a new unit test. That being said I'm not personally concerned about this issue since it doesn't affect every platform (or more selfishly I should say it doesn't affect the OS I'm working on) and I suspect that not many people are directly interacting with the connection object. What's that old line about open source software... Release early, release often? Jim 2006/1/31, Jim Gallacher [EMAIL PROTECTED]: I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? If that is the case I wonder if we should roll in the changes to support apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The original macro is: #define APR_STATUS_IS_SUCCESS(s) ((s) == APR_SUCCESS \ || (s) == APR_OS_START_SYSERR + NO_ERROR) The discussion on httpd-dev suggested that this macro should be substituted with a simple test such as if (rc != APR_SUCCESS), and the '||' condition was not likely used. So that we are making the fewest possible changes to our current 3.2 codebase, I'd suggest reimplenting APR_STATUS_IS_SUCCESS in our code, and then removing it 3.3. This will give us lot's of time as we work on 3.3 to discover if there are any problems droping the APR_OS_START_SYSERR part of the test. Jim
Re: 3.2.6 test period - how long do we wait?
Deron Meranda wrote: On 1/31/06, Jim Gallacher [EMAIL PROTECTED] wrote: apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The apr_sockaddr_port_get() call was introduced by me to support IPv6 in modpython with MODPYTHON-64. Looking at the APR 1.0 CHANGES file (accompaning httpd 2.2): *) The following deprecated interfaces have been removed: apr_sockaddr_port_get- (access directly) I'm trying to review the APR source to see exactly what that means and the conditions under which it's safe. We don't want to accidentally pull the wrong field and have endianness problems. BTW, I want to get this one fixed before release rather than reverting to the pre-MODPYTHON-64 code...or we'll break under IPv6 again. I say we leave it as is (ie continue use apr_sockaddr_port_get) and worry about it when we do some sort of comprehensive review for apache 2.2 compatibility. Jim
Re: 3.2.6 test period - how long do we wait?
I am having problems with posts to python-dev mailing list from home occassionally disappearing in a black hole. Thus my post on this topic before Jim brought it up in the first place vanished. What I has said was: this code runs smoothly, i.e. no segfaults, all tests passed: FreeBSD 4.9: Apache/2.0.50 (prefork) Python/2.3.4 Apache/2.0.55 (prefork) Python/2.4.2 Okay, this is good. If there is a general consensus that this is a reasonable solution, the question is what should be done with 3.2.6. Should it be released with this problem in it and fix it later in 3.3? Personally, I really don't think that connection handlers in mod_python get used by anyone and so don't see a pressing need to go fixing it right now. Thus I would say go with 3.2.6 as final with how it is. Is there anything else of concern that is holding us up now on an official release of 3.2.6? In other words, I agree with Nicolas, we should just release it. If someone actually came up and said they were using connection handlers for something significant, then I might change this view, but I doubt anyone is using them. I would also point out that JIRA lists lots of other issues, some of them minor and trivial to fix but which would be more worthwhile than fixing than this connection handler issue. We have already said we will defer these in the interest of getting a final version of 3.2 out sooner rather than later. Thus, as much as I would prefer to see as much as possible fixed, I think we just have defer further changes to next version. Graham Jim Gallacher wrote .. Nicolas Lehuen wrote: OK, so shall we schedule the 3.2.x release for 2007, then ? As for the Apache 2.2 version, what if we roll in your suggested patch, Jim, then discover a bunch of problem related to it during the beta tests ? Will we wait until they are all fixed to release the 3.2 version ? Apache 2.2 is quite new so we'll likely to have to squish bugs, due for example to new interaction between Apache filters and mod_python. That's a wild guess but filters have been modified in Apache 2.2 so I'm sure something evil lurks there. I'm totally happy to defer changes for Apache 2.2 as I don't personally plan on migrating any time soon. The only reason I brought it up is that some of the code which seems to be causing issues for Apache 2.2 happens to reside in connobject.c, which is an area of interest for the _conn_read issue. Even if we were to roll fixes for apache 2.2 into mp 3.2, I'm not suggesting we advertise apache 2.2 compatibility just yet. bitterOr we could simply forget about making the release one day and tell every user to use the latest snapshot from subversion. Sorry to be like that, but we have users out there that would be perfectly happy with the current state of the 3.2.6 version, and a lot of our answers on the mailing list are yup, we know this bug, it's already been fixed one year ago, but don't worry, you'll get the bugfix soon enough./bitter Point taken, and it is a very good point indeed. Once again, it seems that no regression have been introduced in 3.2.6 vs 3.1.4, so we should release it ASAP and try to keep a steady release rythm afterwards. When we'll get momentum we'll solve a bunch of problem pretty fast, but it's been a year now that we are paralysed by perfectionism. What could be worse than leaving our users out there with the current 3.1.4 version ? I *still* wonder why the whole ConnectionHandler issue was not seen on FreeBSD with mp 3.1.4 though. I'll need to check the svn logs again but I'm pretty sure this is not a new unit test. That being said I'm not personally concerned about this issue since it doesn't affect every platform (or more selfishly I should say it doesn't affect the OS I'm working on) and I suspect that not many people are directly interacting with the connection object. What's that old line about open source software... Release early, release often? Jim 2006/1/31, Jim Gallacher [EMAIL PROTECTED]: I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? If that is the case I wonder if we should roll in the changes to support apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The original macro is: #define APR_STATUS_IS_SUCCESS(s) ((s) == APR_SUCCESS \ || (s) == APR_OS_START_SYSERR + NO_ERROR) The discussion on httpd-dev suggested that this macro should be substituted with a simple test such as if (rc != APR_SUCCESS), and the '||' condition was not likely used. So that we are making the fewest possible changes to our current 3.2 codebase, I'd suggest reimplenting APR_STATUS_IS_SUCCESS in our code, and then removing it 3.3. This will give us lot's of time as we work on 3.3 to
Re: 3.2.6 test period - how long do we wait?
Good enough for me. Shall we vote? If so I am: +1 Jim Graham Dumpleton wrote: I am having problems with posts to python-dev mailing list from home occassionally disappearing in a black hole. Thus my post on this topic before Jim brought it up in the first place vanished. What I has said was: this code runs smoothly, i.e. no segfaults, all tests passed: FreeBSD 4.9: Apache/2.0.50 (prefork) Python/2.3.4 Apache/2.0.55 (prefork) Python/2.4.2 Okay, this is good. If there is a general consensus that this is a reasonable solution, the question is what should be done with 3.2.6. Should it be released with this problem in it and fix it later in 3.3? Personally, I really don't think that connection handlers in mod_python get used by anyone and so don't see a pressing need to go fixing it right now. Thus I would say go with 3.2.6 as final with how it is. Is there anything else of concern that is holding us up now on an official release of 3.2.6? In other words, I agree with Nicolas, we should just release it. If someone actually came up and said they were using connection handlers for something significant, then I might change this view, but I doubt anyone is using them. I would also point out that JIRA lists lots of other issues, some of them minor and trivial to fix but which would be more worthwhile than fixing than this connection handler issue. We have already said we will defer these in the interest of getting a final version of 3.2 out sooner rather than later. Thus, as much as I would prefer to see as much as possible fixed, I think we just have defer further changes to next version. Graham Jim Gallacher wrote .. Nicolas Lehuen wrote: OK, so shall we schedule the 3.2.x release for 2007, then ? As for the Apache 2.2 version, what if we roll in your suggested patch, Jim, then discover a bunch of problem related to it during the beta tests ? Will we wait until they are all fixed to release the 3.2 version ? Apache 2.2 is quite new so we'll likely to have to squish bugs, due for example to new interaction between Apache filters and mod_python. That's a wild guess but filters have been modified in Apache 2.2 so I'm sure something evil lurks there. I'm totally happy to defer changes for Apache 2.2 as I don't personally plan on migrating any time soon. The only reason I brought it up is that some of the code which seems to be causing issues for Apache 2.2 happens to reside in connobject.c, which is an area of interest for the _conn_read issue. Even if we were to roll fixes for apache 2.2 into mp 3.2, I'm not suggesting we advertise apache 2.2 compatibility just yet. bitterOr we could simply forget about making the release one day and tell every user to use the latest snapshot from subversion. Sorry to be like that, but we have users out there that would be perfectly happy with the current state of the 3.2.6 version, and a lot of our answers on the mailing list are yup, we know this bug, it's already been fixed one year ago, but don't worry, you'll get the bugfix soon enough./bitter Point taken, and it is a very good point indeed. Once again, it seems that no regression have been introduced in 3.2.6 vs 3.1.4, so we should release it ASAP and try to keep a steady release rythm afterwards. When we'll get momentum we'll solve a bunch of problem pretty fast, but it's been a year now that we are paralysed by perfectionism. What could be worse than leaving our users out there with the current 3.1.4 version ? I *still* wonder why the whole ConnectionHandler issue was not seen on FreeBSD with mp 3.1.4 though. I'll need to check the svn logs again but I'm pretty sure this is not a new unit test. That being said I'm not personally concerned about this issue since it doesn't affect every platform (or more selfishly I should say it doesn't affect the OS I'm working on) and I suspect that not many people are directly interacting with the connection object. What's that old line about open source software... Release early, release often? Jim 2006/1/31, Jim Gallacher [EMAIL PROTECTED]: I assume we will be doing a 3.2.7 release if Graham's fix for the ConnectionHandler / MODPYTHON-102 problem works? If that is the case I wonder if we should roll in the changes to support apache 2.2. I scanned mod_python for deprecated or removed apr calls and can find only one (apr_sockaddr_port_get), plus the missing APR_STATUS_IS_SUCCESS macro. The original macro is: #define APR_STATUS_IS_SUCCESS(s) ((s) == APR_SUCCESS \ || (s) == APR_OS_START_SYSERR + NO_ERROR) The discussion on httpd-dev suggested that this macro should be substituted with a simple test such as if (rc != APR_SUCCESS), and the '||' condition was not likely used. So that we are making the fewest possible changes to our current 3.2 codebase, I'd suggest reimplenting APR_STATUS_IS_SUCCESS in our code, and then removing it 3.3. This will give us lot's of time as we work on 3.3 to
Re: Segfaults in ConnectionHander FreeBSD (was Re: 3.2.6 test period - how long do we wait?)
Jim Gallacher wrote: Barry Pederson wrote: I think this is the general kind of thing we're looking for though, with some mistaken pointer/memory operation. Too bad we can't write *everything* in python. :( You haven't been following PyPy then? :-) David
Re: 3.2.6 test period - how long do we wait?
Gregory (Grisha) Trubetskoy wrote: On Sun, 29 Jan 2006, Graham Dumpleton wrote: buffer += bufsize; On a second thought - yes, you're right :-) And if he's not then there is a bug in filter_read since that is what it does and it is very similar to _conn_read. Jim
Re: 3.2.6 test period - how long do we wait?
Graham Dumpleton wrote:
On 29/01/2006, at 1:29 AM, Jim Gallacher wrote:
Volodya wrote:
On Fri, Jan 27, 2006 at 10:28:24AM -0500, Gregory (Grisha)
Trubetskoy wrote:
OK, and I see Ron sent a Solaris 10 +1, which is good. I think we
need a FreeBSD +1 - perhaps not necessarily 6.0, but something...
FreeBSD 4.9
In my case PythonConnectionHandler test fails.
Tested on:
Apache/2.0.55 (prefork) mod_python/3.2.6 Python/2.4.2
Apache/2.0.50 (prefork) mod_python/3.2.6 Python/2.3.4
I think this is the same as Barry Pederson's failure.
Which are both similar to:
http://issues.apache.org/jira/browse/MODPYTHON-102
Although in the Mac OS X case, only happened when on secondary port and
not
main listener port.
I have never seen anyone use connection handlers so speculated that
there was
probably no urgency in doing anything about it.
Graham
I don't know if this is the answer to the problem, but it looks like a
bug anyway. In connobject.c starting at line 133:
/* time to grow destination string? */
if (len == 0 bytes_read == bufsize) {
_PyString_Resize(result, bufsize + HUGE_STRING_LEN);
buffer = PyString_AS_STRING((PyStringObject *) result);
buffer += HUGE_STRING_LEN;
bufsize += HUGE_STRING_LEN;
}
It looks like we've just set the buffer pointer to an address somewhere
inside the buffer. That can't be good. The buffer pointer should be set
to the bytes_read position. Perhaps one of you FreeBSD heads could try
the attached patch.
Jim
Index: src/connobject.c
===
--- src/connobject.c(revision 369511)
+++ src/connobject.c(working copy)
@@ -135,7 +135,7 @@
_PyString_Resize(result, bufsize + HUGE_STRING_LEN);
buffer = PyString_AS_STRING((PyStringObject *) result);
-buffer += HUGE_STRING_LEN;
+buffer += bytes_read;
bufsize += HUGE_STRING_LEN;
}
Re: 3.2.6 test period - how long do we wait?
Barry Pederson wrote .. As I mentioned in another message, I did some experimenting with disabling other unittests and found if you disable just test_fileupload, all the remaining tests including test_connectionhandler pass. If you disable everything except test_fileupload and test_connectionhandler, then test_connectionhandler still crashes. So I suspect that it's code involved with running test_fileupload (Testing 1 MB file upload support) that's really the source of the problem, and it's screwing up some part of memory thats only tripped over later later during the connectionhandler test. FWIW, when I came across this on Mac OS X, it wasn't when running the unit tests. In fact the unit tests work fine. It was happening for the very first request after an Apache stop/start. Thus it couldn't have been the result of a specific prior history of requests. Anyway, I am going to give Jim's suggested change a go in a little while and have a bit of a look at the code again to see if I can spot anything obvious. Since though it was on the very first read and no buffer resize was involved, suspecting that it will not help. Graham
Re: Segfaults in ConnectionHander FreeBSD (was Re: 3.2.6 test period - how long do we wait?)
Jim Gallacher wrote: Dang, it's frustrating not being able to reproduce this bug in Linux. I suppose it's maybe something to do with different malloc implementations or such. I haven't seen any +1s for OpenBSD, which would be interesting to see since they added some stuff in 3.8 to help catch problems with this sort of thing http://kerneltrap.org/node/5584 Anyone been able to use valgrind or similar with mod_python? I Googled and found a couple old messages from '02 and '04 mentioning attempts to use this, but doesn't sound like much came out of it. I think there's a valgrind port on FreeBSD, so I may give that a try. Barry
Re: 3.2.6 test period - how long do we wait?
Grisha wrote ..
On Sun, 29 Jan 2006, Jim Gallacher wrote:
I don't know if this is the answer to the problem, but it looks like
a bug
anyway. In connobject.c starting at line 133:
/* time to grow destination string? */
if (len == 0 bytes_read == bufsize) {
_PyString_Resize(result, bufsize + HUGE_STRING_LEN);
buffer = PyString_AS_STRING((PyStringObject *) result);
buffer += HUGE_STRING_LEN;
bufsize += HUGE_STRING_LEN;
}
It looks like we've just set the buffer pointer to an address somewhere
inside the buffer. That can't be good. The buffer pointer should be set
to
the bytes_read position.
...or bufsize. Of course they are the same, but I think this would read
cleaner:
if (len == 0 bytes_read == bufsize) {
_PyString_Resize(result, bufsize + HUGE_STRING_LEN);
buffer = PyString_AS_STRING((PyStringObject *) result);
buffer = bufsize;
bufsize += HUGE_STRING_LEN;
}
This bug would garble the data if it's at least twice HUGE_STRING_LEN,
since the first time around the code would work OK because bufsize would
equal HUGE_STRING_LEN.
I suspect you mean't:
if (len == 0 bytes_read == bufsize) {
_PyString_Resize(result, bufsize + HUGE_STRING_LEN);
buffer = PyString_AS_STRING((PyStringObject *) result);
buffer += bufsize;
bufsize += HUGE_STRING_LEN;
}
Anyway, this change doesn't help with Mac OS X as it doesn't even get invoked.
Unlike suggestions by someone else that self seemed to be getting corrupted,
it looks fine to me, and code simply crashed down in:
apr_bucket_read(b, data, size, APR_BLOCK_READ)
on very first call to it. Thus need to start tracking into Apache itself and
see what
there may be about bucket structures that isn't correct. This is where I got to
last time before I gave up, feeling it wasn't worth the effort at the time.
I'll try
and build a version of Apache with debug so I can get a better stack trace.
Graham
Re: 3.2.6 test period - how long do we wait?
On Sun, 29 Jan 2006, Graham Dumpleton wrote: Grisha wrote .. buffer = bufsize; I suspect you mean't: buffer += bufsize; buffer = bufsize should be correct because you move the pointer to the end of where the bufer was. buffer += bufsize would set it further than you need it. Anyway, this change doesn't help with Mac OS X as it doesn't even get invoked. Unlike suggestions by someone else that self seemed to be getting corrupted, it looks fine to me, and code simply crashed down in: apr_bucket_read(b, data, size, APR_BLOCK_READ) on very first call to it. Thus need to start tracking into Apache itself and see what there may be about bucket structures that isn't correct. This is where I got to last time before I gave up, feeling it wasn't worth the effort at the time. I'll try and build a version of Apache with debug so I can get a better stack trace. The first thing I'd check is for validity of b. Buckets use reference counting much like Python, so sometimes it's possible for a bucket to self-distruct. Grisha
Re: 3.2.6 test period - how long do we wait?
On Sun, 29 Jan 2006, Graham Dumpleton wrote: buffer += bufsize; On a second thought - yes, you're right :-) Grisha
Re: 3.2.6 test period - how long do we wait?
On Fri, Jan 27, 2006 at 10:28:24AM -0500, Gregory (Grisha) Trubetskoy wrote: OK, and I see Ron sent a Solaris 10 +1, which is good. I think we need a FreeBSD +1 - perhaps not necessarily 6.0, but something... FreeBSD 4.9 In my case PythonConnectionHandler test fails. Tested on: Apache/2.0.55 (prefork) mod_python/3.2.6 Python/2.4.2 Apache/2.0.50 (prefork) mod_python/3.2.6 Python/2.3.4 I've tried to reproduce this test manually: $ /usr/local/sbin/httpd -k start -f /home/volodya/mod_python-3.2.6/test/conf/test.conf $ GET http://localhost:2758/tests.py 500 Server closed connection without sending any data back $ telnet localhost 2758 telnet localhost 2758 Trying 127.0.0.1... Connected to localhost.mydomain.tld. Escape character is '^]'. Connection closed by foreign host. Child sigfaults immediately after connection being established. logs/error_log: [notice] mod_python: Creating 8 session mutexes based on 256 max processes and 0 max threads. [notice] Apache/2.0.50 (FreeBSD) mod_python/3.2.6 Python/2.3.4 configured -- resuming normal operations [info] Server built: Jul 15 2004 11:25:58 [debug] prefork.c(955): AcceptMutex: flock (default: flock) [notice] child pid 98975 exit signal Segmentation fault (11), possible coredump in /home/volodya/mod_python-3.2.6/test [notice] child pid 98977 exit signal Segmentation fault (11), possible coredump in /home/volodya/mod_python-3.2.6/test Yes, i know that FreeBSD 4.9 is rather old, but this is the only version i have at my disposal.
Re: 3.2.6 test period - how long do we wait?
Volodya wrote: On Fri, Jan 27, 2006 at 10:28:24AM -0500, Gregory (Grisha) Trubetskoy wrote: OK, and I see Ron sent a Solaris 10 +1, which is good. I think we need a FreeBSD +1 - perhaps not necessarily 6.0, but something... FreeBSD 4.9 In my case PythonConnectionHandler test fails. Tested on: Apache/2.0.55 (prefork) mod_python/3.2.6 Python/2.4.2 Apache/2.0.50 (prefork) mod_python/3.2.6 Python/2.3.4 I think this is the same as Barry Pederson's failure. Jim
Re: 3.2.6 test period - how long do we wait?
Maybe I forgot to send this one in: +1 Solaris 10 Sparc Apache-2.0.55 mpm-prefork Python-2.4.2 cheers, Ron Ron Reisor [EMAIL PROTECTED] (RWR3) University of Delaware Information Technologies/Network and Systems Services Computing Center/192 South Chapel Street/Newark DE, 19716 pgp finger print: 0D 73 06 6F D3 6A 99 D3 F5 D5 6E FF 3B B9 7C 2C
Re: 3.2.6 test period - how long do we wait?
Jim Gallacher wrote .. It seems like any 3.2.6 testing that is going to be done, has been done. How long do we wait before making a decision for an official release. If we don't get cracking on 3.3 soon Graham's gonna fill another couple of pages on JIRA and we'll never catch up. :) You aren't far wrong. I was actually thinking of sending an email to the list warning about an impending avalanche of JIRA bug reports. This is going to be caused by me going through my module importer issues list and logging separate bug reports for each problem where none already exists in JIRA. Also have potentially others as well I know of but aren't in my module importer issues list at the moment but which I will be adding. Also going to create new bugs or ask for some old bugs to be reopened because the issue was only addressed in version 3.2.6 for mod_python.publisher and not mod_python as a whole. Thus, consider yourself warned. Note that none of what I will be creating issues for should stop 3.2.6 (final) being released. Already accepted that they would be 3.3. Graham
Re: 3.2.6 test period - how long do we wait?
On Thu, 26 Jan 2006, Jim Gallacher wrote: It seems like any 3.2.6 testing that is going to be done, has been done. I've been kinda swamped with unrelated things past two weeks, so I wasn't paying much attention. Perhaps an e-mail summarizing the +1's so far and a quick vote of the core group on whether this is enough? Grisha
Re: 3.2.6 test period - how long do we wait?
Gregory (Grisha) Trubetskoy wrote: On Thu, 26 Jan 2006, Jim Gallacher wrote: It seems like any 3.2.6 testing that is going to be done, has been done. I've been kinda swamped with unrelated things past two weeks, so I wasn't paying much attention. Perhaps an e-mail summarizing the +1's so far and a quick vote of the core group on whether this is enough? I will gather the info later tonight. Jim
Re: 3.2.6 test period - how long do we wait?
Gregory (Grisha) Trubetskoy wrote: On Thu, 26 Jan 2006, Jim Gallacher wrote: It seems like any 3.2.6 testing that is going to be done, has been done. I've been kinda swamped with unrelated things past two weeks, so I wasn't paying much attention. Perhaps an e-mail summarizing the +1's so far and a quick vote of the core group on whether this is enough? Grisha Here is the summary of the 3.2.6 testing to date. Let me know if I've missed anyone's tests or you still want to toss you test on the pile. For the 3.2.6b tarball (same as 3.2.6 but with the messed up version string). +1 Gentoo [current], Apache-2.0.55 (mpm-prefork), Python-2.4, gcc-3.4.4 +1 Linux (amd64) Ubuntu Breezy 5.10, apache 2.0.54 mpm-worker python 2.4 +1 Linux Debian sid, apache 2.0.55 (mpm-prefork), python 2.3 +1 Linux Debian sarge, apache 2.0.54 (mpm-worker), python 2.3 +1 Mac OS X 10.3, Apache 2.0.55, python 2.3 +1 Slackware 10.1, Apache 2.0.55 (mpm-prefork), Python 2.4 +1 Windows XP, python 2.4 +1 Windows 2000, python 2.3 For 3.2.6 +1 Fedora Core 4 Apache 2.0.54 Python 2.4.1 +1 Linux Debian sid, apache 2.0.55 (mpm-prefork), python 2.3 +1 Linux Debian sarge, apache 2.0.54 (mpm-worker), python 2.3 +1 MacOS-10.4.4 Apache-2.0.55 mpm-prefork Python-2.4.2 +1 Windows 2000 Server SP4, Python 2.3 +1 Windows XP Pro SP2, Python 2.4 +1 Windows XP Pro SP2 Apache 2.0.54 Python 2.4.2 0 HP Tru64, mpm-worker On the bsd front, Barry Pederson reported a problem with FreeBSD 6.0 but didn't issue an actual -1. FreeBSD and I are still not getting along, but with a small manual tweak of the Makefile I was able to run all the tests successfully. I don't know if there is a configure problem or if it's just my BSD incompetence but I'm willing to issue a shaky +0 for FreeBSD 6.0. Jim
