Re: Segfaults in ConnectionHander FreeBSD (was Re: 3.2.6 test period - how long do we wait?)

2006-01-29 Thread Jim Gallacher

Barry Pederson wrote:
I don't know if this is the answer to the problem, but it looks like a 
bug anyway. In connobject.c starting at line 133:


/* time to grow destination string? */
if (len == 0 && bytes_read == bufsize) {

_PyString_Resize(&result, bufsize + HUGE_STRING_LEN);
buffer = PyString_AS_STRING((PyStringObject *) result);
buffer += HUGE_STRING_LEN;
bufsize += HUGE_STRING_LEN;
}


It looks like we've just set the buffer pointer to an address 
somewhere inside the buffer. That can't be good. The buffer pointer 
should be set to the bytes_read position. Perhaps one of you FreeBSD 
heads could try the attached patch.


Jim






Index: src/connobject.c
===
--- src/connobject.c(revision 369511)
+++ src/connobject.c(working copy)
@@ -135,7 +135,7 @@
 
 _PyString_Resize(&result, bufsize + HUGE_STRING_LEN);

 buffer = PyString_AS_STRING((PyStringObject *) result);
-buffer += HUGE_STRING_LEN;
+buffer += bytes_read;
 bufsize += HUGE_STRING_LEN;
 }
 



Sorry, that doesn't seem to fix it.  I did a fresh extraction of 
mod_python-3.2.6.tgz, applied the patch, did ./configure, make, su, make 
install, exit su, cd test, ran test.py - got the same result as before, 
with the same core dump apparently.


I really didn't think it would help since a buffer of HUGE_STRING_LEN 
(8192) should have been created in the first place. The unit test 
wouldn't be reading that many bytes, so I doubt the buffer is getting 
resized. All the same it still looks like a bug.


I think this is the general kind of thing we're looking for though, with 
some mistaken pointer/memory operation.


Too bad we can't write *everything* in python. :(


---

As I mentioned in another message, I did some experimenting with 
disabling other unittests and found if you disable just 
"test_fileupload", all the remaining tests including 
"test_connectionhandler" pass.


I hadn't forgotten. I'm just trying to understand what might be going on 
in the code and I spotted the bug.


If you disable everything except "test_fileupload" and 
"test_connectionhandler", then "test_connectionhandler" still crashes.


Dang, it's frustrating not being able to reproduce this bug in Linux.

So I suspect that it's code involved with running "test_fileupload" 
(Testing 1 MB file upload support) that's really the source of the 
problem, and it's screwing up some part of memory thats only tripped 
over later later during the connectionhandler test.


One of the things I'm trying to understand is what has changed since 3.1 
that is causing this bug. The only thing different in connobject.c is a 
fix to actually return local_ip and local_host (was returning remote_ip 
and remote_host), plus makeipaddr and makesockaddr now call the 
equivalent apr functions rather than the prior roll-our-own approach. 
Neither of these changes should impact the _conn_read.


Jim




Re: Segfaults in ConnectionHander FreeBSD (was Re: 3.2.6 test period - how long do we wait?)

2006-01-29 Thread Barry Pederson

Jim Gallacher wrote:


Dang, it's frustrating not being able to reproduce this bug in Linux.


I suppose it's maybe something to do with different malloc 
implementations or such.   I haven't seen any +1s for OpenBSD, which 
would be interesting to see since they added some stuff in 3.8 to help 
catch problems with this sort of thing


http://kerneltrap.org/node/5584

Anyone been able to use valgrind or similar with mod_python?  I Googled 
and found a couple old messages from '02 and '04 mentioning attempts to 
use this, but doesn't sound like much came out of it.  I think there's a 
valgrind port on FreeBSD, so I may give that a try.


Barry


Re: Segfaults in ConnectionHander FreeBSD (was Re: 3.2.6 test period - how long do we wait?)

2006-01-30 Thread David Fraser

Jim Gallacher wrote:

Barry Pederson wrote:
I think this is the general kind of thing we're looking for though, 
with some mistaken pointer/memory operation.

Too bad we can't write *everything* in python. :(

You haven't been following PyPy then? :-)

David


Re: Segfaults in ConnectionHander FreeBSD (was Re: 3.2.6 test period - how long do we wait?)

2006-01-30 Thread Jim Gallacher

David Fraser wrote:

Jim Gallacher wrote:


Barry Pederson wrote:

I think this is the general kind of thing we're looking for though, 
with some mistaken pointer/memory operation.


Too bad we can't write *everything* in python. :(


You haven't been following PyPy then? :-)

David



Well, sure, but I think porting will have to wait until at least 
mod_python 3.4. :)


Jim