I noticed that lines 789-793 (in the original file, $Id: requestobject.c
420297) should be moved up to line 780,
765 if (self->rbuff_pos < self->rbuff_len) {
766
767 /* if yes, process that first */
768 while (self->rbuff_pos < self->rbuff_len) {
769
770 buffer[copied++] = self->rbuff[self->rbuff_pos];
771 if ((self->rbuff[self->rbuff_pos++] == '\n') ||
772 (copied == len)) {
773
774 /* our work is done */
775
776 /* resize if necessary */
777 if (copied < len)
778 if(_PyString_Resize(&result, copied))
779 return NULL;
780
781 return result;
782 }
783 }
784 }
So it should look like this:
if (self->rbuff_pos < self->rbuff_len) {
/* if yes, process that first */
while (self->rbuff_pos < self->rbuff_len) {
buffer[copied++] = self->rbuff[self->rbuff_pos];
if ((self->rbuff[self->rbuff_pos++] == '\n') ||
(copied == len)) {
/* our work is done */
/* resize if necessary */
if (copied < len)
if(_PyString_Resize(&result, copied))
return NULL;
if (self->rbuff_pos >= self->rbuff_len && self->rbuff !=
NULL) { // dispose of buffer since we no longer need it
free(self->rbuff);
self->rbuff = NULL;
}
return result;
}
}
}
The reason is that we are returning results, line 781 (original file),
as the last time that we will ever read from buffer without deallocating
the buffer memory.
Even with this fix, which I'm not 100% sure that is a fix, memory still
leaking but in a lower rate.
Any help will be greatly appreciated.
/amn
Alexis Marrero wrote:
Experimenting on this issue, I noticed that neither of the following
set of "ifs" are ever met:
786 /* Free old rbuff as the old contents have been copied
over and
787 we are about to allocate a new rbuff. Perhaps this
could be reused
788 somehow? */
789 if (self->rbuff_pos >= self->rbuff_len && self->rbuff !=
NULL)
790 {
791 free(self->rbuff);
792 self->rbuff = NULL;
793 }
--------
846 /* Free rbuff if we're done with it */
847 if (self->rbuff_pos >= self->rbuff_len && self->rbuff !=
NULL)
848 {
849 free(self->rbuff);
850 self->rbuff = NULL;
851 }
I noticed that by putting some statements to write to the output
stream. They never execute.
/amn
On Aug 10, 2006, at 1:43 PM, Alexis Marrero wrote:
All,
We are trying to nail down a memory leak that happens only when
documents are POSTed to the server.
For testing we have a short script that does:
while True:
dictionary_of_parameters = {'field1': 'a'*100000}
post('url...', dictionary_of_parameters)
Then we run "top" on the server and watch the server memory grow
without bound. Why do we know that the problem is in
request.readline()? If I go to
mod_python.util.FieldStorage.read_to_boundary() and add the following
statement:
def read_to_boundary(...):
return True
...
as the first executable line in the function the memory does not grow.
I have read the req_readline a 1000 time and I can't figure out where
the problem is.
My config:
Python 2.4.1
mod_python 3.2.10
Our request handler does nothing other than using
util.FieldStorage(req) and req.write('hello').
I have some suspicion that it has to do with:
....
19 * requestobject.c
20 *
21 * $Id: requestobject.c 420297 2006-07-09 13:53:06Z nlehuen $
22 *
23 */
....
846 /* Free rbuff if we're done with it */
847 if (self->rbuff_pos >= self->rbuff_len && self->rbuff !=
NULL)
848 {
849 free(self->rbuff);
850 self->rbuff = NULL;
851 }
852
Though, I can't confirm.
/amn
