Incremental memory use when performing streaming over long period.
------------------------------------------------------------------
Key: MODPYTHON-241
URL: https://issues.apache.org/jira/browse/MODPYTHON-241
Project: mod_python
Issue Type: Bug
Components: core
Affects Versions: 3.3.1, 3.2.10
Reporter: Graham Dumpleton
Priority: Minor
As first raised in:
http://www.modpython.org/pipermail/mod_python/2007-July/023964.html
when streaming data from mod_python handler over a long period of time, memory
use will keep increasing.
Note that this isn't actually a memory leak though, as the memory gets
reclaimed at the end of the request.
The underlying problem arises from mod_python using the ap_rwrite() function to
write output data. When this function is used it will create a bucket chain for
each block of data written, with the bucket chain coming from the request
memory pool. There is no attempt to reuse the bucket chain for successive
writes, so memory will increase on every call.
The normal solution to this problem is to retain the bucket chain and use it
from one call to the next. In mod_python though this isn't simple to achieve
because calls to response data may or may not be flushed at the discretion of
the user code. The retaining of the bucket chain and using it from one call to
the next generally can only be done when data is always being flushed. A more
complicated system would be needed which can cope with a need to buffer data at
times and yet flush at other times.
One approach may be to not actually use Apache itself to buffer data between
flushes, but to use Python data structures instead to accumulate references to
Python string objects in a Python list so objects are still valid. When flushed
only then construct a chain of buckets which use direct reference to raw data
of Python string objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
