[jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12447658 ] Jim Gallacher commented on MODPYTHON-172: - [[ Old comment, sent by email on Fri, 07 Jul 2006 21:26:01 -0400 ]] I've been working through the PyList_Append instances and noticed the cfgtree_walk and req_readlines issues as well. I've confirmed that cfgtree_walk does indeed leak. I haven't tested req_readlines but I can't see why it wouldn't leak. Can you spot the other bug in req_readlines? Hint: (size = size) will always be true. ;) Once that is fixed I think we need to alter the docs for req.readlines() as I'm not sure the description for sizehint accurately reflects the way it works. Jim Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55 -- Key: MODPYTHON-172 URL: http://issues.apache.org/jira/browse/MODPYTHON-172 Project: mod_python Issue Type: Bug Components: core Affects Versions: 3.2.8 Environment: Win32 XP SP1 / SP2 Apache 2.0.55 installed from binary (.MSI) Python 2.4.2 or 2.4.3installed from binary from www.python.org Reporter: Laurent Blanquet Fix For: 3.3, 3.2.10 I encounter memory leaks [~ 16 K per request) using the configuration described below. = Python configuration from Httpd.conf: = Alias /python/ d:/python24/B2B/ Directory d:/python24/B2B AddHandler mod_python .py PythonHandler pyHandlerHTTP PythonDebug On /Directory = Test handler - pyHandlerHTTP.py : = import mod_python from mod_python import util def handler(req): #Removing this line solves the problem. F=util.FieldStorage( req ) return mod_python.apache.OK = HTTP Request (dump using TCPWATCH): = POST http://localhost:80/python/Alertes.py HTTP/1.0 Content-Type: multipart/form-data; boundary=061006144341906 Content-Length: 209 Proxy-Connection: keep-alive Host: www.tx2-localhost Accept: text/html, */* User-Agent: Mozilla/3.0 (compatible; Indy Library) Proxy-Authorization: Basic Og== --061006144341906 Content-Disposition: form-data; name=TYPE LAST_ALERTS --061006144341906 Content-Disposition: form-data; name=FILEAGE 180 --061006144341906 -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-184) Memory leak apache.table()
[
http://issues.apache.org/jira/browse/MODPYTHON-184?page=comments#action_12447665
]
Alexis Marrero commented on MODPYTHON-184:
--
[[ Old comment, sent by email on Tue, 15 Aug 2006 15:54:50 -0400 ]]
Jim,
This are my results for the memory leak search in apache.table().
The table object creates a memory pool by using apr_pool_create_ex()
and destroys the pool using apr_pool_destroy(). I added a line in
MpTable_New() before return (PyObject*)t to destroy the pool and
ran 1M iterations and I notice that there was no memory leak.
Therefore the apache functions seems to be working fine.
I couldn't fix the problem but here is a work around. In mod_python/
util.py instead of using apache.make_table() use a regular Python
dictionary. So the line that looks like:
headers = apache.make_table()
now looks like:
headers = {}
The apache table is basically used a Python dictionary. The only
functionality that is lost is that apache tables are case
insensitive, and that can be easily fixed by creating a class in
Python that inherits from dict type and override the __getitem__ and
__setitem__ methods.
For the moment I'm going to keep this changes until modpython.org
release a patch. I spent quite sometime trying to investigate and
solve the memory leak problem but the best I was able to do was to
work around it.
BTW, apache.table, apache.make_table or _apache.table is only being
used in mod_python/util.py.
/amn
Memory leak apache.table()
--
Key: MODPYTHON-184
URL: http://issues.apache.org/jira/browse/MODPYTHON-184
Project: mod_python
Issue Type: Bug
Components: core
Affects Versions: 3.3, 3.2.10
Reporter: Jim Gallacher
Assigned To: Jim Gallacher
Fix For: 3.3
Attachments: MP184-2006-08-25-grahamd-1.diff
There is a memory leak in apache.table().
from mod_python import apache
def handler(req):
req.content_type = 'text/plain'
t = apache.make_table()
req.write('ok table:')
return apache.OK
Using mpm-worker with StartServers 2, and 2 requests results in memory
consumption going from 1.2% to 9.3% per process. (ie approx 8k per request)
This will have an impact on FieldStorage which makes use of
apache.make_table(), which is the deprecated name for apache.table()
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-184) Memory leak apache.table()
[
http://issues.apache.org/jira/browse/MODPYTHON-184?page=comments#action_12447666
]
Jim Gallacher commented on MODPYTHON-184:
-
[[ Old comment, sent by email on Wed, 16 Aug 2006 17:44:21 -0400 ]]
Actually I don't think apr_pool_destroy() in table_dealloc is actually
destroying the pool. I've been poking around in the code and there is
something odd going on here.
I tried registering a cleanup in MpTable_New() using:
apr_pool_cleanup_register(t-pool,
pool cleanup called,
cleanup_test,
apr_pool_cleaunp_null);
The cleanup_test callback just logs the pool cleanup called message to
a file.
apr_pool_destroy() is getting called in table_dealloc, but cleanup_test
never gets called which indicates that the pool is *not* being
destroyed, and hence our memory leak.
I tried your trick of immediately calling apr_pool_destroy in
MpTable_New(), and cleanup_test does get called there.
So, the big question is... why is the pool not being destroyed?
Can anyone offer some insight here?
The attached diff is for trunk if anyone wants to play around with it.
Jim
Index: tableobject.c
===
--- tableobject.c (revision 431994)
+++ tableobject.c (working copy)
@@ -59,6 +59,19 @@
return (PyObject *)result;
}
+
+
+apr_status_t cleanup_test(void *msg)
+{
+FILE *f;
+f = fopen(/tmp/debug_table.log, a+);
+fprintf(f, %s\n, (char *)msg);
+fclose(f);
+
+return 0;
+}
+
+
/**
** MpTable_New
**
@@ -78,6 +91,8 @@
tableobject *t;
apr_pool_t *p;
+cleanup_test(MpTable_New() called);
+
/* XXX need second arg abort function to report mem error */
apr_pool_create_ex(p, NULL, NULL, NULL);
@@ -86,7 +101,12 @@
/* remember the pointer to our own pool */
t-pool = p;
+apr_pool_cleanup_register(p, pool cleanup called, cleanup_test,
apr_pool_cleanup_null);
+/* Uncomment this to test that cleanup_test is getting called correctly.
+apr_pool_destroy(t-pool);
+*/
+
return (PyObject *)t;
}
@@ -99,10 +119,13 @@
static void table_dealloc(register tableobject *self)
{
+cleanup_test(table_dealloc:);
if (MpTable_Check(self)) {
-if (self-pool)
+if (self-pool) {
+cleanup_test( preparing to destroy the pool);
apr_pool_destroy(self-pool);
+}
PyObject_Del(self);
}
else
Memory leak apache.table()
--
Key: MODPYTHON-184
URL: http://issues.apache.org/jira/browse/MODPYTHON-184
Project: mod_python
Issue Type: Bug
Components: core
Affects Versions: 3.3, 3.2.10
Reporter: Jim Gallacher
Assigned To: Jim Gallacher
Fix For: 3.3
Attachments: MP184-2006-08-25-grahamd-1.diff
There is a memory leak in apache.table().
from mod_python import apache
def handler(req):
req.content_type = 'text/plain'
t = apache.make_table()
req.write('ok table:')
return apache.OK
Using mpm-worker with StartServers 2, and 2 requests results in memory
consumption going from 1.2% to 9.3% per process. (ie approx 8k per request)
This will have an impact on FieldStorage which makes use of
apache.make_table(), which is the deprecated name for apache.table()
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-195) Possible leaking of Win32 event handles when Apache restarted.
[ http://issues.apache.org/jira/browse/MODPYTHON-195?page=comments#action_12447680 ] Jeff Robbins commented on MODPYTHON-195: [[ Old comment, sent from unregistered email on Tue, 24 Oct 2006 06:56:50 -0400 ]] I think the problem with a static is that the code gets called in the context of the parent process and the child process. We need some way of knowing only to run in the context of the child process. mpm_winnt.c uses AP_PARENT_PID as an environmental only visible to the child process so while it is risky in the sense of a dependency, it is how the code that apparently produces this problem by creating the windows-specific process architecture communicates with itself. Possible leaking of Win32 event handles when Apache restarted. -- Key: MODPYTHON-195 URL: http://issues.apache.org/jira/browse/MODPYTHON-195 Project: mod_python Issue Type: Bug Components: core Affects Versions: 3.2.10 Reporter: Graham Dumpleton Jeff Robins in: http://mail-archives.apache.org/mod_mbox/httpd-python-dev/200610.mbox/[EMAIL PROTECTED] indicates a belief that when an Apache restart is performed on Windows that there are a number of Win32 event handles leaked. His belief is that this seems to be linked to mod_python. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-195) Possible leaking of Win32 event handles when Apache restarted.
[ http://issues.apache.org/jira/browse/MODPYTHON-195?page=comments#action_12447679 ] Jeff Robbins commented on MODPYTHON-195: [[ Old comment, sent from unregistered email on Tue, 24 Oct 2006 06:56:50 -0400 ]] I think the problem with a static is that the code gets called in the context of the parent process and the child process. We need some way of knowing only to run in the context of the child process. mpm_winnt.c uses AP_PARENT_PID as an environmental only visible to the child process so while it is risky in the sense of a dependency, it is how the code that apparently produces this problem by creating the windows-specific process architecture communicates with itself. Possible leaking of Win32 event handles when Apache restarted. -- Key: MODPYTHON-195 URL: http://issues.apache.org/jira/browse/MODPYTHON-195 Project: mod_python Issue Type: Bug Components: core Affects Versions: 3.2.10 Reporter: Graham Dumpleton Jeff Robins in: http://mail-archives.apache.org/mod_mbox/httpd-python-dev/200610.mbox/[EMAIL PROTECTED] indicates a belief that when an Apache restart is performed on Windows that there are a number of Win32 event handles leaked. His belief is that this seems to be linked to mod_python. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-195) Possible leaking of Win32 event handles when Apache restarted.
[ http://issues.apache.org/jira/browse/MODPYTHON-195?page=comments#action_12447681 ] Jeff Robbins commented on MODPYTHON-195: [[ Old comment, sent from unregistered email on Tue, 24 Oct 2006 09:13:26 -0400 ]] Graham, I added a printout after the call to apr_pool_user_data_get. What I think is happening is that the hook python_init() is called again in the parent process (so data is already 1) and then called the expected 2 times in the child process (wherein the usual protection works and the bulk of python_init() only runs through on the second time in. The problem is that the parent process on windows is long-lived and is just there to spin up and down the child process that does the real web serving. It is the parent process run through python_init() that needs to be defeated, and the usual protection does no good there. code in mod_python.c: rc = apr_pool_userdata_get(data, userdata_key, s-process-pool); // JSR ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, s, python_init: apr_pool_userdata_get() rc=%d data=%x, rc, data); error log fragment: (with my comments added): [Tue Oct 24 09:04:41 2006] [notice] Parent: Received restart signal -- Restarting the server. [Tue Oct 24 09:04:41 2006] [notice] Child 1952: Exit event signaled. Child process is ending. [Tue Oct 24 09:04:41 2006] [error] python_init: apr_pool_userdata_get() rc=0 data=1 // this is the call in the parent process that we need to skip [Tue Oct 24 09:04:41 2006] [notice] Parent: Created child process 9688 [Tue Oct 24 09:04:41 2006] [error] python_init: apr_pool_userdata_get() rc=0 data=0 // these are the two calls in the new child process [Tue Oct 24 09:04:41 2006] [error] python_init: apr_pool_userdata_get() rc=0 data=1 [Tue Oct 24 09:04:41 2006] [notice] mod_python: Creating 8 session mutexes based on 0 max processes and 50 max threads. [Tue Oct 24 09:04:41 2006] [notice] Child 9688: Child process is running [Tue Oct 24 09:04:42 2006] [notice] Child 1952: Released the start mutex [Tue Oct 24 09:04:42 2006] [notice] Child 9688: Acquired the start mutex. [Tue Oct 24 09:04:42 2006] [notice] Child 9688: Starting 50 worker threads. Given the long-lived nature of the parent process, I'm beginning to think that your idea of a static might be better than checking for the environmental. - Jeff Possible leaking of Win32 event handles when Apache restarted. -- Key: MODPYTHON-195 URL: http://issues.apache.org/jira/browse/MODPYTHON-195 Project: mod_python Issue Type: Bug Components: core Affects Versions: 3.2.10 Reporter: Graham Dumpleton Jeff Robins in: http://mail-archives.apache.org/mod_mbox/httpd-python-dev/200610.mbox/[EMAIL PROTECTED] indicates a belief that when an Apache restart is performed on Windows that there are a number of Win32 event handles leaked. His belief is that this seems to be linked to mod_python. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
