Here you go: ['Queue', 'UserDict', '__builtin__', '__future__', '__main__', '_abcoll', '_codecs', '_collections', '_functools', '_hashlib', '_heapq', '_io', '_locale', '_mod_wsgi_f07fbb7150823bae22c26382ad841b54', '_multiprocessing', '_random', '_socket', '_sre', '_ssl', '_struct', '_sysconfigdata', '_warnings', '_weakref', '_weakrefset', 'abc', 'apache', 'array', 'atexit', 'binascii', 'cPickle', 'cStringIO', 'codecs', 'collections', 'copy_reg', 'encodings', 'encodings.__builtin__', 'encodings.aliases', 'encodings.ascii', 'encodings.codecs', 'encodings.encodings', 'encodings.utf_8', 'errno', 'exceptions', 'fcntl', 'functools', 'gc', 'genericpath', 'hashlib', 'heapq', 'imp', 'io', 'itertools', 'keyword', 'linecache', 'marshal', 'math', 'mathsite', 'mathsite.timeout', 'mathsite.timeout.exceptions', 'mathsite.timeout.mathsite', 'mathsite.timeout.multiprocessing', 'mathsite.timeout.os', 'mathsite.timeout.signal', 'mathsite.timeout.timeout', 'mod_wsgi', 'mod_wsgi.pkgutil', 'multiprocessing', 'multiprocessing.Queue', 'multiprocessing._multiprocessing', 'multiprocessing.array', 'multiprocessing.atexit', 'multiprocessing.cPickle', 'multiprocessing.connection', 'multiprocessing.errno', 'multiprocessing.forking', 'multiprocessing.functools', 'multiprocessing.itertools', 'multiprocessing.managers', 'multiprocessing.multiprocessing', 'multiprocessing.os', 'multiprocessing.pickle', 'multiprocessing.process', 'multiprocessing.signal', 'multiprocessing.socket', 'multiprocessing.subprocess', 'multiprocessing.sys', 'multiprocessing.tempfile', 'multiprocessing.threading', 'multiprocessing.time', 'multiprocessing.traceback', 'multiprocessing.util', 'multiprocessing.weakref', 'operator', 'os', 'os.path', 'pickle', 'pkgutil', 'posix', 'posixpath', 'random', 're', 'select', 'signal', 'site', 'sitecustomize', 'socket', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'struct', 'subprocess', 'sys', 'sysconfig', 'tempfile', 'thread', 'threading', 'time', 'traceback', 'types', 'warnings', 'weakref', 'zipimport']
On Saturday, February 28, 2015 at 11:36:37 AM UTC+2, Graham Dumpleton wrote: > > What code modules does this algorithm require. > > If that is all the (wsgi:service:tasks) processes, then it shows that just > loading all the code and before the process even really does anything is > using 41MB of memory or there about. > > That is quite a lot just for code. You may well still be dragging it too > much stuff you not need. > > That said, there are a lot of Python packages that are really badly > structured and just to use a single function from a module they have a side > effect of dragging in huge amounts of other stuff you do not need in there > parts of the package. > > In the task-queue-manager.py file, just before you you run > 's.serve_forever()', add: > > import sys > print(sorted(sys.modules.keys())) > > Then from Apache error log, get the list of modules you are importing and > post them. > > Graham > > On 28/02/2015, at 12:29 AM, Paul Royik <[email protected] <javascript:>> > wrote: > > I isolated algorithm and it worked. > > However, I don't see any improvement in memory usage. > Actually, things become even worse. > > 27242 80580 (wsgi:localhost:20241:1092) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 27243 45108 (wsgi:service:tasks) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 27244 3076 httpd (mod_wsgi-express) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 27345 2860 httpd (mod_wsgi-express) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 27393 3024 httpd (mod_wsgi-express) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 30377 41568 (wsgi:service:tasks) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 30902 41620 (wsgi:service:tasks) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > 30914 41604 (wsgi:service:tasks) -f > /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG > -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE > -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start > Total 359.777 MB > > Perhaps, because I use external library? I can't isolate it. > > Also, I noticed, that when algorithm finishes tasks process is not killed. > So, after some time I have a huge list of tasks processes. This doesn't > happen with previous version of timeout. > > On Friday, February 27, 2015 at 3:14:48 PM UTC+2, Paul Royik wrote: > > I isolated algorithm and i > > ... -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
