Here is my express-config
mod_wsgi-express setup-server
--working-directory ${SITE_NAME}
--server-root express --port ${SITE_PORT}
--log-directory ${LOG_DIR} --access-log --access-log-format "${LOG_FORMAT}"
--access-log-name access_${APPL_NAME}.log --error-log-name
error_${APPL_NAME}.log
--python-path ${SITE_ROOT}/lib/python${PYTHON_VERSION}
--threads 10
--service-script tasks $SITE_NAME/$SITE_NAME/timeout/task-queue-manager.py
$SITE_NAME/$SITE_NAME/wsgi.py
Don't see where I can tune number of processes.
Also, I will find out about how many instances can external library handle
and let you know.
On Thursday, March 12, 2015 at 3:38:17 AM UTC+2, Graham Dumpleton wrote:
>
>
> On 12/03/2015, at 4:06 AM, Paul Royik <[email protected] <javascript:>>
> wrote:
>
> I still experience problems with memory overflow on webfaction
> But it seems that problem is not in service:tasks (it takes only 9 MB).
> I don't know why there are so many other threads.
>
> simamura - 1MB - 23:37:52 - 1571 -
> /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f
> /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start
> simamura - 43MB - 23:37:52 - 1587 -
> /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f
> /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start
> simamura - 43MB - 23:37:52 - 1588 -
> /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f
> /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start
> simamura - 2MB - 23:37:52 - 1589 -
> /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f
> /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start
>
>
> Your other Apache instance.
>
> simamura - 2MB - 23:42:53 - 23836 - httpd (mod_wsgi-express) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
>
>
> Apache parent process.
>
> simamura - 3MB - 11:08:36 - 4462 - httpd (mod_wsgi-express) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
>
> simamura - 3MB - 13:37:49 - 18634 - httpd (mod_wsgi-express) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
>
>
> Apache child worker processes which proxy to mod_wsgi daemon processes.
>
> simamura - 114MB - 0:16:12 - 10076 - (wsgi:localhost:20241:1092) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
> simamura - 120MB - 0:15:15 - 11838 - (wsgi:localhost:20241:1092) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
> simamura - 131MB - 0:11:15 - 20119 - (wsgi:localhost:20241:1092) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
> simamura - 98MB - 0:10:48 - 20908 - (wsgi:localhost:20241:1092) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
>
>
> The mod_wsgi daemon processes.
>
> simamura - 9MB - 23:42:53 - 23854 - (wsgi:service:tasks) -f
> /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG
> -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE
> -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
>
>
> The service process which acts as parent for the task processes.
>
> What are you currently using for the --processes and --threads option to
> mod_wsgi-express when setting up configuration?
>
> The above suggests you are using '--processes 4' and looks completely
> normal. So I don't know what you mean by are so many threads.
>
> What the above doesn't show is how big your transient task worker
> processes are getting when running your algorithm.
>
> That is important as right now there is no limit on how many of those can
> run concurrently except that of how many concurrent processes your mod_wsgi
> daemon processes can handle.
>
> You need to understand realistically how many concurrent instances of your
> algorithm might handle and if you need to implement a queuing mechanism to
> restrict the number then the queue manager will need to change.
>
> Graham
>
> On Thursday, March 5, 2015 at 9:46:34 AM UTC+2, Paul Royik wrote:
>>
>> If you have time, please respond.
>>
>> On Monday, March 2, 2015 at 2:04:46 PM UTC+2, Paul Royik wrote:
>>
>> I made a research and found no suspicious imports.
>> I give you contents of mathsite.timeout module for further research:
>>
>> exceptions.py
>> class TimeoutException(Exception):
>> pass
>>
>> manager.py
>> from mathsite.timeout.timeout import MyManager
>> import os
>> def get_manager():
>> sockpath = os.path.join(os.path.dirname(__file__),
>> 'task-queue-manager.sock')
>> if True:
>> return
>> try:
>> m = MyManager(address=sockpath, authkey='abracadabra')
>> m.connect()
>> return m.Maths()
>> except:
>> return None
>>
>> task-queue-manager.py
>> import os
>>
>> from mathsite.timeout.timeout import MyManager
>>
>> sockpath = os.path.join(os.path.dirname(__file__),
>> 'task-queue-manager.sock')
>>
>> try:
>> os.unlink(sockpath)
>> except OSError:
>> pass
>>
>> m = MyManager(address=sockpath, authkey='abracadabra')
>> s = m.get_server()
>> s.serve_forever()
>>
>> timeout.py
>> import multiprocessing
>> import os
>> import signal
>> from multiprocessing.managers import BaseManager
>> from mathsite.timeout.exceptions import TimeoutException
>>
>>
>> class RunableProcessing(multiprocessing.Process):
>> def __init__(self, func, *args, **kwargs):
>> self.queue = multiprocessing.Manager().Queue(maxsize=1)
>> args = (func,) + args
>> multiprocessing.Process.__init__(self, target=self.run_func,
>> args=args, kwargs=kwargs)
>>
>> def run_func(self, func, *args, **kwargs):
>> try:
>>
>>
>> ...
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected] <javascript:>
> .
> Visit this group at http://groups.google.com/group/modwsgi.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
