Are you saying that your algorithm actually dives down and uses Django database ORM to get data from database?
That is, it isn't a standalone bit of code which can be given its own inputs and returns the result, totally independent of your web application and whatever database it is using? That would only be required if the algorithm is wanting to access that data setup by Django directly. Ideally you may the service and the algorithm usable independent of your web application. Graham On 27/02/2015, at 10:07 AM, Paul Royik <[email protected]> wrote: > I need to use django setup because of the error described here: > http://stackoverflow.com/questions/25537905/django-1-7-throws-django-core-exceptions-appregistrynotready-models-arent-load > > Can you clarify where code should lie? Do I need to separate timeout and > MyManager? > > On Friday, February 27, 2015 at 1:03:19 AM UTC+2, Graham Dumpleton wrote: > > On 27/02/2015, at 9:51 AM, Paul Royik <[email protected]> wrote: > > I did it directly on Webfaction. > Created webapps/django_math/mathsite/mathsite/task-queue-manager.py with > exact code you've supplied. > > It can't be exactly the same as you have moved where MathClass is defined, > plus the location of the UNIX socket will now no longer match. > > Try something like: > > import os > > from mathsite.utils import MyManager > > # This shouldn't be hardwired, but just to get it working. > > sockpath = '/home/simamura/webapps/django_math/task-queue-manager.sock' > > try: > os.unlink(sockpath) > except OSError: > pass > > m = MyManager(address=sockpath, authkey='abracadabra') > s = m.get_server() > s.serve_forever() > > Then inside webapps/django_math/mathsite/mathsite/utils.py along with timeout > decorator, I created following: > > class RunableProcessing(multiprocessing.Process): > def __init__(self, func, *args, **kwargs): > self.queue = multiprocessing.Manager().Queue(maxsize=1) > args = (func,) + args > multiprocessing.Process.__init__(self, target=self.run_func, > args=args, kwargs=kwargs) > > def run_func(self, func, *args, **kwargs): > try: > result = func(*args, **kwargs) > self.queue.put((True, result)) > except Exception as e: > self.queue.put((False, e)) > > def done(self): > return self.queue.full() > > def result(self): > return self.queue.get() > > import django > django.setup() > > Why is it you need to be importing and setting up Django. > > One of the aims of separating this out as a service would be so that not web > application code is imported or used. This way it can be very lightweight. > You are adding to memory issues again by importing Django and initialising it > in this service process. > > def timeout(seconds, function, *args, **kwargs): > proc = RunableProcessing(function, *args, **kwargs) > proc.start() > proc.join(seconds) > if proc.is_alive(): > proc.terminate() > proc.join(1) > if proc.is_alive(): > try: > os.kill(proc.pid, signal.SIGKILL) > except Exception: > pass > proc.join(1) > raise TimeoutException > success, result = proc.result() > if success: > return result > else: > raise result > > import subprocess > > > class MathClass(object): > def run(self, seconds, func, *args, **kwargs): > return timeout(seconds, func, *args, **kwargs) > > from multiprocessing.managers import BaseManager > class MyManager(BaseManager): > pass > > MyManager.register('Maths', MathClass) > > Next in my view: > > import os > from mathsite.utils import MyManager > sockpath = os.path.join(os.path.dirname(__file__), 'task-queue-manager.sock') > m = MyManager(address=sockpath, authkey='abracadabra') > m.connect() > timeout_func = m.Maths().run > > timeout_func(my_function.....) > > The idea was that this is not importing the actual code for the service, but > going through an abstract interface. > > # The following would be at global scope. > > import os > from multiprocessing.managers import BaseManager > > class MyManager(BaseManager): > pass > > MyManager.register('Maths') > > # This shouldn't be hardwired, but just to get it working. Must match was > service is using. > > sockpath = '/home/simamura/webapps/django_math/task-queue-manager.sock' > > # The following would be within the code of the view handler function. > > m = MyManager(address=sockpath, authkey='abracadabra') > m.connect() > timeout_func = m.Maths().run > > timeout_func(my_function.....) > > Graham > > > On Friday, February 27, 2015 at 12:43:18 AM UTC+2, Graham Dumpleton wrote: > So you tried doing this on WebFaction rather than on your own system with > just the files I gave you and running mod_wsgi-express directly? > > Can you explain exactly what you did? > > Graham > > On 27/02/2015, at 9:37 AM, Paul Royik <[email protected]> wrote: > > [Thu Feb 26 19:58:06.587734 2015] [wsgi:info] [pid 27981:tid 139944202344192] > [remote 127.0.0.1:60] mod_wsgi (pid=27981, process='localhost:20241', > application=''): Loading WSGI script > '/home/simamura/webapps/django_math/express/handler.wsgi'. > [Thu Feb 26 19:58:06.588460 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] mod_wsgi (pid=27981): Target WSGI > script '/home/simamura/webapps/django_math/express/handler.wsgi' cannot be > loaded as Python module. > [Thu Feb 26 19:58:06.588470 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] mod_wsgi (pid=27981): Exception > occurred processing WSGI script > '/home/simamura/webapps/django_math/express/handler.wsgi'. > [Thu Feb 26 19:58:06.588485 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] Traceback (most recent call last): > [Thu Feb 26 19:58:06.588499 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] File > "/home/simamura/webapps/django_math/express/handler.wsgi", line 71, in > <module> > [Thu Feb 26 19:58:06.588522 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] > recorder_directory=recorder_directory) > [Thu Feb 26 19:58:06.588530 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] File > "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1124, in > __init__ > [Thu Feb 26 19:58:06.588544 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote 127.0.0.1:60] exec(code, self.module.__dict__) > [Thu Feb 26 19:58:06.588551 2015] [wsgi:error] [pid 27981:tid > 139944202344192] [remote > ... > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
