Algorithm doesn't hit database.
On Friday, February 27, 2015 at 1:11:05 AM UTC+2, Graham Dumpleton wrote: > > Are you saying that your algorithm actually dives down and uses Django > database ORM to get data from database? > > That is, it isn't a standalone bit of code which can be given its own > inputs and returns the result, totally independent of your web application > and whatever database it is using? > > That would only be required if the algorithm is wanting to access that > data setup by Django directly. > > Ideally you may the service and the algorithm usable independent of your > web application. > > Graham > > On 27/02/2015, at 10:07 AM, Paul Royik <[email protected] <javascript:>> > wrote: > > I need to use django setup because of the error described here: > http://stackoverflow.com/questions/25537905/django-1-7-throws-django-core-exceptions-appregistrynotready-models-arent-load > > Can you clarify where code should lie? Do I need to separate timeout and > MyManager? > > On Friday, February 27, 2015 at 1:03:19 AM UTC+2, Graham Dumpleton wrote: > > > On 27/02/2015, at 9:51 AM, Paul Royik <[email protected]> wrote: > > I did it directly on Webfaction. > Created webapps/django_math/mathsite/mathsite/task-queue-manager.py with > exact code you've supplied. > > > It can't be exactly the same as you have moved where MathClass is defined, > plus the location of the UNIX socket will now no longer match. > > Try something like: > > import os > > from mathsite.utils import MyManager > > # This shouldn't be hardwired, but just to get it working. > > sockpath = '/home/simamura/webapps/django_math/task-queue-manager.sock' > > try: > os.unlink(sockpath) > except OSError: > pass > > m = MyManager(address=sockpath, authkey='abracadabra') > s = m.get_server() > s.serve_forever() > > Then inside webapps/django_math/mathsite/mathsite/utils.py along with > timeout decorator, I created following: > > class RunableProcessing(multiprocessing.Process): > def __init__(self, func, *args, **kwargs): > self.queue = multiprocessing.Manager().Queue(maxsize=1) > args = (func,) + args > multiprocessing.Process.__init__(self, target=self.run_func, > args=args, kwargs=kwargs) > > def run_func(self, func, *args, **kwargs): > try: > result = func(*args, **kwargs) > self.queue.put((True, result)) > except Exception as e: > self.queue.put((False, e)) > > def done(self): > return self.queue.full() > > def result(self): > return self.queue.get() > > import django > django.setup() > > > Why is it you need to be importing and setting up Django. > > One of the aims of separating this out as a service would be so that not > web application code is imported or used. This way it can be very > lightweight. You are adding to memory issues again by importing Django and > initialising it in this service process. > > def timeout(seconds, function, *args, **kwargs): > proc = RunableProcessing(function, *args, **kwargs) > proc.start() > proc.join(seconds) > if proc.is_alive(): > proc.terminate() > proc.join(1) > if proc.is_alive(): > try: > os.kill(proc.pid, signal.SIGKILL) > except Exception: > pass > proc.join(1) > raise TimeoutException > success, result = proc.result() > if success: > return result > else: > raise result > > import subprocess > > > class MathClass(object): > def run(self, seconds, func, *args, **kwargs): > return timeout(seconds, func, *args, **kwargs) > > from multiprocessing.managers import BaseManager > class MyManager(BaseManager): > pass > > MyManager.register('Maths', MathClass) > > Next in my view: > > import os > from mathsite.utils import MyManager > sockpath = os.path.join(os.path.dirname(__file__), > 'task-queue-manager.sock') > m = MyManager(address=sockpath, authkey='abracadabra') > m.connect() > timeout_func = m.Maths().run > > timeout_func(my_function.....) > > > The idea was that this is not importing the actual code for the service, > but going through an abstract interface. > > # The following would be at global scope. > > import os > from multiprocessing.managers import BaseManager > > class MyManager(BaseManager): > pass > > MyManager.register('Maths') > > # This shouldn't be hardwired, but just to get it working. Must match was > service is using. > > sockpath = '/home/simamura/webapps/django_math/task-queue-manager.sock' > > # The following would be within the code of the view handler function. > > m = MyManager(address=sockpath, authkey='abracadabra') > m.connect() > timeout_func = m.Maths().run > > timeout_func(my_function.....) > > Graham > > > On Friday, February 27, 2015 at 12:43:18 AM UTC+2, Graham Dumpleton wrote: > > ... -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
