On 27/02/2015, at 9:51 AM, Paul Royik <[email protected]> wrote:
> I did it directly on Webfaction.
> Created webapps/django_math/mathsite/mathsite/task-queue-manager.py with
> exact code you've supplied.
It can't be exactly the same as you have moved where MathClass is defined, plus
the location of the UNIX socket will now no longer match.
Try something like:
import os
from mathsite.utils import MyManager
# This shouldn't be hardwired, but just to get it working.
sockpath = '/home/simamura/webapps/django_math/task-queue-manager.sock'
try:
os.unlink(sockpath)
except OSError:
pass
m = MyManager(address=sockpath, authkey='abracadabra')
s = m.get_server()
s.serve_forever()
> Then inside webapps/django_math/mathsite/mathsite/utils.py along with timeout
> decorator, I created following:
>
> class RunableProcessing(multiprocessing.Process):
> def __init__(self, func, *args, **kwargs):
> self.queue = multiprocessing.Manager().Queue(maxsize=1)
> args = (func,) + args
> multiprocessing.Process.__init__(self, target=self.run_func,
> args=args, kwargs=kwargs)
>
> def run_func(self, func, *args, **kwargs):
> try:
> result = func(*args, **kwargs)
> self.queue.put((True, result))
> except Exception as e:
> self.queue.put((False, e))
>
> def done(self):
> return self.queue.full()
>
> def result(self):
> return self.queue.get()
>
> import django
> django.setup()
Why is it you need to be importing and setting up Django.
One of the aims of separating this out as a service would be so that not web
application code is imported or used. This way it can be very lightweight. You
are adding to memory issues again by importing Django and initialising it in
this service process.
> def timeout(seconds, function, *args, **kwargs):
> proc = RunableProcessing(function, *args, **kwargs)
> proc.start()
> proc.join(seconds)
> if proc.is_alive():
> proc.terminate()
> proc.join(1)
> if proc.is_alive():
> try:
> os.kill(proc.pid, signal.SIGKILL)
> except Exception:
> pass
> proc.join(1)
> raise TimeoutException
> success, result = proc.result()
> if success:
> return result
> else:
> raise result
>
> import subprocess
>
>
> class MathClass(object):
> def run(self, seconds, func, *args, **kwargs):
> return timeout(seconds, func, *args, **kwargs)
>
> from multiprocessing.managers import BaseManager
> class MyManager(BaseManager):
> pass
>
> MyManager.register('Maths', MathClass)
>
> Next in my view:
>
> import os
> from mathsite.utils import MyManager
> sockpath = os.path.join(os.path.dirname(__file__), 'task-queue-manager.sock')
> m = MyManager(address=sockpath, authkey='abracadabra')
> m.connect()
> timeout_func = m.Maths().run
>
> timeout_func(my_function.....)
The idea was that this is not importing the actual code for the service, but
going through an abstract interface.
# The following would be at global scope.
import os
from multiprocessing.managers import BaseManager
class MyManager(BaseManager):
pass
MyManager.register('Maths')
# This shouldn't be hardwired, but just to get it working. Must match was
service is using.
sockpath = '/home/simamura/webapps/django_math/task-queue-manager.sock'
# The following would be within the code of the view handler function.
m = MyManager(address=sockpath, authkey='abracadabra')
m.connect()
timeout_func = m.Maths().run
timeout_func(my_function.....)
Graham
>
> On Friday, February 27, 2015 at 12:43:18 AM UTC+2, Graham Dumpleton wrote:
> So you tried doing this on WebFaction rather than on your own system with
> just the files I gave you and running mod_wsgi-express directly?
>
> Can you explain exactly what you did?
>
> Graham
>
> On 27/02/2015, at 9:37 AM, Paul Royik <[email protected]> wrote:
>
> [Thu Feb 26 19:58:06.587734 2015] [wsgi:info] [pid 27981:tid 139944202344192]
> [remote 127.0.0.1:60] mod_wsgi (pid=27981, process='localhost:20241',
> application=''): Loading WSGI script
> '/home/simamura/webapps/django_math/express/handler.wsgi'.
> [Thu Feb 26 19:58:06.588460 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] mod_wsgi (pid=27981): Target WSGI
> script '/home/simamura/webapps/django_math/express/handler.wsgi' cannot be
> loaded as Python module.
> [Thu Feb 26 19:58:06.588470 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] mod_wsgi (pid=27981): Exception
> occurred processing WSGI script
> '/home/simamura/webapps/django_math/express/handler.wsgi'.
> [Thu Feb 26 19:58:06.588485 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] Traceback (most recent call last):
> [Thu Feb 26 19:58:06.588499 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] File
> "/home/simamura/webapps/django_math/express/handler.wsgi", line 71, in
> <module>
> [Thu Feb 26 19:58:06.588522 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60]
> recorder_directory=recorder_directory)
> [Thu Feb 26 19:58:06.588530 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] File
> "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1124, in
> __init__
> [Thu Feb 26 19:58:06.588544 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] exec(code, self.module.__dict__)
> [Thu Feb 26 19:58:06.588551 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] File
> "/home/simamura/webapps/django_math/mathsite/mathsite/wsgi.py", line 27, in
> <module>
> [Thu Feb 26 19:58:06.588564 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] from django.core.wsgi import
> get_wsgi_application
> [Thu Feb 26 19:58:06.588578 2015] [wsgi:error] [pid 27981:tid
> 139944202344192] [remote 127.0.0.1:60] ImportError: No module named
> django.core.wsgi
>
> On Thursday, February 26, 2015 at 11:33:18 PM UTC+2, Graham Dumpleton wrote:
> The service script failed on startup most likely. You need to look back in
> the Apache error log to see what failed when server was started.
>
> Graham
>
> On 27/02/2015, at 7:00 AM, Paul Royik <[email protected]> wrote:
>
> I've got following error:
> File "/usr/local/lib/python2.7/multiprocessing/managers.py", line 500, in
> connect
> conn = Client(self._address, authkey=self._authkey)
> File "/usr/local/lib/python2.7/multiprocessing/connection.py", line 169, in
> Client
> c = SocketClient(address)
> File "/usr/local/lib/python2.7/multiprocessing/connection.py", line 304, in
> SocketClient
> s.connect(address)
> File "/usr/local/lib/python2.7/socket.py", line 224, in meth
> return getattr(self._sock,name)(*args)
> error: [Errno 2] No such file or directory
>
> Maybe I mixed directories? Is layout of files and code strict?
> Code lies in the same directory, but it is not root directory.
>
> On Thursday, February 26, 2015 at 3:00:09 PM UTC+2, Paul Royik wrote:
> Thank you very much for you response.
> I will explore it, however, it is hard fo me to imagine, how to insert my
> long-running functions into manager class.
>
> On Thursday, February 26, 2015 at 12:15:28 PM UTC+2, Graham Dumpleton wrote:
> You will need to install latest mod_wsgi from git repo as I made some tweaks
> to make this easier.
>
> So on your own personal system (not Webfaction) so you can experiment, do:
>
> pip install -U
> https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip
>
> Now create three files.
>
> The first called 'task-queue-client.py'.
>
> This contains the following and is a simple WSGI hello world program.
>
> import os
>
> from multiprocessing.managers import BaseManager
>
> class MyManager(BaseManager):
> pass
>
> MyManager.register('Maths')
>
> sockpath = os.path.join(os.path.dirname(__file__), 'task-queue-manager.sock')
>
> def application(environ, start_response):
> status = '200 OK'
> output = b'Hello World!'
>
> m = MyManager(address=sockpath, authkey='abracadabra')
> m.connect()
>
> maths = m.Maths()
> print maths.add
> ...
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/modwsgi.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
