OK. I will explore that. Thanks. On Sunday, February 22, 2015 at 12:55:20 PM UTC+2, Graham Dumpleton wrote: > > I can only offer you a direction of where to look right now. I have some > things I have to deal with over the new few days and so you may not get a > quick response. > > On signals and using SIGALRM in particular, no that cannot be used. > > As far as implementing a simple task execution system, start researching > the examples in: > > https://docs.python.org/2/library/multiprocessing.html#customized-managers > > https://docs.python.org/2/library/multiprocessing.html#using-a-remote-manager > > I believe that functionality from the multiprocessing module could be > used, but I simply don't have the time to be able to explain it right now. > > Graham > > On 22/02/2015, at 9:00 AM, Paul Royik <[email protected] <javascript:>> > wrote: > > OK. Let's summarize it. > I have an external function and want to interrupt it in the cleanest and > most natural way. > I'm newbie to all this threading and multiprocessing. > You gave me multiprocessing code and then warned that it might work not > very clean. > > Can you tell me possible way and give all pros and cons, so I can make > some conclusion? > You said something about separate task system? I have no idea how to > implement it. All python solutions seemed to be around multithreading, > signal, multiprocessing. > By the way, will signal solution work with new mod_wsgi? What are pros and > cons? > > import signal > def signal_handler(signum, frame): > raise Exception("Timed out!") > > signal.signal(signal.SIGALRM, signal_handler) > signal.alarm(10) # Ten secondstry: > long_function_call()except Exception, msg: > print "Timed out!" > > > Thank you for your time. > > On Saturday, February 21, 2015 at 10:44:22 PM UTC+2, Paul Royik wrote: > > *Which is part of the reason why I said it isn't a good idea to be forking > worker subprocesses from your web application to do work. When you do that, > the starting memory usage will be whatever the process was that was > handling the web request to begin with. Your algorithm will then add on top > of that.* > > *So if you web application has very fat memory usage, that carries > through.* > > *This is why I was saying that you are better off with a separate task > execution system. Its processes would have very small amount of memory > usage to start with as it doesn't have all your web application memory as > well. The separate task execution system can also control how many parallel > tasks run and so limit overall memory usage.* > > *So, this seems not very good solution. I have only 512 MB. How to > implement separate task system?* > > *Concerning your last question. I use FTP manager like Filezilla and that > time I reuploaded all project (even that wsgi.py wasn't changed).* > > > On Saturday, February 21, 2015 at 10:00:53 PM UTC+2, Graham Dumpleton > wrote: > > On 22/02/2015, at 5:33 AM, Paul Royik <[email protected]> wrote: > > From time to time I've got following error: > [Sat Feb 21 12:24:27.181049 2015] [wsgi:info] [pid 5482:tid > 139829356984064] [remote 127.0.0.1:12680] mod_wsgi (pid=5482): Force > restart of process 'localhost:20241'. > > Is it normal? > > > That means that the WSGI script file for your application was modified. > This is detected and the mod_wsgi daemon process restarted automatically. > > This is another reason why you shouldn't be doing stuff under web > application processes. > > When this occurs, if you had an algorithm worker process running at the > time, the killing off of the daemon process running your web application > may not result in the worker being killed and so it could keep running with > nothing then killing it. > > Will have to check that though, as depends on whether when daemon process > is killed that children also killed automatically. > > How do you upload code changes to your system? Are you just using ssh to > copy your whole code base every time and so the WSGI script file > modification time will be updated even if not changed? > > Graham > > On Saturday, February 21, 2015 at 3:14:16 PM UTC+2, Paul Royik wrote: > > Thank you very much. > It works now. > But I noticed one thing. When calculator works, memory jumps from 180 to > 250 (and the back to 180 as expected). When I used thread method, I didn't > see such big jump. Can you tell me why in this case the jump is so big? I > guess, that if 10 people will do it simulaneously, then I will run out of > memory. > > Anyway, this is the cleanest and best solution. > > > On Saturday, February 21, 2015 at 2:26:40 PM UTC+2, Graham Dumpleton wrote: > > The short answer is that code will not work with the version of mod_wsgi > you are using. It will not kill the forked process properly and would > actually kill the WSGI application daemon process instead, leaving the > forked process still running and thus why it consumed memory. That it was > killing the WSGI application daemon process would have been evident in the > error logs which is why I asked about logs. > > Some extra tricks were required with a special directive needing to be > given to mod_wsgi for the version you are using to allow signal handlers to > be overridden and the forked process would need to reset the signal > handlers before running the job. > > Since it was a pain to have to use the fiddles I worked out a way in > mod_wsgi to avoid needing to do all that. You will need to upgrade to the > latest mod_wsgi version though, which is 4.4.9. > > That isn't all that is required though as that code doesn't properly wait > on the process when terminating it. This will result in a zombie process > hanging around, which isn't a good idea. In worst case, since terminate() > only tries SIGTERM, it may not go away. > > Now you can't simply go run 'pip install -U > > ...
-- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
