On Sat, Aug 14, 2010 at 1:28 PM, ydjango <[email protected]> wrote: > I have a online user initiated synchronous process which runs anywhere > between 1-5 minutes and gives user status message at the end. It is a > very DB intensive process that reads and updates lots of mysql rows > and does many calculations. The process is run as part of a view > method.
long-running tasks don't belong to the request-responce cycle, send them to a background process. one of the easiest solutions is to use "Ghetto Queues". it can look like this: - create a new 'task' table, with fields to specify how to process a task and a status field (preparing, ready, processing, done, failed), if needed a 'result' field and if possible a 'percent done' field too. - when the user initiates a task, don't process it in the view function. just add a new record to the task table, with all the info needed to process it and set in 'ready' state. then return. don't forget to note the task record ID somewhere you can relate to the user's action. - another process, separate from the Django server (can be a cron task in many cases) checks the task table and picks any record in 'ready' state to do all the needed processing, first changing it to 'processing' state. it should update the 'percent done' field, and set to 'done' or 'failed' when finished. - make some way for the user to get the status of the task, either revisiting the same URL (which should show the status if there's a task in 'processing'), or via some AJAX query. there are many variations of this strategy: you can have several 'workers' if you pay attention to the way it picks a task and sets to 'processing' (transactions make it easy to atomically pick one), if you need low latency between queuing a task and picking it by the worker process, you can replace the task table with a queue manager (check carrot and/or celery) -- Javier -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
