On Aug 6, 1:46 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Hi, > > I am designing a web page with Web.py and since I don't have that much > experience in Web programming I am writing to the list to ask for > advice. > > My question if about what is the best thing to do when some job > submitted to Web.py will take a ver long time. In my case, what I am > working on is uploading large files then processing within Python and > storing them in SQLite. The processing part is what really takes long, > possible up to an hour. One option I had been thinking about just > forking the process, the parent goes on to report to the user that > file was received correctly, while the child goes to perform the > processing in the background. Since, this is not going to be done very > frequently, I've thought also about disabling access to the website > will the database is updated. > > My question is. Is there a more elegant way to handle this? Using a > 3rd party solution? Or within Web.py?
You would me better of writing the uploaded image to a file in a temporary directory. Once it is completely written, move the file into a pending directory. A distinct process to the web application, possibly even a cron job, could then periodically wake up and check for new files in the pending directory, do the necessary processing and update the database. If cron job is used, because a second cron job may run while the first is still processing the first image, the file should be moved out of the pending directory into a working directory so a second cron job doesn't also start processing it. In other words, you use the filesystem as a staging area, moving the files between directories in an atomic operation so you always know what phase of processing they are up to. Separate processes, not even necessarily Python, but perhaps shell scripts invoking command line tools, can then be employed appropriately to do whatever is required. Importantly, it is removing the burden from the web application itself and would generally allow things to keep operating even if the web application dies or has to be restarted. Graham --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web.py" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/webpy?hl=en -~----------~----~----~----~------~----~------~--~---
