Hi Ron. Many thanks for your reply. This is also a long running document processing challenge, there can be 100's (or even thousands) of records which is why the time problem. The code it uses in my tool in the CMF. How is it that you send a signal to the long running process? Is the long running process cronned to look for a record in the database or is this starting another server of some type.

What I need is something like what you are suggesting - wakes up when there is work in the hopper and chugs along until it is done and then goes to sleep (sort of the way a printing queue works). At the same time it would be great if it was something that had a small RAM footprint or ran without consuming any more than X mbs. The other problem I have with this is that it needs to do work in Zope itself since the final docs end up as objects. I have just found Chris Wither's product called Stepper. I am not sure it this is for this type of situation or more for cronned maintenance. I want to be able to initiate the process right away but asynchronously from the main zope threads.


On Tuesday, September 13, 2005, at 06:05 PM, Ron Bickers wrote:

On Tue September 13 2005 02:52 pm, David Pratt wrote:

Hi. I have a workflow that is triggered by a file upload and the
processing of the file can be minutes of processing depending upon the
size of the file uploaded.  I am concerned about number of threads
available to serve zope so I believe this is a good candidate for an
asynchronous process.  I am looking for some type of outline to do
this.  Currently a tool does the work that is triggered by workflow
script. My hope is to have this process run and send the user an email
to advise when the process has completed instead of the user waiting
for a response or potentially timing out waiting for one. What steps
could I take to make this an ansynchronous process?

I had to do something like this when processing a lot of data to create PDF documents to send via email. The time to do so was too long for them to sit and wait, so I created a separate process to do the job. I don't know if this is the best way, but it wasn't very difficult and it has been working
without problems for many months.

When a user requests the document, I add a record of needed information to a
MySQL table (the "queue"), send a signal to the long-running process
(described next) and immediately return a thank you page.  A
separate-from-Zope long-running Python process waits for a signal, reads the queue table, does what it needs to do, empties the processed items from the queue, and goes idle. It can handle things like getting a signal while it's processing a queue and "catching up" occassionally if it missed a signal
(for whatever reason).

Hope this helps.


Zope maillist  -  Zope@zope.org
**   No cross posts or HTML encoding!  **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to