I made one tweak to the double-fork procedure, adding a waitpid call in the grandparent process (the original zope thread) before it returns out of the external method... the code then looked something like:


[...prefork code up to here...]
pid1 = os.fork()
if pid1 > 0:
   #grandparent waits for its child before returning
   os.waitpid(pid1, 0)
   return RESPONSE.redirect("wait_page")
pid2 = os.fork()
os.setsid()
if pid2 > 0:
   #child quits, orphaning grandchild
   sys.exit(0)
[...grandchild-only code after here...]

The waitpid call seems to be preventing the zombies... this is good! (though if you find something bad about this approach, please do speak up)

But in the process, a new bug was created, having to do with the MySQL connection. I now receive a "Lost connection to MySQL server during query" error in what appears to be a final db flush from the grandparent's publish function (ZPublisher.Publish, line 104). The good news is that the grandchild continues to do it's work, but the bad news is that the user receives an error page instead of the redirect to the "wait_page".

My guess is that this has something to do with the various forked processes sharing the MySQL connection, and in particular, I'm guessing the child process that exit(0)'s is closing the connection, which is then unavailable when the grandparent tries to use it.

Here's the interesting part: if I put a sleep(1) line in between the waitpid() line and the return line, I get no error. It would appear that the 1 second pause provides enough time for a new MySQL connection to be made after the previous one is killed (less than 1 second is not enough).

This is a very ugly fix, though... I can't guarantee that 1 second will always be enough... can anyone suggest a cleaner solution? Perhaps I shouldn't kill the child with sys.exit(0)?

-Daniel



On Fri, 13 Oct 2006, Jonathan wrote:

Pls keep your posts on the list - so that others can help and so that others can search for problems/solutions!

----- Original Message -----
Sent: Friday, October 13, 2006 12:13 PM
Subject: zope fork in external method



Jonathan-

I'm emailing you because I saw your post a few months back at:

http://mail.zope.org/pipermail/zope/2006-May/166574.html

I've been doing my best to find answers on existing posts, but to no avail. Perhaps you can send further pointers?

On my system, a user can hit a page that initiates a long database query. If the user hits the stop button and tries to refresh, it messes things up (by interrupting the code upon return of the query), so I came up with a solution that almost works:

I put all the code into an external method, which forks. The parent returns a redirect to a "waiting" throbber page that uses Ajax to poll whether the query is done. The child then goes on to do the hard work.

This is working as desired, with a single exception: zombie child processes. Where can I learn more about forking processes in zope external methods? I've read about the double-fork method, but that hasn't resolved the problem...

A few ideas for reaping dead child processes:

1) implement a SIGCHLD handler

2) when a spawned child process is finished its 'zope processing' have it write its process id to a file (be careful when accessing files, you will need locking to eliminate problems that may be caused by zope's multiple threads) and then have a clean up routine which kills all of the processes listed in the file (this could be an independant clean-up routine which wakes up on a regular basis, or it could be built into the 'spawning' process which does the clean up before it spawns a child).

3) as the last thing it does, have the child process issue a kill -9 system command to kill its own process (I haven't tried this myself, but it may work and it would be simple to implement)

Good luck!

Jonathan



_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to