I spoke too soon. It was working, but consistent with the Curse of Fork, the mysql error eventually started showing up inconsistently. At first I thought it might be dependent on zope's debug mode, but it started showing up either way.

So I did what I didn't want to do: split out the 'grandchild' code into a command-line script... with the resulting basic structure:

#do some zope stuff here
rv = os.system("shell_script.sh")
if rv:
  #handle error here, such as command not found
return RESPONSE.redirect("wait_page")

#do some shell stuff here
python_script.py &
exit 0

#all the grandchild code here

My understanding of this is that (1) os.system creates a shell in a new process [without copying the parent's stack and files?], (2) the ampersand line in the shell script creates a grandchild process, and (3) the shell script exits, orphaning the python_script and allowing the grandparent to proceed.

Maybe it's a little roundabout, but this seems to be cleaner and more reliable so far. Comments?


On Tue, 17 Oct 2006, Daniel Lopez wrote:

You're good, my friend. I was hoping something like this existed, and so far, it seems to be working exactly as desired.

One note for others: it's actually os._exit() not sys._exit()

Re: the other suggestions:

(1) I was already using runzope

(2) subprocess module: I'm using an older version of zope and python, and can't currently risk upgrading core components.

For true cleanliness, in order to avoid the use of fork(), I could try to pull out the grandchild's code into a self-standing script that could get spawned by the grandparent, but that would require a great deal of work, as the grandchild makes a few calls back into zope, along with using some information in the request, which I'd need to pass to the standalone script. I'm happy the lazier approach worked.

Thanks to all...

On Tue, 17 Oct 2006, Dieter Maurer wrote:

Daniel Lopez wrote at 2006-10-16 13:31 -0700:
I made one tweak to the double-fork procedure, adding a waitpid call in the grandparent process (the original zope thread) before it returns out of the
external method... the code then looked something like:

[...prefork code up to here...]
pid1 = os.fork()
if pid1 > 0:
   #grandparent waits for its child before returning
   os.waitpid(pid1, 0)
   return RESPONSE.redirect("wait_page")
pid2 = os.fork()
if pid2 > 0:
   #child quits, orphaning grandchild
[...grandchild-only code after here...]

The waitpid call seems to be preventing the zombies... this is good!
(though if you find something bad about this approach, please do speak up)

But in the process, a new bug was created, having to do with the MySQL
connection.  I now receive a "Lost connection to MySQL server during
query" error in what appears to be a final db flush from the grandparent's
publish function (ZPublisher.Publish, line 104).  The good news is that
the grandchild continues to do it's work, but the bad news is that the
user receives an error page instead of the redirect to the "wait_page".

Maybe, the MySQL client library installed an "atexit" hook that closes
the database connection.

You may try "sys._exit(0)" (instead of "sys.exit(0)") to finish
the child. "_exit" is for purposes when you do not want standard
exit handling (such as flushing buffers and calling "atexit" hooks).


Zope maillist  -  Zope@zope.org
**   No cross posts or HTML encoding!  **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to