Hi Simon, On Tue, Apr 28, 2009 at 11:44 +1000, Simon wrote: > On 28/04/2009, at 02:32 , holger krekel holger-at-merlinux.eu |py-dev| > wrote: > > > Hi Simon, > > > > On Thu, Apr 23, 2009 at 15:05 +1000, Simon wrote: > >> The core problem here appears to be that when the receive thread > >> exits > >> (because the ssh session has terminated), the main thread which is > >> running the remote_exec is effectively orphaned. If the users remote > >> exec task doesn't explicitly handle this, the main thread will stay > >> open > >> indefinitely. > >> > >> The attached patch corrects this by running the receive thread as the > >> main thread and the remote_exec as a child daemon thread. > > > > thanks for your investigations and patch! > > > > When I run your example script with your patch > > and immediately do a "ps aux" afterwards > > i still see the subprocess but it dies > > after around 10 seconds. Do you also see this > > behaviour? > Yes that is correct. When the main thread which handles communications > starts to shutdown, it will signal the child thread running the exec > by pushing a None into the queue. Then the main thread will attempt to > join the child thread with a 10 second timeout. If the child thread > reacts and closed cleanly, then it will shutdown and the main thread > can complete immediately. If the child thread fails to shutdown > cleanly within 10 seconds, the main thread will exit and the > daemonised child thread will be cleaned up by the OS. > > The 10 second delay is somewhat arbitrary, we put it in to allow the > thread some time to shutdown cleanly (in the event that it hasn't > actually hung and is just a little busy). > > > > I'd like py.execnet remain able to remotely invoke > > GUI applications or other apps which need the main > > thread for their event loop. So I am wondering if there is a > > different way to achieve the same result of a clean shutdown. > Do you mean that you want the applications to outlive the execnet > connection? If that is so, it should be trivial to make the > daemonisation optional which would mean that the exec process could > continue on beyond comms shutdown.
i meant that GUI threads need the main thread to run their event loop. With your patch execution will always run in sub threads on the remote side. The issue you mention is another one. Are you aware of the gateway.remote_init_threads() method, btw? It will run all subsequent execution in sub daemon threads on the remote side. This should effectively do the same as your patch. I'd be curious if that solves your problem. Independently, we probably need to parametrize gateway.exit(). If exit() is triggered implicitely by process exit or garbage collection it should immediately kill the remote process including all its execution threads, i.e. with a zero timeout. If one wants to be more careful with remote execution threads one would need to call exit() with a timeout paramter or "no timeout" which would keep the process alive until all execution finishes. Does this make sense to you? Needs to work for both the main-thread and multi-exec-thread configuration, of course. cheers, holger > Simon > > > I just created a branch here: > > > > http://bitbucket.org/hpk42/py-shutdown/ > > > > and intend to first to write some tests. > > Please feel free to follow and contribute. > > > > cheers, > > holger > > > > > >> Simon > >> > >> On 21/04/2009, at 11:25 , Simon 5kycsae02-at-sneakemail.com |py-dev| > >> wrote: > >> > >>> This actually looks like an execnet issue, or maybe I'm not quite > >>> understanding how this should work. > >>> > >>> If I run the following code, I get a bunch of stuff left over. Aside > >>> from the ssh and shell instances, the main problem appears to be the > >>> python exec instance on the remote machine (in this example > >>> 'localhost'). If this process is killed, everything else seems to > >>> sort > >>> itself out. > >>> > >>> Calling the exit() method of the gateway doesn't seem to help. Nor > >>> does calling the close() method on the channel instance. > >>> > >>> ======================================= > >>> cmd = """ > >>> while True: > >>> pass > >>> """ > >>> > >>> import py > >>> g=py.execnet.SshGateway("localhost") > >>> c=g.remote_exec(cmd) > >>> ======================================= > >>> > >>> Simon > >> > > > > > >> > > > >> _______________________________________________ > >> py-dev mailing list > >> py-dev@codespeak.net > >> http://codespeak.net/mailman/listinfo/py-dev > > > > > > -- > > Metaprogramming, Python, Testing: http://tetamap.wordpress.com > > Python, PyPy, pytest contracting: http://merlinux.eu > > _______________________________________________ > py-dev mailing list > py-dev@codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu _______________________________________________ py-dev mailing list py-dev@codespeak.net http://codespeak.net/mailman/listinfo/py-dev
