Tomer, I'm building a distributed test harness that needs to do the following: 1) Runs a "client" controller script which communicates with remote hosts and 2) The remote hosts should not accept requests to start new jobs, once the controller has "started" them. The remote hosts should be able to return a status when queried no matter what state they're in e.g. "busy" or "free". 3) On the remote hosts, I want to run some commands synchronously from the controller but others should run asynchronously e.g. host 1 would run command 1 and block until it can return the result to the client, then run command 2 without thblocking and respond that command 2 has started. The controller would then be able to poll the remote host later to see if/when command 2 finished. 4) I'll be executing shell commands on the remote hosts with subprocess and I need a robust way to kill process trees, especially on Windows (my code needs to be cross-platform). I think the Google library psutil will help out nicely with that, it looks good thus far. I'll need a way for the controller to stop the remote code prematurely. So I'm thinking that the controller should also be able to issue a kill command to any set of remote hosts.
That's my basic sketch/set of requirements. Eventually, I might try to build a web UI for all of this but for now, it will be purely command line driven. Rpyc seems like the perfect fit for the remote communication. Do you have any ideas of a good starting point for me? I'm thinking that I can use some combination of multiprocessing/threading/subprocess to have a listener thread/process on the servers which processes the messages from the controller and a separate thread/process (or set of threads/processes) to actually execute the commands. The subprocess module offers plenty of options for running commands with and without waiting. I don't think I need to use Rpyc's threading mechanism but I could be wrong. I'm also not keen on using threading as I do want to bypass the GIL. Some of the testing I'll be doing needs to be as parallel as possible. Anyway, I appreciate any help/advice you can offer. Thanks for building this library too, it's great. Matt
