On Wed, Aug 13, 2014 at 12:17 AM, Douglas A. Augusto <daaugu...@gmail.com> wrote:
> Firstly, thank you all for contributing to the development of GNU Parallel, it > is a very useful tool. Thanks. If you like GNU Parallel: * (Re-)walk through the tutorial if you have not done so in the past year (http://www.gnu.org/software/parallel/parallel_tutorial.html) * Give a demo at your local user group/team/colleagues * Post the intro videos and tutorial on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists * Request or write a review for your favourite blog or magazine * Invite me for your next conference If you use GNU Parallel for research: * Please ask your researchers to cite GNU Parallel in their publications (use --bibtex) If GNU Parallel saves you money: * (Have your company) donate to FSF https://my.fsf.org/donate/ > Suppose the following scenario. There are lots of parallel jobs running > locally > and remotely, but in the meantime additional remote servers become available. > It would be nice if those new servers could be included to the pool of compute > servers without restarting the current execution. > > So, instead of statically defining the servers (via sshlogin or sshloginfile), > I am wondering if it is possible to include/exclude servers on-the-fly. This > feature would be analogous to the way in which the load can be dynamically > configured by simply changing a file ("procfile"). --sshloginfile already takes a file, so it will be natural to re-read that. Probably using this method: When a job finishes, and it is more than 1 second since we checked last: Check if the file has changed modification time. If yes: re-read it. It ought to be fairly easy to _add_ new servers. It will probably screw up the timings in --eta/--bar, but that is hardly a serious problem. Removing is, however, a completely different ballgame: What do you do about jobs currently running on the servers? Also there is no infrastructure to say: Don't start new jobs on this server and remove it when the last job completes. The easiest is probably to add a 'dont start new jobs' flag to the server-object, and leave the data structure in place. It will, however, cost a few cycles to skip the server every time a new job is started. --filter-hosts does the "removal" by not adding the host in the first place. Now added to the wishlist https://savannah.gnu.org/bugs/index.php?42983 /Ole
