Bumpity bump :) Is there any interest in having asynchronous parallel SSH connections in Ansible?
The use case is tasks in parallel over SSH on a number of servers which is currently comparatively slow and very resource intensive, cpu usage and system load wise. On Monday, June 6, 2016 at 10:58:35 AM UTC+1, Dan wrote: > > Hello dev list, > > Many thanks for your efforts on ansible, very useful tool. > > Have a new feature I'd like to work on that would like to float by you for > comments and if it sounds like a good feature to implement before > submitting a PR. > > In short, would like to add the ability to use the ParallelSSH client > library <https://github.com/pkittenis/parallel-ssh> to run SSH commands > via ansible. The library uses paramiko and gevent to provide an > asynchronous parallel SSH client python library. > > This would greatly benefit ansible when running multiple commands in > parallel. The current parallel implementation uses multiprocessing and > increases system load on the host running ansible linearly as # of threads > increases. > > Benefits of using the ParallelSSH library: > > * Asynchronous parallel SSH commands > * No system load induced on the host running it, regardless of # of > parallel connections > * Persistent connections by default, one connection per host > * Stdout/stderr and exit code retrieval built in > * Configurable parallelism level > * Native proxying/tunneling support, also asynchronous. No ProxyCommand, > no subprocess per tunnel > * Native SFTP support > * Pure python library, no cmd line tools used > * No openssh or other ssh cmd line tool dependencies > * Posix systems and windows compatible > > Effect on ansible: > * Uses gevent's monkey patching to make paramiko asynchronous. Monkey > patching of subprocess in particular will also affect ansible's use of > subprocess, though it should be compatible. > * gevent library would be a new requirement for ansible > > > Is this of interest? If so, where do you think it should go, a new > connection type perhaps or overriding existing paramiko implementation? I > would think overriding paramiko implementation to use the library would > make most sense. As the library itself uses paramiko it is already > compatible. > > As a bonus if use of paramiko is overridden, most of the code in > paramiko_ssh.py > <https://github.com/pkittenis/ansible/blob/devel/lib/ansible/plugins/connection/paramiko_ssh.py> > > can go as the client implementation lives entirely in the library. > > Would argue that the subprocess use of the ssh binary in ssh.py should be > overridden as well, at least for parallel commands, but given backwards > compatibility requirements doubt that would be feasible. > > Comments appreciated, thanks. > > - Dan > > -- You received this message because you are subscribed to the Google Groups "Ansible Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
