-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15/01/16 13:16, Matthew Allen wrote: > Yeah in hindsight it wasn't the best. I was trying to do: > > while still running: p.communicate etc > > Poll is not the right method to call.
It especially is the wrong method in your scenario since you want to dynamically read and write from the subprocess. You'll want stdin/out/err all to be subprocess.PIPE. You'll possibly get lucky with the SQLite shell, but it is a *very* good idea to do the reading of their stdout and writing to their stdin in different threads. The reason is that many of these tools have an loop that looks like this: repeat: - write prompt to stdout - read a command from stdin - write output to stdout, errors to stderr If the output is bigger than the pipe buffer size, then the "write output" step blocks until there is space in the pipe, and will do so before reading the next command. Remember that you don't know when the output is done - in theory you could try to detect the prompt and hope that something similar is not in the data, but that is brittle. If you look at the implementation of the communicate method, you'll see it addresses this issue by using multiple threads (~one per pipe of interest). Since you are using Python 2, another issue you need to be aware of is that the subprocess module is buggy when your python code is multi-threaded. This issue affects non-Windows systems (eg Linux and Mac) and you can end up with the wrong child process being accessed! It isn't theoretical - happened to me two weeks ago and was a pain to realise what had happened and debug. The good news is Google have a fix: https://github.com/google/python-subprocess32 But finally I'd recommend you just avoid all this subprocess stuff. SQLite will quite happily take commands from a script. Consequently you can create a script file like the following and have SQLite execute it (command line parameter): .bail on .open c:\path\to\your\db .output c:\...\tmp\dump.sql .dump .exit You can use the tempfile module to get a temporary filename. The script approach avoids many text vs binary, quoting, encoding, testing and other issues. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlaajaQACgkQmOOfHg372QQ68QCeJuZzafMJMAju3F7twPbyToAM XOcAnic84HeEtyTIg/u0r7gbRHt/x/+o =Xyid -----END PGP SIGNATURE-----