-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 15/01/16 13:16, Matthew Allen wrote:
> Yeah in hindsight it wasn't the best. I was trying to do:
>
> while still running: p.communicate etc
>
> Poll is not the right method to call.
It especially is the wrong method in your scenario since you want to
dynamically read and write from the subprocess. You'll want
stdin/out/err all to be subprocess.PIPE.
You'll possibly get lucky with the SQLite shell, but it is a *very*
good idea to do the reading of their stdout and writing to their stdin
in different threads. The reason is that many of these tools have an
loop that looks like this:
repeat:
- write prompt to stdout
- read a command from stdin
- write output to stdout, errors to stderr
If the output is bigger than the pipe buffer size, then the "write
output" step blocks until there is space in the pipe, and will do so
before reading the next command. Remember that you don't know when
the output is done - in theory you could try to detect the prompt and
hope that something similar is not in the data, but that is brittle.
If you look at the implementation of the communicate method, you'll
see it addresses this issue by using multiple threads (~one per pipe
of interest).
Since you are using Python 2, another issue you need to be aware of is
that the subprocess module is buggy when your python code is
multi-threaded. This issue affects non-Windows systems (eg Linux and
Mac) and you can end up with the wrong child process being accessed!
It isn't theoretical - happened to me two weeks ago and was a pain to
realise what had happened and debug. The good news is Google have a fix:
https://github.com/google/python-subprocess32
But finally I'd recommend you just avoid all this subprocess stuff.
SQLite will quite happily take commands from a script. Consequently
you can create a script file like the following and have SQLite
execute it (command line parameter):
.bail on
.open c:\path\to\your\db
.output c:\...\tmp\dump.sql
.dump
.exit
You can use the tempfile module to get a temporary filename.
The script approach avoids many text vs binary, quoting, encoding,
testing and other issues.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAlaajaQACgkQmOOfHg372QQ68QCeJuZzafMJMAju3F7twPbyToAM
XOcAnic84HeEtyTIg/u0r7gbRHt/x/+o
=Xyid
-----END PGP SIGNATURE-----
