Re: [Tutor] subprocess.call list vs. str argument

[Albert-Jan Roskam]

 From: Oscar Benjamin oscar.j.benja...@gmail.com
To: Albert-Jan Roskam fo...@yahoo.com 
Cc: Dave Angel da...@davea.name; eryksun eryk...@gmail.com; 
Tutor@python.org tutor@python.org 
Sent: Wednesday, February 26, 2014 3:38 PM
Subject: Re: [Tutor] subprocess.call list vs. str argument
 

On 26 February 2014 08:50, Albert-Jan Roskam fo...@yahoo.com wrote:


 Yesterday evening (it was *late* so forgive me if I wrong) I realized that 
 part of my confusion was also caused by the fact that I ran my code in Idle.
 If I called subprocess.call with a list argument, it  returned code 0 
 (success) but the generated sphinx code was not the same as when I ran the 
 program from the terminal. I concluded that this is some weird interaction 
 between Idle (which may also run in a subprocess??) and my own program. I 
 also had no problems when I ran (in the terminal): python -c import 
 subprocess; subprocess.call(['sphinx-apidoc'...(etc)])

 I was surprised that the list elements of the argument for subprocess.call 
 *have to be* separate tokens, e.g. (earlier in one of Danny's replies): a 
 token (list element) '-f -F' won't work, it has to be two separate 
 elements/tokens: '-f', '-F'. Apparently, subprocess.call is more than just a 
 convenience function that  .joins the list and escapes the resulting 
 string.

At the operating system level there is no command line as such on
posix (Linux, Mac etc.). The command to run a program is a string
giving the name of the program and you can also pass a list of strings
to the program as arguments. The command line is something that
happens in the shell (e.g. bash). Whereas Python would use quotes and
commas to separate strings in a list of strings the shell just assumes
that whitespace separates strings.

So in Python you would write
    ['qwe', 'asd', 'zxc zxc']
and it's clear that you have a list of three strings. In the shell
(e.g. bash) you would type
    qwe asd zxc zxc
and the shell understands that as three separate strings. The quotes
are needed to distinguish it from 4 strings because shell syntax
assumes that spaces separate strings.

So the idea of the command line as a single string that must be
split on whitespace is just something that happens in the shell. It is
supposed to be a convenient way of typing commands in the interactive
shell i.e. it would slow me down if I has to put in square brackets
quotes and commas every time I wanted to type a command in the shell
like ['ls', '-l', '~/mystuff']. But this convenience comes at a price
in that it becomes hard to handle strings that have spaces in them.
I've never really figured out how to write a non-trivial shell script
that can properly handle whitespace (it's easier to just avoid putting
spaces in filenames - or write the script in Python).

So really I think it's a good thing that Python lets you clearly
delineate each string in the argument list:
    subprocess.call(['ls', '-l', foldername])
without needing to worry about whether or not foldername contains spaces.


Thank you everybody for all your replies! I will use a list from now on. Also 
good to know that this works a little differently under Windows. 


Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Albert-Jan Roskam]

Regards,

Albert-Jan



~~

All right, but apart from the sanitation, the medicine, education, wine, public 
order, irrigation, roads, a 

fresh water system, and public health, what have the Romans ever done for us?

~~


On Tue, 2/25/14, eryksun eryk...@gmail.com wrote:

 Subject: Re: [Tutor] subprocess.call list vs. str argument
 To: Dave Angel da...@davea.name
 Cc: tutor@python.org
 Date: Tuesday, February 25, 2014, 11:30 PM
 
 On Tue, Feb 25, 2014 at 4:54 PM, Dave
 Angel da...@davea.name
 wrote:
  CreateProcess has its own design bobbles as well. For
 example,  if
  you forget to put quotes around the program name, 
 it will
  happily try to add .exe to *multiple* places in the
 hopes that
  one of them will work.
 
  Adding a file c:\program.exe to a system will blow up
 lots of
  programs that were working by mistake for years.
 
 snip

Yesterday evening (it was *late* so forgive me if I wrong) I realized that part 
of my confusion was also caused by the fact that I ran my code in Idle.
If I called subprocess.call with a list argument, it  returned code 0 (success) 
but the generated sphinx code was not the same as when I ran the program from 
the terminal. I concluded that this is some weird interaction between Idle 
(which may also run in a subprocess??) and my own program. I also had no 
problems when I ran (in the terminal): python -c import subprocess; 
subprocess.call(['sphinx-apidoc'...(etc)])

I was surprised that the list elements of the argument for subprocess.call 
*have to be* separate tokens, e.g. (earlier in one of Danny's replies): a token 
(list element) '-f -F' won't work, it has to be two separate elements/tokens: 
'-f', '-F'. Apparently, subprocess.call is more than just a convenience 
function that  .joins the list and escapes the resulting string.

regards,
Albert-Jan




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[eryksun]

On Wed, Feb 26, 2014 at 3:50 AM, Albert-Jan Roskam fo...@yahoo.com wrote:
 On Tue, Feb 25, 2014 at 4:54 PM, Dave Angel da...@davea.name wrote:
 CreateProcess has its own design bobbles as well. For
 example,  if you forget to put quotes around the program
 name, it will happily try to add .exe to *multiple*
 places in the hopes that one of them will work.
 Adding a file c:\program.exe to a system will blow up
 lots of programs that were working by mistake for years.

 Yesterday evening (it was *late* so forgive me if I wrong) I
 realized that part of my confusion was also caused by the fact
 that I ran my code in Idle. If I called subprocess.call with a
 list argument, it returned code 0 (success) but the generated
 sphinx code was not the same as when I ran the program from
 the terminal. I concluded that this is some weird interaction
 between Idle (which may also run in a subprocess??) and my own
 program. I also had no problems when I ran (in the terminal):
 python -c import subprocess;
 subprocess.call(['sphinx-apidoc'...(etc)])

Run IDLE from the terminal to have sphinx-apidoc inherit the terminal
for standard I/O. Then you can see the TTY output. I assume you're
using a POSIX system. I tacked on the bit about Windows CreateProcess
just so you'd be aware of the differences.

I'd flip the quoting around in a POSIX shell: python -c 'import
subprocess; subprocess.call([sphinx-apidoc, ...])'.

The shell expands $ and `cmd` in a double-quoted string:

$ msg=spam
$ echo $msg
spam
$ echo `uname -o`
GNU/Linux

But not in a single-quoted string:

$ echo '$msg'
$msg
$ echo '`uname -o`'
`uname -o`

 I was surprised that the list elements of the argument for
 subprocess.call *have to be* separate tokens, e.g. (earlier in
 one of Danny's replies): a token (list element) '-f -F' won't
 work, it has to be two separate elements/tokens: '-f', '-F'.
 Apparently, subprocess.call is more than just a convenience
 function that  .joins the list and escapes the resulting
 string.

subprocess.call doesn't join the arguments:

def call(*popenargs, **kwargs):
return Popen(*popenargs, **kwargs).wait()

On a POSIX system, Popen._execute_child doesn't join the arguments,
either. It sets executable=args[0] and calls os.execvp(executable,
args) in the forked child process. If executable is a relative path,
os.execvp uses a loop over the paths in the PATH environment variable
to create an absolute path.

In CPython, os.execvp calls posix.execv(path, args), which is written
in C to call the system function execv(const char *path, char *const
argv[]). This replaces the current process image with the executable
file located at the absolute `path`.

To set up the system call, posix.execv transforms the tuple/list of
args into an array of char * pointers of length len(args) + 1. The
final item of the array is the NULL terminator. Any unicode strings
are encoded with the file-system encoding (probably UTF-8).

If the new process image isn't successfully executed (replacing Python
in the process), then OSError is raised. This is in the forked child,
so Popen._execute_child has to pickle the exception and write it back
to the parent process via os.write(errpipe_write,
pickle.dumps(exc_value)). It reads the `data` from the other end of
the pipe in the parent, and if it's non-empty it'll `raise
pickle.loads(data)`. That's the source of the OSError from calling
subprocess.call(cmd), where cmd was the entire command line as a
string.

As to what a program does if you mash two options into a single item
of its argv array, such as '-f -F', assuming it isn't trying to be
clever and expects the command-line to be parsed by a standard POSIX
shell, then '-f -F' won't match any of its known options.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Oscar Benjamin]

On 26 February 2014 08:50, Albert-Jan Roskam fo...@yahoo.com wrote:

 Yesterday evening (it was *late* so forgive me if I wrong) I realized that 
 part of my confusion was also caused by the fact that I ran my code in Idle.
 If I called subprocess.call with a list argument, it  returned code 0 
 (success) but the generated sphinx code was not the same as when I ran the 
 program from the terminal. I concluded that this is some weird interaction 
 between Idle (which may also run in a subprocess??) and my own program. I 
 also had no problems when I ran (in the terminal): python -c import 
 subprocess; subprocess.call(['sphinx-apidoc'...(etc)])

 I was surprised that the list elements of the argument for subprocess.call 
 *have to be* separate tokens, e.g. (earlier in one of Danny's replies): a 
 token (list element) '-f -F' won't work, it has to be two separate 
 elements/tokens: '-f', '-F'. Apparently, subprocess.call is more than just a 
 convenience function that  .joins the list and escapes the resulting string.

At the operating system level there is no command line as such on
posix (Linux, Mac etc.). The command to run a program is a string
giving the name of the program and you can also pass a list of strings
to the program as arguments. The command line is something that
happens in the shell (e.g. bash). Whereas Python would use quotes and
commas to separate strings in a list of strings the shell just assumes
that whitespace separates strings.

So in Python you would write
['qwe', 'asd', 'zxc zxc']
and it's clear that you have a list of three strings. In the shell
(e.g. bash) you would type
qwe asd zxc zxc
and the shell understands that as three separate strings. The quotes
are needed to distinguish it from 4 strings because shell syntax
assumes that spaces separate strings.

So the idea of the command line as a single string that must be
split on whitespace is just something that happens in the shell. It is
supposed to be a convenient way of typing commands in the interactive
shell i.e. it would slow me down if I has to put in square brackets
quotes and commas every time I wanted to type a command in the shell
like ['ls', '-l', '~/mystuff']. But this convenience comes at a price
in that it becomes hard to handle strings that have spaces in them.
I've never really figured out how to write a non-trivial shell script
that can properly handle whitespace (it's easier to just avoid putting
spaces in filenames - or write the script in Python).

So really I think it's a good thing that Python lets you clearly
delineate each string in the argument list:
subprocess.call(['ls', '-l', foldername])
without needing to worry about whether or not foldername contains spaces.


Oscar
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Albert-Jan Roskam]

- Original Message -

 From: Danny Yoo d...@hashcollision.org
 To: Peter Otten __pete...@web.de
 Cc: Python Tutor Mailing List tutor@python.org
 Sent: Monday, February 24, 2014 11:01 PM
 Subject: Re: [Tutor] subprocess.call list vs. str argument
 
 Last comment (apologies for being so fragmented!).  I don't know why
 the literal strings there are raw there.  Altogether, I'd expect:
 
 #
 cmd1 = ['sphinx-apidoc',
        '-f',
        '-F',
        '-H', title,
        '-A', author,
        '-V', version,
        '-o', output_dir,
        input_dir]
 retcode = subprocess.call(cmd1, shell=False)
 
 #


Hi Danny, Peter,

Thanks for helping me. Danny, your example works indeed and it is also very 
readable. And now I do not need to use shell=True anymore. The raw strings were 
a leftover from earlier attempts that contained the entire path (not even 
relevant under Linux, but I still did it). Here is why I used shell=True 
before. Is it related to the file paths?

    cmd = (r'sphinx-apidoc '
   r'-f -F '
   r'-H %(title)s '
   r'-A %(author)s '
   r'-V %(version)s '
   r'-o %(output_dir)s %(input_dir)s') % locals()
    retcode = subprocess.call(cmd)
Traceback (most recent call last):
  File /home/albertjan/Downloads/documenter.py, line 188, in module
    rst_to_html(input_dir, output_dir, title=Test, author=getpass.getuser(), 
version=1.0.0)
  File /home/albertjan/Downloads/documenter.py, line 118, in rst_to_html
    retcode = subprocess.call(cmd)
  File /usr/lib/python2.7/subprocess.py, line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File /usr/lib/python2.7/subprocess.py, line 679, in __init__
    errread, errwrite)
  File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

regards,
Albert-Jan
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Dave Angel]

 Albert-Jan Roskam fo...@yahoo.com Wrote in message:
 
 
 - Original Message -
 
 From: Danny Yoo d...@hashcollision.org
 To: Peter Otten __pete...@web.de
 Cc: Python Tutor Mailing List tutor@python.org
 Sent: Monday, February 24, 2014 11:01 PM
 Subject: Re: [Tutor] subprocess.call list vs. str argument
 
 Last comment (apologies for being so fragmented!).  I don't know why
 the literal strings there are raw there.  Altogether, I'd expect:
 
 #
 cmd1 = ['sphinx-apidoc',
'-f',
'-F',
'-H', title,
'-A', author,
'-V', version,
'-o', output_dir,
input_dir]
 retcode = subprocess.call(cmd1, shell=False)
 
 #
 
 
 Hi Danny, Peter,
 
 Thanks for helping me. Danny, your example works indeed and it is also very 
 readable. And now I do not need to use shell=True anymore. The raw strings 
 were a leftover from earlier attempts that contained the entire path (not 
 even relevant under Linux, but I still did it). Here is why I used 
 shell=True before. Is it related to the file paths?
 
 cmd = (r'sphinx-apidoc '
r'-f -F '
r'-H %(title)s '
r'-A %(author)s '
r'-V %(version)s '
r'-o %(output_dir)s %(input_dir)s') % locals()
 retcode = subprocess.call(cmd)
 Traceback (most recent call last):
   File /home/albertjan/Downloads/documenter.py, line 188, in module
 rst_to_html(input_dir, output_dir, title=Test, 
 author=getpass.getuser(), version=1.0.0)
   File /home/albertjan/Downloads/documenter.py, line 118, in rst_to_html
 retcode = subprocess.call(cmd)
   File /usr/lib/python2.7/subprocess.py, line 493, in call
 return Popen(*popenargs, **kwargs).wait()
   File /usr/lib/python2.7/subprocess.py, line 679, in __init__
 errread, errwrite)
   File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child
 raise child_exception
 OSError: [Errno 2] No such file or directory
 

Here you are feeding a string to the call function,  while Danny
 was using a list.  With a string,  you must use shell=True, so
 the shell can turn it into a list. Usually much easier to use a
 list in the first place. 

-- 
DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[eryksun]

On Tue, Feb 25, 2014 at 2:52 PM, Albert-Jan Roskam fo...@yahoo.com wrote:
 Here is why I used shell=True before. Is it related to the
 file paths?

 cmd = (r'sphinx-apidoc '
r'-f -F '
r'-H %(title)s '
r'-A %(author)s '
r'-V %(version)s '
r'-o %(output_dir)s %(input_dir)s') % locals()
 retcode = subprocess.call(cmd)

Take a look at the code in subprocess._execute_child for POSIX platforms:

if isinstance(args, types.StringTypes):
args = [args]
else:
args = list(args)

if shell:
args = [/bin/sh, -c] + args
if executable:
args[0] = executable

if executable is None:
executable = args[0]

It calls os.execvp in the child process, for which you want the
executable file argument to be 'sphinx-apidoc'. But without
shell=True, you can see that Popen uses your entire cmd string as the
executable.

FYI, in Windows the situation is different. CreateProcess takes a
string argument, so the setup code changes to the following:

if not isinstance(args, types.StringTypes):
args = list2cmdline(args)

# Process startup details
if startupinfo is None:
startupinfo = STARTUPINFO()
if None not in (p2cread, c2pwrite, errwrite):
startupinfo.dwFlags |= _subprocess.STARTF_USESTDHANDLES
startupinfo.hStdInput = p2cread
startupinfo.hStdOutput = c2pwrite
startupinfo.hStdError = errwrite

if shell:
startupinfo.dwFlags |= _subprocess.STARTF_USESHOWWINDOW
startupinfo.wShowWindow = _subprocess.SW_HIDE
comspec = os.environ.get(COMSPEC, cmd.exe)
args = '{} /c {}'.format (comspec, args)

It's fine to use a string for args in Windows, and you may need to if
the program parses the command line differently than list2cmdline,
which escapes the arguments according to the rules used by Microsoft's
CRT.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Danny Yoo]

See the comment about 'args' in:

http://docs.python.org/2/library/subprocess.html#frequently-used-arguments

which says:

args is required for all calls and should be a string, or a
sequence of program arguments. Providing a sequence of arguments is
generally preferred, as it allows the module to take care of any
required escaping and quoting of arguments (e.g. to permit spaces in
file names). If passing a single string, either shell must be True
(see below) or else the string must simply name the program to be
executed without specifying any arguments.


This is an unfortunate pain-point in the subprocess API.  Personally,
I think that having subprocess.call optionally accept a single string
is a wrong API decision because it causes so much confusion.   You can
tell there's a schizophrenia in that part of the documentation: it
says something to the effect of, If you want to use a single string,
you have to use shell=True.  And a few paragraphs later, it says,
But don't use shell=True!

I would strongly suggest not trying to use the single string interface
to subprocess: it's too easy to get wrong.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Dave Angel]

 eryksun eryk...@gmail.com Wrote in message:

 
 FYI, in Windows the situation is different. CreateProcess takes a
 string argument, so the setup code changes to the following:
 

 
 It's fine to use a string for args in Windows, and you may need to if
 the program parses the command line differently than list2cmdline,
 which escapes the arguments according to the rules used by Microsoft's
 CRT.

CreateProcess has its own design bobbles as well. For example,  if
 you forget to put quotes around the program name,  it will
 happily try to add .exe to *multiple* places in the hopes that
 one of them will work.

Adding a file c:\program.exe to a system will blow up lots of
 programs that were working by mistake for years.


-- 
DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[eryksun]

On Tue, Feb 25, 2014 at 4:54 PM, Dave Angel da...@davea.name wrote:
 CreateProcess has its own design bobbles as well. For example,  if
 you forget to put quotes around the program name,  it will
 happily try to add .exe to *multiple* places in the hopes that
 one of them will work.

 Adding a file c:\program.exe to a system will blow up lots of
 programs that were working by mistake for years.

Yes, using a string requires quoting the path for the executable if
you aren't using lpApplicationName (i.e. Popen's executable). I
noticed for shell=True they aren't using list2cmdline for COMPSPEC;
they just trust it. So I wrote the following test to exemplify the
problem you're talking about:

 import os, subprocess
 open('temp.c', 'w').write(r'''
... #include stdio.h
... int main(int argc, char *argv[]) {
... printf(\n\n*** spam! ***\n\n);
... return 0;
... }''')
107
 os.system('cl temp.c /FeC:\\Program.exe 1nul 21')
0
 os.environ['COMSPEC'] = r'C:\Program Files\doesnt\matter'
 p = subprocess.Popen('dir', shell=True)


*** spam! ***
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] subprocess.call list vs. str argument

[Albert-Jan Roskam]

Hi,

In the code below, cmd1 and cmd2 are equivalent, as in:  .join(cmd1) == cmd2.
But the first example returns a code 2, whereas the second runs successfully.
What is the difference? I prefer using a list as it looks a little cleaner.
Btw, shell=True is needed here.


# Python 2.7.3 (default, Jan  2 2013, 13:56:14) [GCC 4.7.2] on linux2

import subprocess

#1# returns retcode 2 (whatever error that maybe)
title, author, version = title, author, 1.0.0
output_dir, input_dir = '/tmp/input', '/tmp/output'
cmd1 = [r'sphinx-apidoc',
   r'-f -F',
   r'-H', '%s' % title,
   r'-A', '%s' % author,
   r'-V', '%s' % version,
   r'-o', output_dir, input_dir]
retcode = subprocess.call(cmd1, shell=True)
assert not retcode, retcode

#2# returns retcode 0 (succes)
cmd2 = (r'sphinx-apidoc '
   r'-f -F '
   r'-H %(title)s '
   r'-A %(author)s '
   r'-V %(version)s '
   r'-o %(output_dir)s %(input_dir)s') % locals()

retcode = subprocess.call(cmd2, shell=True)
assert not retcode, retcode

# no AssertionError
assert  .join(cmd1) == cmd2



Thanks in advance!
 
Regards,

Albert-Jan




~~

All right, but apart from the sanitation, the medicine, education, wine, public 
order, irrigation, roads, a 

fresh water system, and public health, what have the Romans ever done for us?

~~ 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Danny Yoo]

 cmd1 = [r'sphinx-apidoc',
r'-f -F',


This part looks suspicious.  Are you sure you don't mean:

'-f', '-F'

here?  They need to be separate arguments.



Also, you mention:

 Btw, shell=True is needed here.

Why do you need shell expansion on the arguments?  This can be
dangerous unless you know what you're doing.  See all the
red-backgrounded warnings the subprocess documentation.  For example:

http://docs.python.org/2/library/subprocess.html#frequently-used-arguments
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Peter Otten]

Albert-Jan Roskam wrote:

 Hi,
 
 In the code below, cmd1 and cmd2 are equivalent, as in:  .join(cmd1) ==
 cmd2. But the first example returns a code 2, whereas the second runs
 successfully. What is the difference? I prefer using a list as it looks a
 little cleaner. Btw, shell=True is needed here.
 
 
 # Python 2.7.3 (default, Jan  2 2013, 13:56:14) [GCC 4.7.2] on linux2
 
 import subprocess
 
 #1# returns retcode 2 (whatever error that maybe)
 title, author, version = title, author, 1.0.0
 output_dir, input_dir = '/tmp/input', '/tmp/output'
 cmd1 = [r'sphinx-apidoc',
 r'-f -F',
 r'-H', '%s' % title,

title becomes \title\, i. e. Python puts in an extra effort to have the 
quotes survive the subsequent parsing process of the shell:

 print subprocess.list2cmdline(['title'])
\title\


 r'-A', '%s' % author,
 r'-V', '%s' % version,
 r'-o', output_dir, input_dir]
 retcode = subprocess.call(cmd1, shell=True)
 assert not retcode, retcode
 
 #2# returns retcode 0 (succes)
 cmd2 = (r'sphinx-apidoc '
 r'-f -F '
 r'-H %(title)s '
 r'-A %(author)s '
 r'-V %(version)s '
 r'-o %(output_dir)s %(input_dir)s') % locals()
 
 retcode = subprocess.call(cmd2, shell=True)
 assert not retcode, retcode
 
 # no AssertionError
 assert  .join(cmd1) == cmd2
 
 
 
 Thanks in advance!
 
 Regards,
 
 Albert-Jan
 
 
 
 
 ~~
 
 All right, but apart from the sanitation, the medicine, education, wine,
 public order, irrigation, roads, a
 
 fresh water system, and public health, what have the Romans ever done for
 us?
 
 ~~
 ___
 Tutor maillist  -  Tutor@python.org
 To unsubscribe or change subscription options:
 https://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Peter Otten]

Peter Otten wrote:

 r'-f -F',
 r'-H', '%s' % title,
 
 title becomes \title\, i. e. Python puts in an extra effort to have
 the quotes survive the subsequent parsing process of the shell:
 
 print subprocess.list2cmdline(['title'])
 \title\

Forget that :( 

Danny spotted the real problem.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Danny Yoo]

There are a few issues there.  I'd also recommend not trying to
shell-quote these manually,

# in the argument list of os.subprocess:
r'-H', '%s' % title,
r'-A', '%s' % author,
r'-V', '%s' % version,


Rather, just do the simpler thing:

r'-H', title,
r'-A', author,
r'-V', version,

in conjunction with passing the shell=False keyword argument.  Don't
escape.  Just pass the arguments as is.


As far as I can tell, trying to do shell escaping is not only
unnecessary here, but doing without it makes the code cleaner safer.
Maybe there's another reason why Albert-Jan's situation is different
enough that shell=True is necessary, but the default situation
should be to avoid it.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess.call list vs. str argument

[Danny Yoo]

Last comment (apologies for being so fragmented!).  I don't know why
the literal strings there are raw there.  Altogether, I'd expect:

#
cmd1 = ['sphinx-apidoc',
   '-f',
   '-F',
   '-H', title,
   '-A', author,
   '-V', version,
   '-o', output_dir,
   input_dir]
retcode = subprocess.call(cmd1, shell=False)
#
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor