[issue16399] argparse: append action with default list adds to list instead of overriding

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.2, Python 
3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[hai shi]

hai shi  added the comment:

I update the doc of argparse and think this bpo could be closed when PR merged.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[hai shi]

Change by hai shi :


--
pull_requests: +17226
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17793

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[Roy Smith]

Roy Smith  added the comment:

I just got bit by this in Python 3.5.3.

I get why it does this.  I also get why it's impractical to change the behavior 
now.  But, it really isn't the obvious behavior, so it should be documented at 
https://docs.python.org/3.5/library/argparse.html?highlight=argparse#default.

--
nosy: +roysmith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[Evgeny]

Evgeny  added the comment:

You don't need action='append'. 
For desired behavior you can pass action='store' with nargs='*'.
I think it's a simplest workaround.

--
nosy: +gun146

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[paul j3]

paul j3 added the comment:

One thing that this default behavior does is allow us to append values to any 
object, just so long as it has the `append` method.  The default does not have 
to be a standard list.

For example, in another bug/issue someone asked for an `extend` action.  I 
could provide that with `append` and a custom list class

class MyList(list):
def append(self,arg):
if isinstance(arg,list):
self.extend(arg)
else:
super(MyList, self).append(arg)

This just modifies `append` so that it behaves like `extend` when given a list 
argument.

 parser = argparse.ArgumentParser()
 a = parser.add_argument('-f', action='append', nargs='*',default=[])
 args = parser.parse_args('-f 1 2 3 -f 4 5'.split())

produces a nested list:

 In [155]: args
 Out[155]: Namespace(f=[['1', '2', '3'], ['4', '5']])

but if I change the `default`: 

 a.default = MyList([])
 args = parser.parse_args('-f 1 2 3 -f 4 5'.split())

produces a flat list:

 In [159]: args
 Out[159]: Namespace(f=['1', '2', '3', '4', '5'])

I've tested this idea with an `array.array` and `set` subclass.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[paul j3]

paul j3 added the comment:

It may help to know something about how defaults are handled - in general.

`add_argument` and `set_defaults` set the `default` attribute of the Action 
(the object created by `add_argument` to hold all of its information).  The 
default `default` is `None`.

At the start of `parse_args`, a fresh Namespace is created, and all defaults 
are loaded into it (I'm ignoring some details).

The argument strings are then parsed, and individual Actions update the 
Namespace with their values, via their `__call__` method.

At the end of parsing it reviews the Namespace.  Any remaining defaults that 
are strings are evaluated (passed through `type` function that converts a 
commandline string).  The handling of defaults threads a fine line between 
giving you maximum power, and keeping things simple and predictable.

The important thing for this issue is that the defaults are loaded into the 
Namespace at the start of parsing.

The `append` call fetches the value from the Namespace, replaces it with `[]` 
if it is None, appends the new value(s), and puts it back on the Namespace.  
The first `--foo` append is handled in just the same way as the 2nd and third 
(fetch, append, and put back).  The first can't tell that the list it fetches 
from the namespace came from the `default` as opposed to a previous `append`.  

The `__call__` for `append` was intentionally kept simple, and predictable.  As 
I demonstrated earlier it is possible to write an `append` that checks the 
namespace value against some default, and does something different.  But that 
is more complicated.

The simplest alternative to this behavior is to leave the default as None.  If 
after parsing the value is still None, put the desired list (or any other 
object) there.  

The primary purpose of the parser is to parse the commandline - to figure out 
what the user wants to tell you.  There's nothing wrong with tweaking (and 
checking) the `args` Namespace after parsing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[Michał Klich]

Michał Klich added the comment:

The documentation for argparse still does not mention this behaviour.
I decided to make a patch based no the optparse issue.
Hopefully it is good enough to be merged.

--
keywords: +patch
nosy: +michal.klich
Added file: http://bugs.python.org/file44934/append_to_default.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[Gabriel Devenyi]

Gabriel Devenyi added the comment:

>From what I can tell a workaround for this still isn't documented.

--
nosy: +Gabriel Devenyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[Yclept Nemo]

Yclept Nemo added the comment:

Well that won't work. Example:

import argparse

class TestAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
print(default: {}({})\tdest: {}({}).format(self.default, 
type(self.default), getattr(namespace, self.dest), type(getattr(namespace, 
self.dest
if getattr(namespace, self.dest) is self.default:
print(Replacing with: , values)
setattr(namespace, self.dest, values)
# extra logical code not necessary for testcase

parser = argparse.ArgumentParser()

parser.add_argument\
( -o, --output
, type=int
, action=TestAction
, default=42
)

args = parser.parse_args()

$ ./argparse_test -o 42 -o 100
default: 42(class 'int')  dest: 42(class 'int')
Replacing with:  42
default: 42(class 'int')  dest: 42(class 'int')
Replacing with:  100
100

Use this approach:
class ExtendAction(argparse.Action):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
if not isinstance(self.default, collections.abc.Iterable):
self.default = [self.default]
self.reset_dest = False
def __call__(self, parser, namespace, values, option_string=None):
if not self.reset_dest:
setattr(namespace, self.dest, [])
self.reset_dest = True
getattr(namespace, self.dest).extend(values)

Anyway, this should be properly documented...

--
nosy: +Yclept.Nemo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[paul j3]

paul j3 added the comment:

In my suggestion I used 

if 'current_value is default':

without going into detail.  The use of an 'is' test for integer values is 
tricky, as discussed in http://bugs.python.org/issue18943

int(42) is 42# True
int(257) is 257  # False

As with your 'self.reset_dest', in 18943 I suggested using a boolean flag 
instead of the 'is' test.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[SylvainDe]

SylvainDe added the comment:

As this is likely not to get solved, is there a recommanded way to work around 
this issue ?

Here's what I have done :

  import argparse
  def main():
  Main function
  parser = argparse.ArgumentParser()
  parser.add_argument('--foo', action='append')
  for arg_str in ['--foo 1 --foo 2', '']:
  args = parser.parse_args(arg_str.split())
  if not args.foo:
  args.foo = ['default', 'value']
  print(args)

printing

  Namespace(foo=['1', '2'])
  Namespace(foo=['default', 'value'])

as expected but I wanted to know if there a more argparse-y way to do this. I 
have tried using `set_defaults` without any success.

Also, as pointed out the doc for optparse describes the behavior in a simple 
way : The append action calls the append method on the current value of the 
option. This means that any default value specified must have an append method. 
It also means that if the default value is non-empty, the default elements will 
be present in the parsed value for the option, with any values from the command 
line appended after those default values.

--
nosy: +SylvainDe

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[paul j3]

paul j3 added the comment:

It should be easy to write a subclass of Action, or append Action, that does 
what you want.  It just needs a different `__call__` method.  You just need a 
way of identifying an default that needs to be overwritten as opposed to 
appended to.


def __call__(self, parser, namespace, values, option_string=None):
current_value = getattr(namspace, self.dest)
if 'current_value is default':
setattr(namespace, self.dest, values)
return
else:
# the normal append action
items = _copy.copy(_ensure_value(namespace, self.dest, []))
items.append(values)
setattr(namespace, self.dest, items)

People on StackOverFlow might have other ideas.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[paul j3]

paul j3 added the comment:

The test file, test_argparse.py, has a test case for this:
'class TestOptionalsActionAppendWithDefault'

argument_signatures = [Sig('--baz', action='append', default=['X'])]
successes = [
('--baz a --baz b', NS(baz=['X', 'a', 'b'])),
]

--
nosy: +paul.j3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[Markus Amalthea Magnuson]

Changes by Markus Amalthea Magnuson markus.magnu...@gmail.com:


--
title: argparse: - argparse: append action with default list adds to list 
instead of overriding

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16399] argparse: append action with default list adds to list instead of overriding

[R. David Murray]

R. David Murray added the comment:

This behavior is inherited from optparse.  I think it is more-or-less 
intentional, and in any case it is of long enough standing that I don't think 
we can change it.  We documented it for optparse in another issue, but I don't 
think we made the corresponding improvement to the argparse docs.

--
assignee:  - docs@python
components: +Documentation
nosy: +bethard, docs@python, r.david.murray
versions: +Python 3.2, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16399
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com