[issue18943] argparse: default args in mutually exclusive groups

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee: rhettinger -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Raymond Hettinger]

Change by Raymond Hettinger :


--
pull_requests:  -5593

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee: bethard -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Daniel Himmelstein]

Change by Daniel Himmelstein :


--
pull_requests: +5593
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Vincas Dargis]

Vincas Dargis  added the comment:

On 2017-12-06 20:28, paul j3 wrote:
> The default value is used *if the flag is not provided at all.*
> 
> "nargs='?'" provides a third option, assigning the 'const' value *if the flag 
> is used without an argument*.

This did a "click" in my head. It works now with `nargs='?'` and 
`const='MediaProfile000'` as expected, thanks!

I am really sorry for the noise, due to misunderstanding while reading 
(skipping-throuhg?) Python documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3  added the comment:

That's not how flagged (optionals) arguments work.

The default value is used if the flag is not provided at all.  One of your 
arguments is a 'store_true'.  Its default value if False, which is changed to 
True if the '--device-get-capabilities' flag is provided.

"nargs='?'" provides a third option, assigning the 'const' value if the flag is 
used without an argument.

In any case your problem isn't with a required mutually exclusive group 
(defaults or not).  It has to do with understanding optionals and their 
defaults.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Vincas Dargis]

Vincas Dargis  added the comment:

On 2017-12-06 19:43, paul j3 wrote:
> With one flag but not its argument, I get the error that you display.  That 
> has nothing to do with the grouping.
> 
> 0932:~/mypy/argdev$ python3 issue18943.py --ptz-get-status
> usage: issue18943.py [-h]
>   (--device-get-capabilities | --ptz-absolute-move x y z 
> | --ptz-get-status MEDIA_PROFILE)
> issue18943.py: error: argument --ptz-get-status: expected one argument

In my example I pasted, I had hardcoded arguments:

```
pprint.pprint(parser.parse_args(['--ptz-get-status']))
``

I expected `python myscript.py --ptz-get-status` to work, because default value 
is set.

I do not compute that "With one flag but not its argument", sorry. It has 
default argument set, shoudn't that work?

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3  added the comment:

Did you copy the output right?  Testing your parser:

Without any arguments, I get the exclusive group error - the group is required:

0930:~/mypy/argdev$ python3 issue18943.py 
usage: issue18943.py [-h]
 (--device-get-capabilities | --ptz-absolute-move x y z | 
--ptz-get-status MEDIA_PROFILE)
issue18943.py: error: one of the arguments --device-get-capabilities 
--ptz-absolute-move --ptz-get-status is required

0931:~/mypy/argdev$ python3 --version
Python 3.5.2

With one flag but not its argument, I get the error that you display.  That has 
nothing to do with the grouping.

0932:~/mypy/argdev$ python3 issue18943.py --ptz-get-status
usage: issue18943.py [-h]
 (--device-get-capabilities | --ptz-absolute-move x y z | 
--ptz-get-status MEDIA_PROFILE)
issue18943.py: error: argument --ptz-get-status: expected one argument

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Vincas Dargis]

Vincas Dargis  added the comment:

Any progress with this? I believe it would fix my use case:

```
import argparse
import pprint

parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)

group.add_argument('--device-get-capabilities',
   action='store_true',
   help='Execute GetCapabilities action from ONVIF 
devicemgmt.wsdl')

group.add_argument('--ptz-absolute-move',
   nargs=3,
   metavar=('x', 'y', 'z'),
   help='Execute AbsoluteMove action from ONVIF ptz.wsdl')

group.add_argument('--ptz-get-status',
   metavar='MEDIA_PROFILE',
   default='MediaProfile000',
   help='Execute GetSatus action from ONVIF ptz.wsdl for a 
media profile (default=%(default)s)')

pprint.pprint(parser.parse_args(['--ptz-get-status']))
```

Outputs (using 3.6.3):

```
 python3 ./test-ex-group-with-defult.py 
usage: test-ex-group-with-defult.py [-h]
(--device-get-capabilities | 
--ptz-absolute-move x y z | --ptz-get-status MEDIA_PROFILE)
test-ex-group-with-defult.py: error: argument --ptz-get-status: expected one 
argument
```

Are there know workarounds for this?

--
nosy: +talkless

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

Another manifestation of the complications in handling '?' positionals is in

http://bugs.python.org/issue28734

argparse: successive parsing wipes out nargs=? values

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

Changes by Armin Rigo :


--
nosy:  -arigo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

I haven't downloaded the development distribution to this computer, so can't 
write formal patches at this time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee:  -> bethard
priority: high -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Louie Lu]

Louie Lu added the comment:

paul, will you work on this patch? or I can help this issue, too.

--
nosy: +louielu
type:  -> behavior
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

This came up again, http://bugs.python.org/issue30163

An optional with int type and small integer default.

--
priority: normal -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Wolfgang Maier]

Changes by Wolfgang Maier :


--
nosy: +wolma

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

I need to tweak the last patch so 'using_default' is also set when an 
"nargs='*'" positional is set to the '[]' default.

 if action.default is not None:
 value = action.default
+using_default = True
 else:
 value = arg_strings
+using_default = True  # tweak

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

This patch corrects the handling of `seen_non_default_action` in another case - 
a positional with '?' and `type=int` (or other conversion).

if

parser.add_argument('badger', type=int, nargs='?', default=2) # or '2'

and the original test 'seen_non_default_actions' is:

if argument_values is not action.default

'argument_values' will be an 'int' regardless of the default.  But it will pass 
the 'is' test with the (small) int default but not the string default.

With the patch proposed here, both defaults behave the same - 'badger' will not 
appear in 'seen_non_default_actions' if it did not occur in the 
argument_strings (i.e. match an empty string).

I may add case like this to `test_argparse.py` for this patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

Armin Rigo added the comment:

The patch looks good to me.  It may break existing code, though, as reported on 
https://bugs.pypy.org/issue1595.  I would say that it should only go to trunk.  
We can always fix PyPy (at Python 2.7) in a custom manner, in a "bug-to-bug" 
compatibility mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

This patch uses a narrow criteria - if `_get_values()` sets the value to 
`action.default`, then argument counts as 'not present'.  I am setting a 
`using_default` flag in `_get_values`, and return it for use by `take_action`.

In effect, the only change from previous behavior is that small ints (<257) now 
behave like large ints, strings and other objects.

It removes the nonstandard 'is not action.default' test, and should behave 
consistently across all platforms (including pypy).

--
keywords: +patch
Added file: http://bugs.python.org/file31704/patch_1.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

At the very least the `is not action.default` needs to be changed.  Else where 
in argparse `is` is only used with `None` or constant like `SUPPRESS`.  So 
using it with a user defined parameter is definitely not a good idea.

Possible variations on how `is` behaves across implementations (pypy, 
ironpython) only complicates the issue.  I'm also familiar with a Javascript 
translation of argparse (that uses its `!==` in this context).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

This `argument_values` comes from `_get_values()`.  Most of the time is derived 
from the `argument_strings`.  But in a few cases it is set to `action.default`, 
specifically when the action is an optional postional with an empty 
`argument_strings`.

test_argparse.TestMutuallyExclusiveOptionalAndPositional is such a case.  
`badger` is an optional positional in a mutually exclusive group.  As such it 
can be 'present' without really being there (tricky).  Positionals are always 
processed - otherwise it raises an error.

If this is the case, what we need is a more reliable way of knowing whether 
`_get_values()` is doing this, one that isn't fooled by this small int caching.

We could rewrite the `is not` test as:

if not argument_strings and action.nargs in ['*','?'] and  argument_values 
is action.default:
pass # _get_values() has set: argument_values=action.default
else:
seen_non_default_actions.add(action)
...

is a little better, but still feels like a kludge.  Having `_get_values` return 
a flag that says "I am actually returning action.default" would be clearer, 
but, I think, too big of a change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

Armin Rigo added the comment:

Fwiw I agree with you :-)  I'm just relaying a bug report that originates on 
PyPy (https://bugs.pypy.org/issue1595).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

I should add that defaults with required arguments (or groups?) doesn't make 
much sense.  Still there's nothing in the code that prevents it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

A possibly unintended consequence to this `seen_non_default_actions` testing is 
that default values do not qualify as 'present' when testing for a required 
mutually exclusive group.

p=argparse.ArgumentParser()
g=p.add_mutually_exclusive_group(required=True)
g.add_argument('--foo',default='test')
g.add_argument('--bar',type=int,default=42)
p.parse_args('--bar 42'.split())

raises an `error: one of the arguments --foo --bar is required`

In the original code

p.parse_args('--foo test'.split())

does not raise an error because 'test' does not qualify as default.  But with 
the change I proposed, it does raise the error.

This issue may require adding a `failures_when_required` category to the 
test_argparse.py MEMixin class.  Currently nothing in test_argparse.py tests 
for this issue.

Note that this contrasts with the handling of ordinarily required arguments.

p.add_argument('--baz',type=int,default=42,required=True)

'--baz 42' does not raise an error.  It is 'present' regardless of whether its 
value matches the default or not.

This argues against tightening the `seen_non_default_actions` test.  Because 
the current testing only catches a few defaults (None and small ints) it is 
likely that no user has come across the required group issue.  There might 
actually be fewer compatibility issues if we simply drop the default test (or 
limit it to the case where the default=None).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

Changing the test from

if argument_values is not action.default:

to 

if argument_values is not action.default and \
(action.default is None or argument_values != action.default):

makes the behavior more consistent.  Strings and large ints behave like small 
ints, matching the default and not counting as "present"

Simply using `argument_values != action.default` was not sufficient, since it 
raised errors in existing test cases (such as ones involving Nones).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Antoine Pitrou]

Antoine Pitrou added the comment:

> The patch isn't a good unittest case because it produces an Error, not 
> a Failure.

Please let's not be pedantic about what a "good unittest" is.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

Armin Rigo added the comment:

Getting consistently one behavior or the other would be much better imho; I 
think it's wrong-ish to have the behavior depend uncontrollably on 
implementation details.  But I agree that it's slightly messy to declare which 
of the two possible fixes is the "right" one.  I'm slightly in favor of the 
more permissive solution ("--bar 42" equivalent to no arguments at all if 42 is 
the default) only because the other solution might break someone's existing 
code.  If I had no such backward-compatibility issue in mind, I'd vote for the 
other solution (you can't specify "--bar" with any value, because you already 
specified "--foo").

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
nosy: +bethard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

A further complication on this.  With the arguments I defined in the previous 
post

p.parse_args('--foo test --baz 257'.split())

gives the mutually exclusive error message.  `sys.argv` does the same.

p.parse_args(['--foo', 'test', '--baz', '257'])

does not give an error, because here the 'test' argument string is the same as 
the default 'test'.  So the m_x_g test thinks `--foo' is the default, and does 
not count as an input.

Usually in testing an argparse setup I use the list and split arguments 
interchangeably, but this shows they are not equivalent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[paul j3]

paul j3 added the comment:

The patch isn't a good unittest case because it produces an Error, not a 
Failure.  It does, though, raise a valid question about how a 
Mutually_exclusive_group tests for the use of its arguments.

As you note, argparse does use the `is` test: `argument_values is not 
action.default`.  argument_values is the result of passing an argument_string 
through its 'type' function.

This reworks your test case a bit:

group = parser.add_mutually_exclusive_group()
group.add_argument('--foo', default='test')
group.add_argument('--bar', type=int, default=256)
group.add_argument('--baz', type=int, default=257)

'--foo test --baz 257' will give the `argument --foo: not allowed with argument 
--baz` error message, but '--foo test --baz 256' does not.

So which is right?  Should it complain because 2 exclusive arguments are being 
used?  Or should it be excused from complaining because the values match their 
defaults?

The other issue is whether the values really match the defaults or not.  With 
an `is` test, the `id`s must match. The ids for small integers match all the 
time, while ones >256 differ.

Strings might have the same id or not, depending on how they are created.  If I 
create `x='test'`, and `y='--foo test'.split()[1]`.  `x==y` is True, but `x is 
y` is False.  So '--foo test' argument_value does not match the 'foo.default'.

So large integers (>256) behave like strings when used as defaults in this 
situation.  It's the small integers that have unique ids, and hence don't 
trigger mutually_exclusive_group errors when they should.

This mutually_exclusive_group 'is' test might not be ideal (free from all 
ambiguities), but I'm not sure it needs to be changed.  Maybe there needs to be 
a warning in the docs about mutually_exclusive_groups and defaults other than 
None.

--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

Changes by Armin Rigo :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

Changes by Armin Rigo :


--
keywords:  -patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18943] argparse: default args in mutually exclusive groups

[Armin Rigo]

New submission from Armin Rigo:

In argparse, default arguments have a strange behavior that shows up in 
mutually exclusive groups: specifying explicitly on the command-line an 
argument, but giving it its default value, is sometimes equivalent to not 
specifying the argument at all, and sometimes not.

See the attached test diff: it contains two apparently equivalent pieces of 
code, but one passes and one fails.  The difference is that, in CPython, 
int("42") is 42 but int("4200") is not 4200 (in the sense of the operator "is").

The line that uses "is" in this way is this line in argparse.py (line 1783 in 
2.7 head):

if argument_values is not action.default:

--
files: test_argparse.diff
keywords: patch
messages: 197058
nosy: arigo
priority: normal
severity: normal
status: open
title: argparse: default args in mutually exclusive groups
versions: Python 2.7, Python 3.5
Added file: http://bugs.python.org/file31626/test_argparse.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com