[issue29670] argparse: does not respect required args pre-populated into namespace

[Antonio Valentino]

Change by Antonio Valentino :


--
nosy: +avalentino

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[paul j3]

paul j3 added the comment:

Another pre-existing namespace issue

http://bugs.python.org/issue28734

When positional nargs='?' or '*', the default (or []) overwrites the namespace 
value.  That's because the posiitonals are always 'seen' (by an empty string), 
and `get_values` has special handling.  This action interacts with 
`take_action` and the handling of mutually_exclusive_groups.

--
versions: +Python 3.7 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[paul j3]

paul j3 added the comment:

http://bugs.python.org/issue27859
argparse - subparsers does not retain namespace

may complicate this issue.  Due to changes in http://bugs.python.org/issue9351, 
a user defined namespace is not passed to the subparsers.  They get a fresh 
namespace, which is then copied onto the main namespace.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[paul j3]

paul j3 added the comment:

http://bugs.python.org/issue26394
Have argparse provide ability to require a fallback value be present

is a related issue - getting 'required' values from a config file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[paul j3]

paul j3 added the comment:

One refactoring that I'd like to see is to move all the tests at the end of 
_parse_know_known_args to its caller, parse_known_args.  It would have to 
return more values than the current namespace and extras, in particular the 
see_actions and seen_non_default_actions.

This would have, I think, several benefits:

- it would make easier to customize the testing as proposed by Matthew.  It 
probably would also make it easier to implement the nested-logical-groups that 
I propose in http://bugs.python.org/issue11588.

- It would put the '# Convert action default now instead of doing it before 
parsing arguments' step at the same code level as when the defaults were first 
loaded.

Currently defaults are set at the start of parse_known_args, but evaluated at 
the end of _parse_known_args.  That asymmetry has bugged me for some time.  I'm 
not aware of any bug/issues that have arisen from that, but still it doesn't 
look right.

Last time the 'required' tests were revised, it introduced an error in the 
handling of subparsers.  http://bugs.python.org/issue9253

Subparsers used to be required (default for positionals).  But now they are 
optional unless you add an explicit 'subparsers.required = True' statement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[paul j3]

paul j3 added the comment:

What issue might have changed this behavior?  I'm not aware of any that were 
trying to change either the `seen_actions` and the 'required' tests, not any 
dealing with pre-existing values in Namespace.

The current handling of defaults complicates respecting pre-existing values.  
Defaults are added to the namespace at the start of parsing (provided they 
aren't already present - that respects pre-existing values).  At the end of 
parsing, the same block that tests for required actions also tests if the 
defaults in the Namespace need to be converted with the `type` function.

So the code at the end of _parse_known_args has limited ability to distinguish 
between values in the Namespace that  were preloaded, were loaded as defaults, 
or were loaded during parsing.

I agree that _parse_known_args is complicated and difficult to customize.  But 
it's been like that from the start.  And the handling of defaults is also 
complicated (and made worse by the 'delayed evaluation' change some years ago).

For another bug/issue I'd like to make seen_actions (or 
seen_non_default_actions) available to the user for testing.  But that isn't 
easy in a backward compatible fashion.

I think the best choice is to pre-load non-required Actions only.  Preloading 
should be seen as an alternative way of setting defaults, not a way of getting 
around the 'required' test.  And if that isn't enough, do some of your own 
'required' tests after parsing.

--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[Terry J. Reedy]

Terry J. Reedy added the comment:

Can you test on 3.6?  This might have been fixed and not back-ported.

--
nosy: +bethard, terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29670] argparse: does not respect required args pre-populated into namespace

[Matthew Hall]

New submission from Matthew Hall:

I have some Python code which takes advantage of the ability to prepopulate the 
argparse Namespace before passing it to argparse. This allows me to read 
sensitive settings such as usernames and passwords from DBs, environment 
variables, etc. in addition to the CLI for security so they don't appear as 
process arguments anyone could see.

Some of these scripts have usernames and passwords as required arguments since 
they cannot function without them. Unfortunately you hit this bug when you load 
them into the namespace yourself from some secure place of your choice and then 
pass them in:

args = parser.parse_args(argv, namespace)

The following argparse code which doesn't respect that they were present 
already and throws a fatal error. In addition, the parse_known_args function is 
246 lines of code which uses ugly nested cheater functions, and the 
seen_actions variable is in the stack of the function not a member variable, so 
there is no way you can cleanly override or work around the behavior with a 
subclass, either by replacing the function (too massive) or by fixing up the 
seen_actions variable (can't get to it on the stack from outside).

So, I suggest that this code should to be fixed so that it will respect any 
existing values in the Namespace if they are present:

# make sure all required actions were present, and convert defaults.
for action in self._actions:
if action not in seen_actions:
if action.required:
name = _get_action_name(action)
self.error(_('argument %s is required') % name)

--
components: Library (Lib)
messages: 288667
nosy: mhcptg
priority: normal
severity: normal
status: open
title: argparse: does not respect required args pre-populated into namespace
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com