[issue11588] Add "necessarily inclusive" groups to argparse

[Jack Wong]

Change by Jack Wong :


--
nosy: +iforapsy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add "necessarily inclusive" groups to argparse

[Mike Auty]

Change by Mike Auty :


--
nosy: +ikelos

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add "necessarily inclusive" groups to argparse

[Pedro]

Pedro added the comment:

Just voicing my support for this. I was also looking for a solution on 
StackOverflow and ended up here.

--
nosy: +pgacv2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add "necessarily inclusive" groups to argparse

[paul j3]

paul j3 added the comment:

Another issue requesting a 'mutually dependent group'
http://bugs.python.org/issue23298

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add "necessarily inclusive" groups to argparse

[paul j3]

paul j3 added the comment:

So far I've proposed adding a 'hook' at the end of '_parse_known_args', that 
would give the user access to the 'seen_non_default_actions' variable.  This 
function could perform an almost arbitrarily complex set of logical 
co-occurrence tests on this set (or list) of Actions.

The rest of my proposed patches (nested groups, etc) are user interface 
components that attempt make this testing more user-friendly, both in 
specification and usage display.

It just occurred to me that an alternate stop-gap fix is to make 
'seen_non_default_actions' available to the user for his own testing after 
parsing.  Adding it to the method return is not backward compatible.  But it 
could be added as an attribute to parser.

 self._seen_actions = seen_non_default_actions

It would be the first case of giving the parser a memory of past parsing 
actions, but I don't think that's a problem.

Another possibility is to conditionally add it to the 'namespace'. 

 if hasattr(namespace, 'seen_actions'):
setattr(namespace, 'seen_actions', seen_non_default_actions)

The user could initial this attribute with a custom 'Namespace' object or with 
a 'set_defaults' call.

(I'm  proposing to save 'seen_non_default_actions' because in my earlier tests 
that seemed to be more useful than 'seen_actions'.  It's the one used by 
mutually_exclusive_group testing.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add "necessarily inclusive" groups to argparse

[Chris Mayo]

Changes by Chris Mayo :


--
nosy: +cjmayo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

Attached is a patch for 3.5.0 dev that adds UsageGroups.  Now different 'kinds' 
are implemented as subclasses, which are accessed via the registry:

 _XorUsageGroup - replicate action of MutuallyExclusiveGroups
 _AndUsageGroup - an inclusive group
 _OrUsageGroup -  an 'any' group
 _NorUsageGroup - NotOr - also works as Not
 _NandUsageGroup - NotAnd

Open issues:

- what difference should the 'required' parameter make?
- how should 'errors' in nested groups propagate?
- formatting of error messages
- formatting the usage line with these groups
- any additional 'help' display
- framework for custom test and/or subclasses
- documentation, sample scripts, and formal testing

--
Added file: http://bugs.python.org/file36553/usage_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

http://stackoverflow.com/questions/25626109/python-argparse-conditionally-required-arguments

asks about implementing a 'conditionally-required-arguments' case in 
`argparse`.  The post-parsing test is simple enough:

if args.argument and (args.a is None or args.b is None):
# raise argparse error here

I believe the clearest and shortest expression using Groups is:

p = ArgumentParser(formatter_class=UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind='nand', dest='nand1')
g1.add_argument('--arg', metavar='C')
g11 = g1.add_usage_group(kind='nand', dest='nand2')
g11.add_argument('-a')
g11.add_argument('-b')

The usage is (using !() to mark a 'nand' test):

usage: issue25626109.py [-h] !(--arg C  !(-a A  -b B))

This uses a 'nand' group, with a 'not-all' test (False if all its actions are 
present, True otherwise).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

I have developed a UsageGroup class that can implement nested 'inclusive' 
tests.  Using this, the original example in this issue could be coded as 3 
groups, 2 mutually_exclusive and inclusive one.

parser = ArgumentParser(prog='PROG', 
formatter_class=UsageGroupHelpFormatter)

g1 = parser.add_usage_group(dest='FILE or DIR', kind='mxg', required=True)
a_file= g1.add_argument(-o, --outfile, metavar='FILE')

g2 = g1.add_usage_group(dest='DIR and PS', kind='inc')
a_dir = g2.add_argument(-O, --outdir, metavar='DIR')

g3 = g2.add_usage_group(dest='P or S', kind='mxg')
a_pat = g3.add_argument(-p, --outpattern, metavar='PATTERN')
a_suf = g3.add_argument(-s, --outsuffix, metavar='SUFFIX')
# usage: PROG [-h] (-o FILE | (-O DIR  (-p PATTERN | -s SUFFIX)))

UsageGroup is like MutuallyExclusiveGroup, except that:

- A nested group is added to self._group_actions, but its actions are not.  
Those actions are added separately to the parser._actions list.  Thus the 
'_group_actions' list can be a mix of actions and groups.

- Each group has a 'testfn', a function that tests its own '_group_actions' 
against the 'seen_non_default_actions' provided by the parser. These are 
similar to the 'cross_test' functions I developed
earlier.  The 'testfn' for primary level usage groups are registered with 
cross_tests (here renamed 'usage_tests')).  'testfn' for nested groups are 
handled recursively in the containing testfn.

- Normally 'testfn' is chosen based on 'kind' and 'required' parameters. 'kind' 
can implement logical actions like 'xor' (mutually exclusive), 'and' 
(inclusive), 'or' (any), 'not'.  These seem to cover most possibilities, though 
custom tests are possible.

- Each group has several variables defining how it is to be formatted.  The 
basic ones are 'joiner' (e.g. '|^,') and 'parens' (e.g. (),[],{}).  Or a 
totally custom 'usage' string could be used.

- The Usage Formatter is a derivative of the MultiGroupHelpFormatter I wrote 
for issue 10984.  Each group is formatted individually (and possibly 
recursively).

- Existing actions and usage groups can be added to a usage group.  This adds 
to the flexibility of the usage testing, though the formatted usage line could 
easily become unwieldy.

- A 'mxg' UsageGroup effectively replaces a MutuallyExclusiveGroup.

- self._add_container_actions(parent) knows nothing about this class yet.

The attached 'usagegroup.py' file has the core code for this change.  The full 
working code is at

https://github.com/hpaulj/argparse_issues/tree/nested

It incorporates too many other changes (from other bug issues) to post
here as a simple patch file (yet).  I have a number of test scripts, but 
haven't cast those as unittests.  Same goes for documentation.

--
Added file: http://bugs.python.org/file35608/usagegroup.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

http://stackoverflow.com/questions/22929087
A question that could be addressed with this patch(es)

In a subparser:

I have 4 arguments: -g, -wid, -w1, and -w2.

-w1 and -w2 always appear together

-wid and (-w1 -w2) are mutually exclusive, but one or the other is required

-g is optional; if it is not specified only (-w1 -w2) can appear, but not 
-wid

The `g` case does not fit either the inclusive or exclusive group patterns.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

A couple more thoughts on an expanded argument testing mechanism:

- do we need both 'seen_actions' and 'seen_non_default_actions'?  

'seen_actions' is used only to test whether all required actions have been 
seen.  These 2 sets differ in how positionals with '?*' are categorized.  
Positionals like this are always 'seen', even if they just get the default 
value.  But they are not required (the case of a '*' positional without default 
needs to be revisited.)

- If 'seen_non_default_actions' is changed to a list (currently its a set), 
users could test for repeated use on an optional, or even the order of 
arguments.  

- One way to make this testing mechanism more user-friendly is to provide 
convenience functions via a decorator.  

For example the decorator could wrap the 'seen_non_default_actions' argument in 
a 'seen' function.  Such a function could accept either an Action or a 'dest' 
string, it could accept a single Action, or a list of them, etc.  There could 
be other functions like 'count', 'unique', 'mutually_exclusive', 'inclusive', 
etc.

def testwfnc(func):
# decorator to register function and provide 'seen'
name = func.__name__
def wrapped(parser, seen_actions, *args):
def seen(*args):
actions = seen_actions
if isinstance(args[0], str):
actions = [a.dest for a in actions]
if len(args)1:
return [a in actions for a in args]
else:
return args[0] in actions
return func(parser, seen)
parser.register('cross_tests', name, wrapped)
return wrapped

#@testwfnc
def test(parser, seen, *args):
if seen(a_file):
print(seen(a_dir, a_pat, a_suf))
cnt = sum(seen(a_dir, a_pat, a_suf))
if cnt0:
parser.error('FILE cannot have DIR, PATTERN or SUFFIX')
...

The attached script experiments with several versions of decorators.  Some sort 
of testing Class is probably the way to go if we want to provide many 
convenience methods.

--
Added file: http://bugs.python.org/file34285/issue11588_4.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

The addition of a simple decorator to the 'ArgumentParser' class, would 
simplify registering the tests:

def crosstest(self, func):
# decorator to facilitate adding these functions
name = func.__name__
self.register('cross_tests', name, func)

which would be used as:

@parser.crosstest
def pat_or_suf(parser, seen_actions, *args):
if 2==len(seen_actions.intersection([a_pat, a_suf])):
parser.error('only one of PATTERN and SUFFIX allowed')

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

This is an example of using 'patch_w_mxg2.diff' to handle the inclusive group 
case proposed by the OP.

Since 'seen_non_default_actions' (and 'seen_actions') is a set of 'Actions', it 
is convenient to use 'set' methods with pointers to the actions that a 
collected during setup.  Tests could also be done with the 'dest' or other 
action attributes.

In this example I wrote 3 simple tests corresponding to the 3 proposed groups, 
but they could also have been written as one test.

a_file= parser.add_argument(-o, --outfile, metavar='FILE')
a_dir = parser.add_argument(-O, --outdir, metavar='DIR')
a_pat = parser.add_argument(-p, --outpattern, metavar='PATTERN')
a_suf = parser.add_argument(-s, --outsuffix, metavar='SUFFIX')
...
def dir_inclusive(parser, seen_actions, *args):
if a_dir in seen_actions:
if 0==len(seen_actions.intersection([a_pat, a_suf])):
parser.error('DIR requires PATTERN or SUFFIX')
parser.register('cross_tests', 'dir_inclusive', dir_inclusive)
...

In theory tests like this could be generated from groups as proposed by the OP. 
There is one case in 'test_argparse.py' where a mutually_exclusive_group is 
nested in an argument_group.  But the current groups do not implement nesting.  
A (plain) argument_group does not share its '_group_actions' list with its 
'container'.  A mutually_exclusive_group shares its '_group_actions' but the 
result is a flat list (no nesting).

For now I think it is more useful to give users tools to write custom 
'cross_tests' than to generalize the 'group' classes.

--
Added file: http://bugs.python.org/file34193/example1.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

This patch uses:

tests = self._registries['cross_tests'].values()

to get a list of functions to run at the end of '_parse_known_args'.

I replaced all of the mutually_exclusive_group tests (3 places) in the 
ArgumentParser with a static function defined in class _MutuallyExclusiveGroup, 
and registered this function.  This refactoring should make it easier to add 
other specialized groups (e.g. inclusive) in the future.

I'm using the _registries because they are already being shared among groups.  

A user can also register a custom testing function.  For example:

def inclusive_test(parser, seen, *args):
# require an argument from one of the 2 groups, g0 and g1
g0seen = seen.intersection(g0._group_actions)
g1seen = seen.intersection(g1._group_actions)
if len(g0seen.union(g1seen))==0:
parser.error('one of the 2 groups is required')
parser.register('cross_tests','inclusive', inclusive_test)

This patched 'argparse.py' runs 'test_argparse.py' without error.

This patch does not include the issue18943 changes, which make setting 
'seen_non_default_actions' more reliable.

--
keywords: +patch
Added file: http://bugs.python.org/file34176/patch_w_mxg2.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

The suggestion in this issue is to add a 'mutually_inclusive_group' mechanism, 
one that would let users specify that certain sets of arguments must occur 
together.  Furthermore there was mention of allowing some sort of nesting.

Modeling it on the mutually_exclusive_group would be straight forward.  But 
should it affect the usage and help display?mutually_exclusive_groups add a 
messy layer to the usage formatting.

The only place such a group would act would be at the end of 
'_parse_known_args', where the current code checks for things like required 
actions (and mxgroups). A test at this point could use 'namespace', 
'seen_actions' and 'seen_non_default_actions' to check whether the required 
group actions were seen.

But the only thing that the argument_group contributes to this test is a list 
of argument names ('dest'?).  Why not provide this list directly?  And what if 
the user wants A to occur together with either B or C, but not both?  Or make 
the inclusivity conditional on the value of A?

Currently users can define argument interactions in a couple of ways.  They can 
define custom Actions. In test_argparse.py there's a custom Actions test that 
does something like this (using '--spam' and 'badger').  But tests in Actions 
depend on the order in which arguments are given.

An alternative is to test for interactions of arguments after `parse_args`.  
However the only information that the user has at this point is the args 
namespace.  Reliably distinguishing between non-occurrence of arguments and 
default values can be difficult.

I am proposing 'cross_test' mechanism that would give the user access to the 
'seen_actions' and 'seen_non_default_actions' sets that 
'mutually_exclusive_groups' use.  Specifically an optional function can be 
called at the end of '_parse_known_args' that has access to these sets as well 
as the parser and the namespace.

The core of the change would be adding

cross_test = getattr(self, 'cross_test', None)
if cross_test:
cross_test(self, namespace, extras, seen_actions, 
seen_non_default_actions)

at the end of 'parser._parse_known_args'.  In addition 'cross_test' (or some 
other name) could be added to the 'ArgumentParser.__init__' arguments.

The feature could be used by defining such a 'cross_test' function and adding 
it to the parser (either instance or subclass)

def foobar(self, namespace, extras, seen_actions, seen_non_default_actions):
...
(raise self.error(...))

parser.cross_test = foobar

The patch proposed http://bugs.python.org/issue18943 should be included
with any changes here since it refines the setting of 
'seen_non_default_actions'.

I am working on tests and examples of such functionality.

--
nosy: +paul.j3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[paul j3]

paul j3 added the comment:

Regarding a usage line like:

(-o FILE | (-O DIR  (-p PATTERN | -s SUFFIX))

The simplest option is to just a custom written 'usage' parameter.

With the existing HelpFormatter, a nested grouping like this is next to 
impossible.  It formats the arguments (e.g.'-O DIR'), interleaves the group 
symbols, and then trims out the excess spaces and symbols.

http://bugs.python.org/issue10984  is a request to allow overlapping 
mutually_exclusive_groups.  It loops on the groups, formatting each.  It would 
be easier with that to format several different types of groups, and to handle 
nested ones.

What would it take to convert a usage string like that into a logical 
expression that tests for the proper occurrence (or non-occurrence) of the 
various arguments.  It might, for example be converted to

exc(file, inc(dir, exc(pattern, suffix)))

where 'exc' and 'inc' are exclusive and inclusive tests, and 'file','dir' etc 
are booleans.  And what would be the error message(s) if this expression fails?

I can imagine a factory function that would take usage line (or other 
expression of groupings), and produce a function that would implement a 
cross_test (as outlined in my previous post).  It would be, in effect, a 
micro-language compiler.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[Manveru]

Manveru manv...@manveru.pl added the comment:

I am subscribing to this idea as I've just fall into such use case where I need 
it. I would like to submit a patch, but I still have difficulties to understand 
argparse code not much spare time to spent on this.

--
nosy: +manveru

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[Xuanji Li]

Changes by Xuanji Li xua...@gmail.com:


--
nosy: +xuanji

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[Steven Bethard]

Steven Bethard steven.beth...@gmail.com added the comment:

I think this is a great suggestion. Care to work on a patch?

--
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[John Didion]

New submission from John Didion johndid...@gmail.com:

Just as some options are mutually exclusive, there are others that are 
necessarily inclusive, i.e. all or nothing. I propose the addition of 
ArgumentParser.add_necessarily_inclusive_group(required=True).

This also means that argparse will need to support nested groups. For example, 
if I want to set up options such that the user has to provide an output file OR 
(an output directory AND (an output file pattern OR an output file extension)):

output_group = parser.add_mutually_exclusive_group(required=True)
output_group.add_argument(-o, --outfile)
outdir_group = output_group.add_necessarily_inclusive_group()
outdir_group.add_argument(-O, --outdir)
outfile_group = outdir_group.add_mutually_exclusive_group(required=True)
outfile_group.add_argument(-p, --outpattern)
outfile_group.add_argument(-s, --outsuffix)

The usage should then look like:

(-o FILE | (-O DIR  (-p PATTERN | -s SUFFIX))

--
messages: 131262
nosy: John.Didion
priority: normal
severity: normal
status: open
title: Add necessarily inclusive groups to argparse
type: feature request
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[SilentGhost]

Changes by SilentGhost ghost@gmail.com:


--
nosy: +bethard

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11588] Add necessarily inclusive groups to argparse

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
versions: +Python 3.3 -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11588
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com