Hi all,
So cumin is failing to build in Debian because it does some nasty things
during build (namely using $HOME, because of that setup.py develop --user).
I have a beginning of a fix for this in the bugfix-1026005 branch on
salsa. The problem now is that it fails on *other* parts of the test
suite, which wasn't failing before. I suspect it might be because the
code layout is different, but I can't figure out how or why...
Riccardo, Moritz, any ideas on how this could be fixed?
Here's the failing build log:
anarcat@angela:cumin$ dpkg-buildpackage
dpkg-buildpackage: info: source package cumin
dpkg-buildpackage: info: source version 4.1.1-3
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Antoine Beaupré <anar...@debian.org>
dpkg-buildpackage: info: host architecture amd64
dpkg-source --before-build .
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying 0001-import-version-number-from-setuptools_scm.patch
fakeroot debian/rules clean
dh clean --with python3 --buildsystem=pybuild
dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:240: python3.10 setup.py clean
running clean
removing '/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build' (and
everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.10' does not exist -- can't clean it
dh_autoreconf_clean -O--buildsystem=pybuild
dh_clean -O--buildsystem=pybuild
dpkg-source -b .
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: warning: upstream signing key but no upstream tarball signature
dpkg-source: info: building cumin using existing ./cumin_4.1.1.orig.tar.gz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: building cumin in cumin_4.1.1-3.debian.tar.xz
dpkg-source: info: building cumin in cumin_4.1.1-3.dsc
debian/rules build
dh build --with python3 --buildsystem=pybuild
dh_update_autotools_config -O--buildsystem=pybuild
dh_autoreconf -O--buildsystem=pybuild
dh_auto_configure -O--buildsystem=pybuild
I: pybuild base:240: python3.10 setup.py config
running config
dh_auto_build -O--buildsystem=pybuild
I: pybuild base:240: /usr/bin/python3 setup.py build
running build
running build_py
creating /home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
copying cumin/transport.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
copying cumin/color.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
copying cumin/cli.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
copying cumin/__init__.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
copying cumin/grammar.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
copying cumin/query.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin
creating
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/backends
copying cumin/backends/direct.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/backends
copying cumin/backends/openstack.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/backends
copying cumin/backends/puppetdb.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/backends
copying cumin/backends/knownhosts.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/backends
copying cumin/backends/__init__.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/backends
creating
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/transports
copying cumin/transports/clustershell.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/transports
copying cumin/transports/__init__.py ->
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build/cumin/transports
dh_auto_test -O--buildsystem=pybuild
I: pybuild base:240: cd
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build; python3.10 -m
pytest /home/anarcat/dist/cumin/cumin/tests/unit
==================================================================== test
session starts
====================================================================
platform linux -- Python 3.10.8, pytest-7.1.2, pluggy-1.0.0+repack
rootdir: /home/anarcat/dist/cumin, configfile: pytest.ini
plugins: arraydiff-0.5.0, betamax-0.8.1, astropy-header-0.2.2,
remotedata-0.3.3, requests-mock-1.9.3, hypothesis-6.36.0, openfiles-0.5.0,
mock-3.8.2, astropy-0.10.0, cov-4.0.0, doctestplus-0.12.1,
filter-subpackage-0.1.1
collected 416 items
../../../cumin/tests/unit/test_backends.py .
[ 0%]
../../../cumin/tests/unit/test_cli.py ..............................
[ 7%]
../../../cumin/tests/unit/test_color.py .........
[ 9%]
../../../cumin/tests/unit/test_grammar.py F..........
[ 12%]
../../../cumin/tests/unit/test_init.py .............................
[ 19%]
../../../cumin/tests/unit/test_query.py ...F......F....F
[ 23%]
../../../cumin/tests/unit/test_transport.py ....
[ 24%]
../../../cumin/tests/unit/backends/test_direct.py ..F
[ 24%]
../../../cumin/tests/unit/backends/test_grammars.py FF......
[ 26%]
../../../cumin/tests/unit/backends/test_knownhosts.py
......F.F.................
[ 32%]
../../../cumin/tests/unit/backends/test_openstack.py ...........
[ 35%]
../../../cumin/tests/unit/backends/test_puppetdb.py
...............................................................................................
[ 58%]
../../../cumin/tests/unit/transports/test_clustershell.py
.............................................
[ 69%]
../../../cumin/tests/unit/transports/test_init.py
...................................................................................................
[ 93%]
.............................
[100%]
=========================================================================
FAILURES
==========================================================================
____________________________________________________________________
test_valid_strings
_____________________________________________________________________
def test_valid_strings():
"""Run quick pyparsing test over valid grammar strings."""
results = grammar.grammar(REGISTERED_BACKENDS.keys()).runTests(
get_fixture(os.path.join('grammar', 'valid_grammars.txt'),
as_string=True))
> assert results[0]
E assert False
../../../cumin/tests/unit/test_grammar.py:22: AssertionError
------------------------------------------------------------------- Captured
stdout call --------------------------------------------------------------------
# Valid grammars
A:alias_name
[['A:alias_name']]
[0]:
['A:alias_name']
- alias: 'alias_name'
A:alias-name
[['A:alias-name']]
[0]:
['A:alias-name']
- alias: 'alias-name'
A:alias.name
[['A:alias.name']]
[0]:
['A:alias.name']
- alias: 'alias.name'
A:alias+name
[['A:alias+name']]
[0]:
['A:alias+name']
- alias: 'alias+name'
P{puppetdb query (with subgroups) "and even P{}}, if quoted"}
[['P{puppetdb query (with subgroups) "and even P{}}, if quoted"}']]
[0]:
['P{puppetdb query (with subgroups) "and even P{}}, if quoted"}']
- backend: 'P'
- query: 'puppetdb query (with subgroups) "and even P{}}, if quoted"'
D{direct (query) with and or ((subgroups[10-20]))}
[['D{direct (query) with and or ((subgroups[10-20]))}']]
[0]:
['D{direct (query) with and or ((subgroups[10-20]))}']
- backend: 'D'
- query: 'direct (query) with and or ((subgroups[10-20]))'
P{query1} and D{query2}
[['P{query1}'], ['and', 'D{query2}']]
[0]:
['P{query1}']
- backend: 'P'
- query: 'query1'
[1]:
['and', 'D{query2}']
- backend: 'D'
- bool: 'and'
- query: 'query2'
P{query1} and not D{query2}
P{query1} and not D{query2}
^
ParseException: Expected end of text, found 'and' (at char 10), (line:1,
col:11)
FAIL: Expected end of text, found 'and' (at char 10), (line:1, col:11)
P{query1} or D{query1}
[['P{query1}'], ['or', 'D{query1}']]
[0]:
['P{query1}']
- backend: 'P'
- query: 'query1'
[1]:
['or', 'D{query1}']
- backend: 'D'
- bool: 'or'
- query: 'query1'
P{query1} xor D{query1}
[['P{query1}'], ['xor', 'D{query1}']]
[0]:
['P{query1}']
- backend: 'P'
- query: 'query1'
[1]:
['xor', 'D{query1}']
- backend: 'D'
- bool: 'xor'
- query: 'query1'
P{query} and A:alias
[['P{query}'], ['and', 'A:alias']]
[0]:
['P{query}']
- backend: 'P'
- query: 'query'
[1]:
['and', 'A:alias']
- alias: 'alias'
- bool: 'and'
P{query} and not A:alias
P{query} and not A:alias
^
ParseException: Expected end of text, found 'and' (at char 9), (line:1, col:10)
FAIL: Expected end of text, found 'and' (at char 9), (line:1, col:10)
P{query} or A:alias
[['P{query}'], ['or', 'A:alias']]
[0]:
['P{query}']
- backend: 'P'
- query: 'query'
[1]:
['or', 'A:alias']
- alias: 'alias'
- bool: 'or'
P{query} xor A:alias
[['P{query}'], ['xor', 'A:alias']]
[0]:
['P{query}']
- backend: 'P'
- query: 'query'
[1]:
['xor', 'A:alias']
- alias: 'alias'
- bool: 'xor'
A:alias and P{query}
[['A:alias'], ['and', 'P{query}']]
[0]:
['A:alias']
- alias: 'alias'
[1]:
['and', 'P{query}']
- backend: 'P'
- bool: 'and'
- query: 'query'
A:alias and not P{query}
A:alias and not P{query}
^
ParseException: Expected end of text, found 'and' (at char 8), (line:1, col:9)
FAIL: Expected end of text, found 'and' (at char 8), (line:1, col:9)
A:alias or P{query}
[['A:alias'], ['or', 'P{query}']]
[0]:
['A:alias']
- alias: 'alias'
[1]:
['or', 'P{query}']
- backend: 'P'
- bool: 'or'
- query: 'query'
A:alias xor P{query}
[['A:alias'], ['xor', 'P{query}']]
[0]:
['A:alias']
- alias: 'alias'
[1]:
['xor', 'P{query}']
- backend: 'P'
- bool: 'xor'
- query: 'query'
A:alias1 and (P{query1} or D{query2}) and not A:alias2 xor D{query3}
A:alias1 and (P{query1} or D{query2}) and not A:alias2 xor D{query3}
^
ParseException: Expected end of text, found 'and' (at char 38), (line:1,
col:39)
FAIL: Expected end of text, found 'and' (at char 38), (line:1, col:39)
A:alias1 and ((P{query1} or D{query2}) and not A:alias2) xor D{query3}
A:alias1 and ((P{query1} or D{query2}) and not A:alias2) xor D{query3}
^
ParseException: Expected end of text, found 'and' (at char 9), (line:1, col:10)
FAIL: Expected end of text, found 'and' (at char 9), (line:1, col:10)
________________________________________________________________
test_execute_global_and_not
________________________________________________________________
self = <cumin.query.Query object at 0x7fda656e3c70>, query_string =
'D{host[1-5]} and not D{host2}'
def execute(self, query_string):
"""Override parent class execute method to implement the multi-query
capability.
:Parameters:
according to parent
:py:meth:`cumin.backends.BaseQueryAggregator.execute`.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
Raises:
cumin.backends.InvalidQueryError: if unable to parse the query.
"""
if 'default_backend' not in self.config:
try: # No default backend set, using directly the global grammar
> return super().execute(query_string)
../../../cumin/query.py:54:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656e3c70>, query_string =
'D{host[1-5]} and not D{host2}'
def execute(self, query_string):
"""Build and execute the query, return the NodeSet of FQDN hostnames
that matches.
Arguments:
query_string (str): the query string to be parsed and executed.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
"""
> self._build(query_string)
../../../cumin/backends/__init__.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656e3c70>, query_string =
'D{host[1-5]} and not D{host2}'
def _build(self, query_string):
"""Override parent method to reset the stack and log it.
:Parameters:
according to parent :py:meth:`cumin.backends.BaseQuery._build`.
"""
self.stack = self._get_stack_element()
self.stack_pointer = self.stack
> super()._build(query_string)
../../../cumin/backends/__init__.py:111:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656e3c70>, query_string =
'D{host[1-5]} and not D{host2}'
def _build(self, query_string):
"""Parse the query string according to the grammar and build the query
for later execution.
Arguments:
query_string (str): the query string to be parsed.
"""
self.logger.trace('Parsing query: %s', query_string)
> parsed = self.grammar.parseString(query_string.strip(), parseAll=True)
../../../cumin/backends/__init__.py:76:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Forward: {Group:({{Combine:({Combine:({D | K | O | P '{'}) SkipTo:('}')
'}'}) | Combine:({'A' ':' W:(+-.0-9A-Z_a-z)})}...e:({Combine:({D | K | O | P
'{'}) SkipTo:('}') '}'}) | Combine:({'A' ':' W:(+-.0-9A-Z_a-z)})} | {{'(' :
...} ')'})]...}
instring = 'D{host[1-5]} and not D{host2}', parse_all = False
def parse_string(
self, instring: str, parse_all: bool = False, *, parseAll: bool = False
) -> ParseResults:
"""
Parse a string with respect to the parser definition. This function is
intended as the primary interface to the
client code.
:param instring: The input string to be parsed.
:param parse_all: If set, the entire input string must match the
grammar.
:param parseAll: retained for pre-PEP8 compatibility, will be removed
in a future release.
:raises ParseException: Raised if ``parse_all`` is set and the input
string does not match the whole grammar.
:returns: the parsed data as a :class:`ParseResults` object, which may
be accessed as a `list`, a `dict`, or
an object with attributes if the given parser includes results names.
If the input string is required to match the entire grammar,
``parse_all`` flag must be set to ``True``. This
is also equivalent to ending the grammar with :class:`StringEnd`().
To report proper column numbers, ``parse_string`` operates on a copy of
the input string where all tabs are
converted to spaces (8 spaces per tab, as per the default in
``string.expandtabs``). If the input string
contains tabs and the grammar uses parse actions that use the ``loc``
argument to index into the string
being parsed, one can ensure a consistent view of the input string by
doing one of the following:
- calling ``parse_with_tabs`` on your grammar before calling
``parse_string`` (see :class:`parse_with_tabs`),
- define your parse action using the full ``(s,loc,toks)`` signature,
and reference the input string using the
parse action's ``s`` argument, or
- explicitly expand the tabs in your input string before calling
``parse_string``.
Examples:
By default, partial matches are OK.
>>> res = Word('a').parse_string('aaaaabaaa')
>>> print(res)
['aaaaa']
The parsing behavior varies by the inheriting class of this abstract
class. Please refer to the children
directly to see more examples.
It raises an exception if parse_all flag is set and instring does not
match the whole grammar.
>>> res = Word('a').parse_string('aaaaabaaa', parse_all=True)
Traceback (most recent call last):
...
pyparsing.ParseException: Expected end of text, found 'b' (at char 5),
(line:1, col:6)
"""
parseAll = parse_all or parseAll
ParserElement.reset_cache()
if not self.streamlined:
self.streamline()
for e in self.ignoreExprs:
e.streamline()
if not self.keepTabs:
instring = instring.expandtabs()
try:
loc, tokens = self._parse(instring, 0)
if parseAll:
loc = self.preParse(instring, loc)
se = Empty() + StringEnd()
se._parse(instring, loc)
except ParseBaseException as exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clearing out
pyparsing internal stack trace
> raise exc.with_traceback(None)
E pyparsing.exceptions.ParseException: Expected end of text,
found 'and' (at char 13), (line:1, col:14)
/usr/lib/python3/dist-packages/pyparsing/core.py:1134: ParseException
During handling of the above exception, another exception occurred:
def test_execute_global_and_not():
"""Executing an 'and not' between two queries should return the
difference of the hosts."""
query = Query({})
> hosts = query.execute('D{host[1-5]} and not D{host2}')
../../../cumin/tests/unit/test_query.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656e3c70>, query_string =
'D{host[1-5]} and not D{host2}'
def execute(self, query_string):
"""Override parent class execute method to implement the multi-query
capability.
:Parameters:
according to parent
:py:meth:`cumin.backends.BaseQueryAggregator.execute`.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
Raises:
cumin.backends.InvalidQueryError: if unable to parse the query.
"""
if 'default_backend' not in self.config:
try: # No default backend set, using directly the global grammar
return super().execute(query_string)
except ParseException as e:
> raise InvalidQueryError(("Unable to parse the query '{query}'
> with the global grammar and no "
"default backend is
set:\n{error}").format(query=query_string, error=e))
E cumin.backends.InvalidQueryError: Unable to parse the query
'D{host[1-5]} and not D{host2}' with the global grammar and no default backend
is set:
E Expected end of text, found 'and' (at char 13), (line:1,
col:14)
../../../cumin/query.py:56: InvalidQueryError
__________________________________________________________________
test_execute_subgroups
___________________________________________________________________
self = <cumin.query.Query object at 0x7fda656ae9b0>, query_string = '(D{host1}
or D{host2}) and not (D{host1})'
def execute(self, query_string):
"""Override parent class execute method to implement the multi-query
capability.
:Parameters:
according to parent
:py:meth:`cumin.backends.BaseQueryAggregator.execute`.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
Raises:
cumin.backends.InvalidQueryError: if unable to parse the query.
"""
if 'default_backend' not in self.config:
try: # No default backend set, using directly the global grammar
> return super().execute(query_string)
../../../cumin/query.py:54:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656ae9b0>, query_string = '(D{host1}
or D{host2}) and not (D{host1})'
def execute(self, query_string):
"""Build and execute the query, return the NodeSet of FQDN hostnames
that matches.
Arguments:
query_string (str): the query string to be parsed and executed.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
"""
> self._build(query_string)
../../../cumin/backends/__init__.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656ae9b0>, query_string = '(D{host1}
or D{host2}) and not (D{host1})'
def _build(self, query_string):
"""Override parent method to reset the stack and log it.
:Parameters:
according to parent :py:meth:`cumin.backends.BaseQuery._build`.
"""
self.stack = self._get_stack_element()
self.stack_pointer = self.stack
> super()._build(query_string)
../../../cumin/backends/__init__.py:111:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656ae9b0>, query_string = '(D{host1}
or D{host2}) and not (D{host1})'
def _build(self, query_string):
"""Parse the query string according to the grammar and build the query
for later execution.
Arguments:
query_string (str): the query string to be parsed.
"""
self.logger.trace('Parsing query: %s', query_string)
> parsed = self.grammar.parseString(query_string.strip(), parseAll=True)
../../../cumin/backends/__init__.py:76:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Forward: {Group:({{Combine:({Combine:({D | K | O | P '{'}) SkipTo:('}')
'}'}) | Combine:({'A' ':' W:(+-.0-9A-Z_a-z)})}...e:({Combine:({D | K | O | P
'{'}) SkipTo:('}') '}'}) | Combine:({'A' ':' W:(+-.0-9A-Z_a-z)})} | {{'(' :
...} ')'})]...}
instring = '(D{host1} or D{host2}) and not (D{host1})', parse_all = False
def parse_string(
self, instring: str, parse_all: bool = False, *, parseAll: bool = False
) -> ParseResults:
"""
Parse a string with respect to the parser definition. This function is
intended as the primary interface to the
client code.
:param instring: The input string to be parsed.
:param parse_all: If set, the entire input string must match the
grammar.
:param parseAll: retained for pre-PEP8 compatibility, will be removed
in a future release.
:raises ParseException: Raised if ``parse_all`` is set and the input
string does not match the whole grammar.
:returns: the parsed data as a :class:`ParseResults` object, which may
be accessed as a `list`, a `dict`, or
an object with attributes if the given parser includes results names.
If the input string is required to match the entire grammar,
``parse_all`` flag must be set to ``True``. This
is also equivalent to ending the grammar with :class:`StringEnd`().
To report proper column numbers, ``parse_string`` operates on a copy of
the input string where all tabs are
converted to spaces (8 spaces per tab, as per the default in
``string.expandtabs``). If the input string
contains tabs and the grammar uses parse actions that use the ``loc``
argument to index into the string
being parsed, one can ensure a consistent view of the input string by
doing one of the following:
- calling ``parse_with_tabs`` on your grammar before calling
``parse_string`` (see :class:`parse_with_tabs`),
- define your parse action using the full ``(s,loc,toks)`` signature,
and reference the input string using the
parse action's ``s`` argument, or
- explicitly expand the tabs in your input string before calling
``parse_string``.
Examples:
By default, partial matches are OK.
>>> res = Word('a').parse_string('aaaaabaaa')
>>> print(res)
['aaaaa']
The parsing behavior varies by the inheriting class of this abstract
class. Please refer to the children
directly to see more examples.
It raises an exception if parse_all flag is set and instring does not
match the whole grammar.
>>> res = Word('a').parse_string('aaaaabaaa', parse_all=True)
Traceback (most recent call last):
...
pyparsing.ParseException: Expected end of text, found 'b' (at char 5),
(line:1, col:6)
"""
parseAll = parse_all or parseAll
ParserElement.reset_cache()
if not self.streamlined:
self.streamline()
for e in self.ignoreExprs:
e.streamline()
if not self.keepTabs:
instring = instring.expandtabs()
try:
loc, tokens = self._parse(instring, 0)
if parseAll:
loc = self.preParse(instring, loc)
se = Empty() + StringEnd()
se._parse(instring, loc)
except ParseBaseException as exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clearing out
pyparsing internal stack trace
> raise exc.with_traceback(None)
E pyparsing.exceptions.ParseException: Expected end of text,
found 'and' (at char 23), (line:1, col:24)
/usr/lib/python3/dist-packages/pyparsing/core.py:1134: ParseException
During handling of the above exception, another exception occurred:
def test_execute_subgroups():
"""Executing a query with multiple subgroups should return the matching
hosts."""
query = Query({})
> hosts = query.execute('(D{host1} or D{host2}) and not (D{host1})')
../../../cumin/tests/unit/test_query.py:92:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda656ae9b0>, query_string = '(D{host1}
or D{host2}) and not (D{host1})'
def execute(self, query_string):
"""Override parent class execute method to implement the multi-query
capability.
:Parameters:
according to parent
:py:meth:`cumin.backends.BaseQueryAggregator.execute`.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
Raises:
cumin.backends.InvalidQueryError: if unable to parse the query.
"""
if 'default_backend' not in self.config:
try: # No default backend set, using directly the global grammar
return super().execute(query_string)
except ParseException as e:
> raise InvalidQueryError(("Unable to parse the query '{query}'
> with the global grammar and no "
"default backend is
set:\n{error}").format(query=query_string, error=e))
E cumin.backends.InvalidQueryError: Unable to parse the query
'(D{host1} or D{host2}) and not (D{host1})' with the global grammar and no
default backend is set:
E Expected end of text, found 'and' (at char 23), (line:1,
col:24)
../../../cumin/query.py:56: InvalidQueryError
________________________________________________________________
test_execute_complex_global
________________________________________________________________
self = <cumin.query.Query object at 0x7fda65524f70>
query_string = '(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and
not D{host1[20-30]}) and D{host1[01,15,30]})'
def execute(self, query_string):
"""Override parent class execute method to implement the multi-query
capability.
:Parameters:
according to parent
:py:meth:`cumin.backends.BaseQueryAggregator.execute`.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
Raises:
cumin.backends.InvalidQueryError: if unable to parse the query.
"""
if 'default_backend' not in self.config:
try: # No default backend set, using directly the global grammar
> return super().execute(query_string)
../../../cumin/query.py:54:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda65524f70>
query_string = '(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and
not D{host1[20-30]}) and D{host1[01,15,30]})'
def execute(self, query_string):
"""Build and execute the query, return the NodeSet of FQDN hostnames
that matches.
Arguments:
query_string (str): the query string to be parsed and executed.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
"""
> self._build(query_string)
../../../cumin/backends/__init__.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda65524f70>
query_string = '(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and
not D{host1[20-30]}) and D{host1[01,15,30]})'
def _build(self, query_string):
"""Override parent method to reset the stack and log it.
:Parameters:
according to parent :py:meth:`cumin.backends.BaseQuery._build`.
"""
self.stack = self._get_stack_element()
self.stack_pointer = self.stack
> super()._build(query_string)
../../../cumin/backends/__init__.py:111:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda65524f70>
query_string = '(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and
not D{host1[20-30]}) and D{host1[01,15,30]})'
def _build(self, query_string):
"""Parse the query string according to the grammar and build the query
for later execution.
Arguments:
query_string (str): the query string to be parsed.
"""
self.logger.trace('Parsing query: %s', query_string)
> parsed = self.grammar.parseString(query_string.strip(), parseAll=True)
../../../cumin/backends/__init__.py:76:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Forward: {Group:({{Combine:({Combine:({D | K | O | P '{'}) SkipTo:('}')
'}'}) | Combine:({'A' ':' W:(+-.0-9A-Z_a-z)})}...e:({Combine:({D | K | O | P
'{'}) SkipTo:('}') '}'}) | Combine:({'A' ':' W:(+-.0-9A-Z_a-z)})} | {{'(' :
...} ')'})]...}
instring = '(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and not
D{host1[20-30]}) and D{host1[01,15,30]})', parse_all = False
def parse_string(
self, instring: str, parse_all: bool = False, *, parseAll: bool = False
) -> ParseResults:
"""
Parse a string with respect to the parser definition. This function is
intended as the primary interface to the
client code.
:param instring: The input string to be parsed.
:param parse_all: If set, the entire input string must match the
grammar.
:param parseAll: retained for pre-PEP8 compatibility, will be removed
in a future release.
:raises ParseException: Raised if ``parse_all`` is set and the input
string does not match the whole grammar.
:returns: the parsed data as a :class:`ParseResults` object, which may
be accessed as a `list`, a `dict`, or
an object with attributes if the given parser includes results names.
If the input string is required to match the entire grammar,
``parse_all`` flag must be set to ``True``. This
is also equivalent to ending the grammar with :class:`StringEnd`().
To report proper column numbers, ``parse_string`` operates on a copy of
the input string where all tabs are
converted to spaces (8 spaces per tab, as per the default in
``string.expandtabs``). If the input string
contains tabs and the grammar uses parse actions that use the ``loc``
argument to index into the string
being parsed, one can ensure a consistent view of the input string by
doing one of the following:
- calling ``parse_with_tabs`` on your grammar before calling
``parse_string`` (see :class:`parse_with_tabs`),
- define your parse action using the full ``(s,loc,toks)`` signature,
and reference the input string using the
parse action's ``s`` argument, or
- explicitly expand the tabs in your input string before calling
``parse_string``.
Examples:
By default, partial matches are OK.
>>> res = Word('a').parse_string('aaaaabaaa')
>>> print(res)
['aaaaa']
The parsing behavior varies by the inheriting class of this abstract
class. Please refer to the children
directly to see more examples.
It raises an exception if parse_all flag is set and instring does not
match the whole grammar.
>>> res = Word('a').parse_string('aaaaabaaa', parse_all=True)
Traceback (most recent call last):
...
pyparsing.ParseException: Expected end of text, found 'b' (at char 5),
(line:1, col:6)
"""
parseAll = parse_all or parseAll
ParserElement.reset_cache()
if not self.streamlined:
self.streamline()
for e in self.ignoreExprs:
e.streamline()
if not self.keepTabs:
instring = instring.expandtabs()
try:
loc, tokens = self._parse(instring, 0)
if parseAll:
loc = self.preParse(instring, loc)
se = Empty() + StringEnd()
se._parse(instring, loc)
except ParseBaseException as exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clearing out
pyparsing internal stack trace
> raise exc.with_traceback(None)
E pyparsing.exceptions.ParseException: Expected end of text,
found 'or' (at char 36), (line:1, col:37)
/usr/lib/python3/dist-packages/pyparsing/core.py:1134: ParseException
During handling of the above exception, another exception occurred:
def test_execute_complex_global():
"""Executing a valid complex query should return the matching hosts."""
query = Query({})
> hosts = query.execute(
'(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and not
D{host1[20-30]}) and D{host1[01,15,30]})')
../../../cumin/tests/unit/test_query.py:127:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <cumin.query.Query object at 0x7fda65524f70>
query_string = '(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and
not D{host1[20-30]}) and D{host1[01,15,30]})'
def execute(self, query_string):
"""Override parent class execute method to implement the multi-query
capability.
:Parameters:
according to parent
:py:meth:`cumin.backends.BaseQueryAggregator.execute`.
Returns:
ClusterShell.NodeSet.NodeSet: with the FQDNs of the matching hosts.
Raises:
cumin.backends.InvalidQueryError: if unable to parse the query.
"""
if 'default_backend' not in self.config:
try: # No default backend set, using directly the global grammar
return super().execute(query_string)
except ParseException as e:
> raise InvalidQueryError(("Unable to parse the query '{query}'
> with the global grammar and no "
"default backend is
set:\n{error}").format(query=query_string, error=e))
E cumin.backends.InvalidQueryError: Unable to parse the query
'(D{(host1 or host2) and host[1-5]}) or ((D{host[100-150]} and not
D{host1[20-30]}) and D{host1[01,15,30]})' with the global grammar and no
default backend is set:
E Expected end of text, found 'or' (at char 36), (line:1, col:37)
../../../cumin/query.py:56: InvalidQueryError
_______________________________________________________________
TestDirectQuery.test_execute
________________________________________________________________
self = <cumin.tests.unit.backends.test_direct.TestDirectQuery object at
0x7fda659037c0>
def test_execute(self):
"""Calling execute() should return the list of hosts."""
assert self.query.execute('host1 or host2') == nodeset('host[1-2]')
assert self.query.execute('host1 and host2') == nodeset()
> assert self.query.execute('host1 and not host2') == nodeset('host1')
../../../cumin/tests/unit/backends/test_direct.py:29:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../cumin/backends/__init__.py:47: in execute
self._build(query_string)
../../../cumin/backends/__init__.py:111: in _build
super()._build(query_string)
../../../cumin/backends/__init__.py:76: in _build
parsed = self.grammar.parseString(query_string.strip(), parseAll=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Forward: {Group:({{~{{'and not' | 'and' | 'xor' | 'or'}}
W:(!&,-.0-9A-[]-...)} | {{'(' : ...} ')'}}) [Group:({'and not' | 'and' | 'xor'
| 'or'} {{~{{'and not' | 'and' | 'xor' | 'or'}} W:(!&,-.0-9A-[]-...)} | {{'(' :
...} ')'}})]...}
instring = 'host1 and not host2', parse_all = False
def parse_string(
self, instring: str, parse_all: bool = False, *, parseAll: bool = False
) -> ParseResults:
"""
Parse a string with respect to the parser definition. This function is
intended as the primary interface to the
client code.
:param instring: The input string to be parsed.
:param parse_all: If set, the entire input string must match the
grammar.
:param parseAll: retained for pre-PEP8 compatibility, will be removed
in a future release.
:raises ParseException: Raised if ``parse_all`` is set and the input
string does not match the whole grammar.
:returns: the parsed data as a :class:`ParseResults` object, which may
be accessed as a `list`, a `dict`, or
an object with attributes if the given parser includes results names.
If the input string is required to match the entire grammar,
``parse_all`` flag must be set to ``True``. This
is also equivalent to ending the grammar with :class:`StringEnd`().
To report proper column numbers, ``parse_string`` operates on a copy of
the input string where all tabs are
converted to spaces (8 spaces per tab, as per the default in
``string.expandtabs``). If the input string
contains tabs and the grammar uses parse actions that use the ``loc``
argument to index into the string
being parsed, one can ensure a consistent view of the input string by
doing one of the following:
- calling ``parse_with_tabs`` on your grammar before calling
``parse_string`` (see :class:`parse_with_tabs`),
- define your parse action using the full ``(s,loc,toks)`` signature,
and reference the input string using the
parse action's ``s`` argument, or
- explicitly expand the tabs in your input string before calling
``parse_string``.
Examples:
By default, partial matches are OK.
>>> res = Word('a').parse_string('aaaaabaaa')
>>> print(res)
['aaaaa']
The parsing behavior varies by the inheriting class of this abstract
class. Please refer to the children
directly to see more examples.
It raises an exception if parse_all flag is set and instring does not
match the whole grammar.
>>> res = Word('a').parse_string('aaaaabaaa', parse_all=True)
Traceback (most recent call last):
...
pyparsing.ParseException: Expected end of text, found 'b' (at char 5),
(line:1, col:6)
"""
parseAll = parse_all or parseAll
ParserElement.reset_cache()
if not self.streamlined:
self.streamline()
for e in self.ignoreExprs:
e.streamline()
if not self.keepTabs:
instring = instring.expandtabs()
try:
loc, tokens = self._parse(instring, 0)
if parseAll:
loc = self.preParse(instring, loc)
se = Empty() + StringEnd()
se._parse(instring, loc)
except ParseBaseException as exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clearing out
pyparsing internal stack trace
> raise exc.with_traceback(None)
E pyparsing.exceptions.ParseException: Expected end of text,
found 'host2' (at char 14), (line:1, col:15)
/usr/lib/python3/dist-packages/pyparsing/core.py:1134: ParseException
________________________________________________________________
test_valid_grammars[direct]
________________________________________________________________
backend_name = 'direct'
@pytest.mark.parametrize('backend_name', BACKENDS)
def test_valid_grammars(backend_name):
"""Run quick pyparsing test over valid grammar strings for each backend
that has the appropriate fixture."""
try:
backend =
importlib.import_module('cumin.backends.{backend}'.format(backend=backend_name))
except ImportError:
return # Backend not available
results = backend.grammar().runTests(
get_fixture(os.path.join(BASE_PATH,
'{backend}_valid.txt'.format(backend=backend_name)), as_string=True))
> assert results[0]
E assert False
../../../cumin/tests/unit/backends/test_grammars.py:26: AssertionError
------------------------------------------------------------------- Captured
stdout call --------------------------------------------------------------------
# Valid grammars
hostname
[['hostname']]
[0]:
['hostname']
- hosts: ['hostname']
host-name
[['host-name']]
[0]:
['host-name']
- hosts: ['host-name']
host_name.domain
[['host_name.domain']]
[0]:
['host_name.domain']
- hosts: ['host_name.domain']
hostname and host_name.domain.tld
[['hostname'], ['and', 'host_name.domain.tld']]
[0]:
['hostname']
- hosts: ['hostname']
[1]:
['and', 'host_name.domain.tld']
- bool: 'and'
- hosts: ['host_name.domain.tld']
host1 or host2
[['host1'], ['or', 'host2']]
[0]:
['host1']
- hosts: ['host1']
[1]:
['or', 'host2']
- bool: 'or'
- hosts: ['host2']
host1 and host2
[['host1'], ['and', 'host2']]
[0]:
['host1']
- hosts: ['host1']
[1]:
['and', 'host2']
- bool: 'and'
- hosts: ['host2']
host1 and not host2
host1 and not host2
^
ParseException: Expected end of text, found 'host2' (at char 14), (line:1,
col:15)
FAIL: Expected end of text, found 'host2' (at char 14), (line:1, col:15)
host10[10-20,30-50]
[['host10[10-20,30-50]']]
[0]:
['host10[10-20,30-50]']
- hosts: ['host10[10-20,30-50]']
(hostname.domain.tld)
[['(', ['hostname.domain.tld'], ')']]
[0]:
['(', ['hostname.domain.tld'], ')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['hostname.domain.tld']
- hosts: ['hostname.domain.tld']
[2]:
)
(host1 or host2) and host1
[['(', ['host1'], ['or', 'host2'], ')'], ['and', 'host1']]
[0]:
['(', ['host1'], ['or', 'host2'], ')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['host1']
- hosts: ['host1']
[2]:
['or', 'host2']
- bool: 'or'
- hosts: ['host2']
[3]:
)
[1]:
['and', 'host1']
- bool: 'and'
- hosts: ['host1']
((host1[0-9] or host01) and host[01-10])
[['(', ['(', ['host1[0-9]'], ['or', 'host01'], ')'], ['and', 'host[01-10]'],
')']]
[0]:
['(', ['(', ['host1[0-9]'], ['or', 'host01'], ')'], ['and', 'host[01-10]'],
')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['(', ['host1[0-9]'], ['or', 'host01'], ')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['host1[0-9]']
- hosts: ['host1[0-9]']
[2]:
['or', 'host01']
- bool: 'or'
- hosts: ['host01']
[3]:
)
[2]:
['and', 'host[01-10]']
- bool: 'and'
- hosts: ['host[01-10]']
[3]:
)
______________________________________________________________
test_valid_grammars[knownhosts]
______________________________________________________________
backend_name = 'knownhosts'
@pytest.mark.parametrize('backend_name', BACKENDS)
def test_valid_grammars(backend_name):
"""Run quick pyparsing test over valid grammar strings for each backend
that has the appropriate fixture."""
try:
backend =
importlib.import_module('cumin.backends.{backend}'.format(backend=backend_name))
except ImportError:
return # Backend not available
results = backend.grammar().runTests(
get_fixture(os.path.join(BASE_PATH,
'{backend}_valid.txt'.format(backend=backend_name)), as_string=True))
> assert results[0]
E assert False
../../../cumin/tests/unit/backends/test_grammars.py:26: AssertionError
------------------------------------------------------------------- Captured
stdout call --------------------------------------------------------------------
# Valid grammars
hostname
[['hostname']]
[0]:
['hostname']
- hosts: ['hostname']
host-name
[['host-name']]
[0]:
['host-name']
- hosts: ['host-name']
host_name.domain
[['host_name.domain']]
[0]:
['host_name.domain']
- hosts: ['host_name.domain']
hostname and host_name.domain.tld
[['hostname'], ['and', 'host_name.domain.tld']]
[0]:
['hostname']
- hosts: ['hostname']
[1]:
['and', 'host_name.domain.tld']
- bool: 'and'
- hosts: ['host_name.domain.tld']
host1 or host2
[['host1'], ['or', 'host2']]
[0]:
['host1']
- hosts: ['host1']
[1]:
['or', 'host2']
- bool: 'or'
- hosts: ['host2']
host1 and host2
[['host1'], ['and', 'host2']]
[0]:
['host1']
- hosts: ['host1']
[1]:
['and', 'host2']
- bool: 'and'
- hosts: ['host2']
host1 and not host2
host1 and not host2
^
ParseException: Expected end of text, found 'host2' (at char 14), (line:1,
col:15)
FAIL: Expected end of text, found 'host2' (at char 14), (line:1, col:15)
host10[10-20,30-50]
[['host10[10-20,30-50]']]
[0]:
['host10[10-20,30-50]']
- hosts: ['host10[10-20,30-50]']
host10[10-20,30-50]*
[['host10[10-20,30-50]*']]
[0]:
['host10[10-20,30-50]*']
- hosts: ['host10[10-20,30-50]*']
host?0[10-20,30-50]*
[['host?0[10-20,30-50]*']]
[0]:
['host?0[10-20,30-50]*']
- hosts: ['host?0[10-20,30-50]*']
(hostname.domain.tld)
[['(', ['hostname.domain.tld'], ')']]
[0]:
['(', ['hostname.domain.tld'], ')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['hostname.domain.tld']
- hosts: ['hostname.domain.tld']
[2]:
)
(host1 or host2) and host1
[['(', ['host1'], ['or', 'host2'], ')'], ['and', 'host1']]
[0]:
['(', ['host1'], ['or', 'host2'], ')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['host1']
- hosts: ['host1']
[2]:
['or', 'host2']
- bool: 'or'
- hosts: ['host2']
[3]:
)
[1]:
['and', 'host1']
- bool: 'and'
- hosts: ['host1']
((host1[0-9] or host01) and host[01-10])
[['(', ['(', ['host1[0-9]'], ['or', 'host01'], ')'], ['and', 'host[01-10]'],
')']]
[0]:
['(', ['(', ['host1[0-9]'], ['or', 'host01'], ')'], ['and', 'host[01-10]'],
')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['(', ['host1[0-9]'], ['or', 'host01'], ')']
- close_subgroup: ')'
- open_subgroup: '('
[0]:
(
[1]:
['host1[0-9]']
- hosts: ['host1[0-9]']
[2]:
['or', 'host01']
- bool: 'or'
- hosts: ['host01']
[3]:
)
[2]:
['and', 'host[01-10]']
- bool: 'and'
- hosts: ['host[01-10]']
[3]:
)
_________________________________________________________
TestKnownhostsQuery.test_execute_and_not
__________________________________________________________
self = <cumin.tests.unit.backends.test_knownhosts.TestKnownhostsQuery object at
0x7fda656ef130>
def test_execute_and_not(self):
"""Calling execute() with two hosts with 'and not' should return the
first host."""
expected = NodeSet('host1.domain', resolver=RESOLVER_NOGROUP)
> assert self.query.execute('host1.domain and not host2.domain') ==
> expected
../../../cumin/tests/unit/backends/test_knownhosts.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../cumin/backends/__init__.py:47: in execute
self._build(query_string)
../../../cumin/backends/knownhosts.py:103: in _build
super()._build(query_string)
../../../cumin/backends/__init__.py:111: in _build
super()._build(query_string)
../../../cumin/backends/__init__.py:76: in _build
parsed = self.grammar.parseString(query_string.strip(), parseAll=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Forward: {Group:({{~{{'and not' | 'and' | 'xor' | 'or'}}
W:(!&*,-.0-9?A-[...)} | {{'(' : ...} ')'}}) [Group:({'and not' | 'and' | 'xor'
| 'or'} {{~{{'and not' | 'and' | 'xor' | 'or'}} W:(!&*,-.0-9?A-[...)} | {{'(' :
...} ')'}})]...}
instring = 'host1.domain and not host2.domain', parse_all = False
def parse_string(
self, instring: str, parse_all: bool = False, *, parseAll: bool = False
) -> ParseResults:
"""
Parse a string with respect to the parser definition. This function is
intended as the primary interface to the
client code.
:param instring: The input string to be parsed.
:param parse_all: If set, the entire input string must match the
grammar.
:param parseAll: retained for pre-PEP8 compatibility, will be removed
in a future release.
:raises ParseException: Raised if ``parse_all`` is set and the input
string does not match the whole grammar.
:returns: the parsed data as a :class:`ParseResults` object, which may
be accessed as a `list`, a `dict`, or
an object with attributes if the given parser includes results names.
If the input string is required to match the entire grammar,
``parse_all`` flag must be set to ``True``. This
is also equivalent to ending the grammar with :class:`StringEnd`().
To report proper column numbers, ``parse_string`` operates on a copy of
the input string where all tabs are
converted to spaces (8 spaces per tab, as per the default in
``string.expandtabs``). If the input string
contains tabs and the grammar uses parse actions that use the ``loc``
argument to index into the string
being parsed, one can ensure a consistent view of the input string by
doing one of the following:
- calling ``parse_with_tabs`` on your grammar before calling
``parse_string`` (see :class:`parse_with_tabs`),
- define your parse action using the full ``(s,loc,toks)`` signature,
and reference the input string using the
parse action's ``s`` argument, or
- explicitly expand the tabs in your input string before calling
``parse_string``.
Examples:
By default, partial matches are OK.
>>> res = Word('a').parse_string('aaaaabaaa')
>>> print(res)
['aaaaa']
The parsing behavior varies by the inheriting class of this abstract
class. Please refer to the children
directly to see more examples.
It raises an exception if parse_all flag is set and instring does not
match the whole grammar.
>>> res = Word('a').parse_string('aaaaabaaa', parse_all=True)
Traceback (most recent call last):
...
pyparsing.ParseException: Expected end of text, found 'b' (at char 5),
(line:1, col:6)
"""
parseAll = parse_all or parseAll
ParserElement.reset_cache()
if not self.streamlined:
self.streamline()
for e in self.ignoreExprs:
e.streamline()
if not self.keepTabs:
instring = instring.expandtabs()
try:
loc, tokens = self._parse(instring, 0)
if parseAll:
loc = self.preParse(instring, loc)
se = Empty() + StringEnd()
se._parse(instring, loc)
except ParseBaseException as exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clearing out
pyparsing internal stack trace
> raise exc.with_traceback(None)
E pyparsing.exceptions.ParseException: Expected end of text,
found 'host2' (at char 21), (line:1, col:22)
/usr/lib/python3/dist-packages/pyparsing/core.py:1134: ParseException
--------------------------------------------------------------------- Captured
log call ---------------------------------------------------------------------
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 24 (not enough fields) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
host10.domain ssh-rsa
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 26 (not enough fields) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
@cert-authority host11.domain ssh-rsa
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 28 (unknown marker) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
@marker host12.domain ssh-rsa AAAA...=
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 50 (not enough fields) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
invalid line
_________________________________________________________
TestKnownhostsQuery.test_execute_complex
__________________________________________________________
self = <cumin.tests.unit.backends.test_knownhosts.TestKnownhostsQuery object at
0x7fda656eee60>
def test_execute_complex(self):
"""Calling execute() with a complex query should return the matching
hosts."""
expected = NodeSet('host[1,5,8].domain', resolver=RESOLVER_NOGROUP)
> assert self.query.execute('host1.domain or (host[5-9].domain and not
> host7.domain)') == expected
../../../cumin/tests/unit/backends/test_knownhosts.py:70:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../cumin/backends/__init__.py:47: in execute
self._build(query_string)
../../../cumin/backends/knownhosts.py:103: in _build
super()._build(query_string)
../../../cumin/backends/__init__.py:111: in _build
super()._build(query_string)
../../../cumin/backends/__init__.py:76: in _build
parsed = self.grammar.parseString(query_string.strip(), parseAll=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Forward: {Group:({{~{{'and not' | 'and' | 'xor' | 'or'}}
W:(!&*,-.0-9?A-[...)} | {{'(' : ...} ')'}}) [Group:({'and not' | 'and' | 'xor'
| 'or'} {{~{{'and not' | 'and' | 'xor' | 'or'}} W:(!&*,-.0-9?A-[...)} | {{'(' :
...} ')'}})]...}
instring = 'host1.domain or (host[5-9].domain and not host7.domain)', parse_all
= False
def parse_string(
self, instring: str, parse_all: bool = False, *, parseAll: bool = False
) -> ParseResults:
"""
Parse a string with respect to the parser definition. This function is
intended as the primary interface to the
client code.
:param instring: The input string to be parsed.
:param parse_all: If set, the entire input string must match the
grammar.
:param parseAll: retained for pre-PEP8 compatibility, will be removed
in a future release.
:raises ParseException: Raised if ``parse_all`` is set and the input
string does not match the whole grammar.
:returns: the parsed data as a :class:`ParseResults` object, which may
be accessed as a `list`, a `dict`, or
an object with attributes if the given parser includes results names.
If the input string is required to match the entire grammar,
``parse_all`` flag must be set to ``True``. This
is also equivalent to ending the grammar with :class:`StringEnd`().
To report proper column numbers, ``parse_string`` operates on a copy of
the input string where all tabs are
converted to spaces (8 spaces per tab, as per the default in
``string.expandtabs``). If the input string
contains tabs and the grammar uses parse actions that use the ``loc``
argument to index into the string
being parsed, one can ensure a consistent view of the input string by
doing one of the following:
- calling ``parse_with_tabs`` on your grammar before calling
``parse_string`` (see :class:`parse_with_tabs`),
- define your parse action using the full ``(s,loc,toks)`` signature,
and reference the input string using the
parse action's ``s`` argument, or
- explicitly expand the tabs in your input string before calling
``parse_string``.
Examples:
By default, partial matches are OK.
>>> res = Word('a').parse_string('aaaaabaaa')
>>> print(res)
['aaaaa']
The parsing behavior varies by the inheriting class of this abstract
class. Please refer to the children
directly to see more examples.
It raises an exception if parse_all flag is set and instring does not
match the whole grammar.
>>> res = Word('a').parse_string('aaaaabaaa', parse_all=True)
Traceback (most recent call last):
...
pyparsing.ParseException: Expected end of text, found 'b' (at char 5),
(line:1, col:6)
"""
parseAll = parse_all or parseAll
ParserElement.reset_cache()
if not self.streamlined:
self.streamline()
for e in self.ignoreExprs:
e.streamline()
if not self.keepTabs:
instring = instring.expandtabs()
try:
loc, tokens = self._parse(instring, 0)
if parseAll:
loc = self.preParse(instring, loc)
se = Empty() + StringEnd()
se._parse(instring, loc)
except ParseBaseException as exc:
if ParserElement.verbose_stacktrace:
raise
else:
# catch and re-raise exception from here, clearing out
pyparsing internal stack trace
> raise exc.with_traceback(None)
E pyparsing.exceptions.ParseException: Expected end of text,
found 'or' (at char 13), (line:1, col:14)
/usr/lib/python3/dist-packages/pyparsing/core.py:1134: ParseException
--------------------------------------------------------------------- Captured
log call ---------------------------------------------------------------------
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 24 (not enough fields) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
host10.domain ssh-rsa
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 26 (not enough fields) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
@cert-authority host11.domain ssh-rsa
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 28 (unknown marker) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
@marker host12.domain ssh-rsa AAAA...=
WARNING cumin.backends.knownhosts.KnownHostsQuery:knownhosts.py:161 Discarded
invalid line 50 (not enough fields) in known hosts file
'/home/anarcat/dist/cumin/cumin/tests/fixtures/backends/knownhosts.txt':
invalid line
================================================================== short test
summary info ==================================================================
FAILED ../../../cumin/tests/unit/test_grammar.py::test_valid_strings - assert
False
FAILED ../../../cumin/tests/unit/test_query.py::test_execute_global_and_not -
cumin.backends.InvalidQueryError: Unable to parse the query 'D{host[1-5]} an...
FAILED ../../../cumin/tests/unit/test_query.py::test_execute_subgroups -
cumin.backends.InvalidQueryError: Unable to parse the query '(D{host1} or
D{host2...
FAILED ../../../cumin/tests/unit/test_query.py::test_execute_complex_global -
cumin.backends.InvalidQueryError: Unable to parse the query '(D{(host1 or ho...
FAILED
../../../cumin/tests/unit/backends/test_direct.py::TestDirectQuery::test_execute
- pyparsing.exceptions.ParseException: Expected end of text, found...
FAILED
../../../cumin/tests/unit/backends/test_grammars.py::test_valid_grammars[direct]
- assert False
FAILED
../../../cumin/tests/unit/backends/test_grammars.py::test_valid_grammars[knownhosts]
- assert False
FAILED
../../../cumin/tests/unit/backends/test_knownhosts.py::TestKnownhostsQuery::test_execute_and_not
- pyparsing.exceptions.ParseException: Expected en...
FAILED
../../../cumin/tests/unit/backends/test_knownhosts.py::TestKnownhostsQuery::test_execute_complex
- pyparsing.exceptions.ParseException: Expected en...
=============================================================== 9 failed, 407
passed in 1.55s ===============================================================
E: pybuild pybuild:386: test: plugin distutils failed with: exit code=1: cd
/home/anarcat/dist/cumin/.pybuild/cpython3_3.10_cumin/build; python3.10 -m
pytest {dir}/cumin/tests/unit
dh_auto_test: error: pybuild --test -i python{version} -p 3.10 returned exit
code 13
make: *** [debian/rules:13 : build] Erreur 13
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
