Re: [PATCH 01/20] test: new test framework to compare json parts

[Tomi Ollila]

On Wed, Jun 06 2018, Daniel Kahn Gillmor wrote:

> On Tue 2018-06-05 22:06:07 -0300, David Bremner wrote:
>> Daniel Kahn Gillmor  writes:
>>> +
>>> +if len(sys.argv) < 2:
>>> +sys.exit("""usage: {} EXPR [EXPR]
>>> +
>>
>> the useage message doesn't seem to work? I get
>>
>> ╭─ zancas:software/upstream/notmuch/test 
>> ╰─ (git)-[master]-% python3 json_check_nodes.py  
>> Traceback (most recent call last):
>>   File "json_check_nodes.py", line 42, in 
>> """.format(sys.argv[0]))
>> KeyError: '"c"'
>>
>> I guess this is not tested with python2?
>>
>> ╭─ zancas:software/upstream/notmuch/test 
>> ╰─ (git)-[master]-%  echo '["a", "b", {"c": 1}]' | python2 
>> json_check_nodes.py 'second_d:[1]="d"' 'no_c:[2]!"c"' 
>> Traceback (most recent call last):
>>   File "json_check_nodes.py", line 60, in 
>> e = 'data{}'.format(expr['address'])
>> TypeError: '_sre.SRE_Match' object has no attribute '__getitem__'
>>
>> The test suite currently supports python2 and python3 (or at least it's
>> supposed to).
>
> thanks for noticing these incompatibilities with python2.  i've
> corrected them in my working tree, and will include the fixes in the
> next revision of this series.

If it is going to be python2 and python3 compatible, you could
add hashbang #!/usr/bin/env python 
and make the file executable.

From tests it still can be run w/ $NOTMUCH_PYTHON but it could
also work standalone should anyone(tm) want to do so...

Tomi

>
>--dkg
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 01/20] test: new test framework to compare json parts

[Daniel Kahn Gillmor]

On Wed 2018-06-06 13:21:13 -0300, David Bremner wrote:
> Just to be clear, the first problem is (was?) also present with python3

yep, got it :)

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 01/20] test: new test framework to compare json parts

[David Bremner]

Daniel Kahn Gillmor  writes:

> On Tue 2018-06-05 22:06:07 -0300, David Bremner wrote:
>> Daniel Kahn Gillmor  writes:
>>> +
>>> +if len(sys.argv) < 2:
>>> +sys.exit("""usage: {} EXPR [EXPR]
>>> +
>>
>> the useage message doesn't seem to work? I get
>>
>> ╭─ zancas:software/upstream/notmuch/test 
>> ╰─ (git)-[master]-% python3 json_check_nodes.py  
>> Traceback (most recent call last):
>>   File "json_check_nodes.py", line 42, in 
>> """.format(sys.argv[0]))
>> KeyError: '"c"'
>>
>> I guess this is not tested with python2?
>>
>> ╭─ zancas:software/upstream/notmuch/test 
>> ╰─ (git)-[master]-%  echo '["a", "b", {"c": 1}]' | python2 
>> json_check_nodes.py 'second_d:[1]="d"' 'no_c:[2]!"c"' 
>> Traceback (most recent call last):
>>   File "json_check_nodes.py", line 60, in 
>> e = 'data{}'.format(expr['address'])
>> TypeError: '_sre.SRE_Match' object has no attribute '__getitem__'
>>
>> The test suite currently supports python2 and python3 (or at least it's
>> supposed to).
>
> thanks for noticing these incompatibilities with python2.  i've
> corrected them in my working tree, and will include the fixes in the
> next revision of this series.

Just to be clear, the first problem is (was?) also present with python3

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 01/20] test: new test framework to compare json parts

[David Bremner]

Daniel Kahn Gillmor  writes:
> +
> +if len(sys.argv) < 2:
> +sys.exit("""usage: {} EXPR [EXPR]
> +

the useage message doesn't seem to work? I get

╭─ zancas:software/upstream/notmuch/test 
╰─ (git)-[master]-% python3 json_check_nodes.py  
Traceback (most recent call last):
  File "json_check_nodes.py", line 42, in 
""".format(sys.argv[0]))
KeyError: '"c"'

I guess this is not tested with python2?

╭─ zancas:software/upstream/notmuch/test 
╰─ (git)-[master]-%  echo '["a", "b", {"c": 1}]' | python2 json_check_nodes.py 
'second_d:[1]="d"' 'no_c:[2]!"c"' 
Traceback (most recent call last):
  File "json_check_nodes.py", line 60, in 
e = 'data{}'.format(expr['address'])
TypeError: '_sre.SRE_Match' object has no attribute '__getitem__'

The test suite currently supports python2 and python3 (or at least it's
supposed to).

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 01/20] test: new test framework to compare json parts

[Daniel Kahn Gillmor]

From: Jameson Graef Rollins 

This makes it easier to write fairly compact, readable tests of json
output, without needing to sanitize away parts that we don't care
about.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/json_check_nodes.py | 112 +++
 test/test-lib.sh |  22 
 2 files changed, 134 insertions(+)
 create mode 100644 test/json_check_nodes.py

diff --git a/test/json_check_nodes.py b/test/json_check_nodes.py
new file mode 100644
index ..5e386aec
--- /dev/null
+++ b/test/json_check_nodes.py
@@ -0,0 +1,112 @@
+import re
+import sys
+import json
+
+
+EXPR_RE = 
re.compile('(?P[a-zA-Z0-9_-]+):(?P[^=!]+)(?:(?P[=!])(?P.*))?',
 re.DOTALL|re.MULTILINE)
+
+
+if len(sys.argv) < 2:
+sys.exit("""usage: {} EXPR [EXPR]
+
+Takes json data on stdin and evaluates test expressions specified in
+arguments.  Each test is evaluated, and output is printed only if the
+test fails.  If any test fails there return value of execution will be
+non-zero.
+
+EXPR can be one of following types:
+
+Value test: test that object in json data found at address is equal to 
specified value:
+
+  label:address|value
+
+Existence test: test that dict or list in json data found at address
+does *not* contain the specified key:
+
+  label:address!key
+
+Extract: extract object from json data found at address and print
+
+  label:address
+
+Results are printed to stdout prefixed by expression label.  In all
+cases the test will fail if object does not exist in data.
+
+Example:
+
+0 $ echo '["a", "b", {"c": 1}]' | python3 json_check_nodes.py 
'second_d:[1]="d"' 'no_c:[2]!"c"'
+second_d: value not equal: data[1] = 'b' != 'd'
+no_c: dict contains key: data[2]["c"] = "c"
+1 $
+
+""".format(sys.argv[0]))
+
+
+# parse expressions from arguments
+exprs = []
+for expr in sys.argv[1:]:
+m = re.match(EXPR_RE, expr)
+if not m:
+sys.exit("Invalid expression: {}".format(expr))
+exprs.append(m)
+
+data = json.load(sys.stdin)
+
+fail = False
+
+for expr in exprs:
+# print(expr.groups(),fail)
+
+e = 'data{}'.format(expr['address'])
+try:
+val = eval(e)
+except SyntaxError:
+fail = True
+print("{}: syntax error on evaluation of object: {}".format(
+expr['label'], e))
+continue
+except:
+fail = True
+print("{}: object not found: data{}".format(
+expr['label'], expr['address']))
+continue
+
+if expr['type'] == '=':
+try:
+obj_val = json.loads(expr['val'])
+except:
+fail = True
+print("{}: error evaluating value: {}".format(
+expr['label'], expr['address']))
+continue
+if val != obj_val:
+fail = True
+print("{}: value not equal: data{} = {} != {}".format(
+expr['label'], expr['address'], repr(val), repr(obj_val)))
+
+elif expr['type'] == '!':
+if not isinstance(val, (dict, list)):
+fail = True
+print("{}: not a dict or a list: data{}".format(
+expr['label'], expr['address']))
+continue
+try:
+idx = json.loads(expr['val'])
+if idx in val:
+fail = True
+print("{}: {} contains key: {}[{}] = {}".format(
+expr['label'], type(val), e, expr['val'], val[idx]))
+except SyntaxError:
+fail = True
+print("{}: syntax error on evaluation of value: {}".format(
+expr['label'], expr['val']))
+continue
+
+
+elif expr['type'] is None:
+print("{}: {}".format(expr['label'], val))
+
+
+if fail:
+sys.exit(1)
+sys.exit(0)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 7e064021..18f34e85 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -497,6 +497,28 @@ test_sort_json () {
 "import sys, json; json.dump(sorted(json.load(sys.stdin)),sys.stdout)"
 }
 
+# test for json objects
+test_json_nodes () {
+exec 1>&6 2>&7 # Restore stdout and stderr
+   if [ -z "$inside_subtest" ]; then
+   error "bug in the test script: test_json_eval without 
test_begin_subtest"
+   fi
+   inside_subtest=
+   test "$#" > 0 ||
+   error "bug in the test script: test_json_nodes needs at least 1 
parameter"
+
+   if ! test_skip "$test_subtest_name"
+   then
+   output=$(PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON 
"$TEST_DIRECTORY"/json_check_nodes.py "$@")
+   if [ "$?" = 0 ]
+   then
+   test_ok_
+   else
+   test_failure_ "$output"
+   fi
+   fi
+}
+
 test_emacs_expect_t () {
test "$#" = 1 ||
error "bug in the test script: not 1 parameter to test_emacs_expect_t"
-- 
2.17.0

___
notmuch