Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On 07/06/18 03:23, Jeff King wrote: > On Thu, Jun 07, 2018 at 01:16:14AM +0100, Ramsay Jones wrote: > >>> Probably. We may want to go the same route as we did for perl in >>> a0e0ec9f7d (t: provide a perl() function which uses $PERL_PATH, >>> 2013-10-28) so that test writers don't have to remember this. >>> >>> That said, I wonder if it would be hard to simply do the python bits >>> here in perl. This is the first use of python in our test scripts (and >> >> Hmm, not quite the _first_ use: >> >> $ git grep PYTHON_PATH -- t >> t/lib-git-p4.sh:(cd / && "$PYTHON_PATH" -c 'import time; >> print(int(time.time()))') >> t/lib-git-p4.sh:"$PYTHON_PATH" "$TRASH_DIRECTORY/marshal-dump.py" >> t/t9020-remote-svn.sh:export PATH PYTHON_PATH GIT_BUILD_DIR >> t/t9020-remote-svn.sh:exec "$PYTHON_PATH" >> "$GIT_BUILD_DIR/contrib/svn-fe/svnrdump_sim.py" "$@" >> t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" >> "$TRASH_DIRECTORY/k_smush.py" <"$cli/k-text-k" >cli-k-text-k-smush && >> t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" >> "$TRASH_DIRECTORY/ko_smush.py" <"$cli/k-text-ko" >cli-k-text-ko-smush && >> t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" >> "$TRASH_DIRECTORY/gendouble.py" >%double.png && >> t/t9810-git-p4-rcs.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/scrub_k.py" >> <"$git/$file" >"$scrub" && >> t/t9810-git-p4-rcs.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/scrub_ko.py" >> <"$git/$file" >"$scrub" && >> $ > > OK, the first for a feature that is not already written in python > (leading into my second claim that python is used only for fringe > commands ;) ). > > Though maybe I am wrong that the remote-svn stuff requires python. I > thought it did, but poking around, it looks like it's all C, and just > the "svnrdump_sim" helper is python. Heh, I was a bit tired last night, so although I knew that I required python to be installed to run the test-suite, I could not remember why. As soon as I went to bed, ... :) I recently installed Ubuntu 18.04, so that I could get a heads up on any possible problems later this month when I install Linux Mint 19. In order to get the test-suite to run, I had to set 'PYTHON_PATH = /usr/bin/python3' in my config.mak file. (yes, I could have set NO_PYTHON, but you get the idea). Ubuntu 18.04 no longer installs python2 by default (it has ported to python3), but presumably you can still install it as '/usr/bin/python' (I didn't check). In any event, it was t9020-remote-svn.sh that was failing which, despite the name, does not depend on 'svn'. As you already noted, it does run svnrdump_sim.py and the git-remote-testsvn. > At any rate, I think the point still stands that perl is our main > scripting language. I'd rather keep us to that unless there's a good > reason not to. Agreed. And I see it has come to pass ... :-D ATB, Ramsay Jones
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On Wed, Jun 06, 2018 at 10:23:53PM -0400, Jeff King wrote: > Though maybe I am wrong that the remote-svn stuff requires python. I > thought it did, but poking around, it looks like it's all C, and just > the "svnrdump_sim" helper is python. I think I was getting this mixed up with the git_remote_helpers python work, which was removed long ago in ae34ac126f (git_remote_helpers: remove little used Python library, 2013-09-07). Note that it really changes much, but I was curious enough to dig down. -Peff
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On Wed, Jun 06, 2018 at 09:49:09PM -0400, Todd Zullinger wrote: > Ramsay Jones wrote: > [...] > > I don't run the p4 or svn tests, so ... :-D > > Heh, lucky you. :) > > I try to run them all as part of the fedora builds since > they cover much more than I'd ever use. That's the main > reason I noticed the bare python. That would trip me up > when it came time to build on a near-future fedora where > python isn't installed by default and I only wanted to > require python3 for the build/runtime scripts. I'm not actually sure those svn bits involving python are usable by real users. The main tool that people us for svn interop is git-svn, which is written in perl. There was an effort to have a real "git-remote-svn" helper (so that you could just seamlessly "git clone svn://..." instead of using the "git svn" wrapper. So we have the work in vcs-svn, which gets built as part of a regular compile. But AFAICT it is only used for git-remote-testsvn, t/helper/test-svn-fe, and contrib/svn-fe. None of which have seen any substantive work since 2012[1]. I suppose it's possible somebody could be using "git clone testsvn://" in the wild, but the name would hopefully warn them off. I have no idea how usable that work is in practice. -Peff [1] Browsing "git log", most of the commits are just tree-wide cleanups, or fixing some compilation or dependency error. The last "real" commit seems to be around 8e43a1d010 (remote-svn: add incremental import, 2012-09-19). I wonder if it is worth dropping this experiment as incomplete and unmaintained. People have obviously spent time dealing with the code for various fixups, but I think the whole thing may essentially just be dead code. Or maybe people really are using contrib/svn-fe. Like I said, I have no clue if this stuff is even usable, but we certainly haven't packaged it to be seen by most users.
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On Thu, Jun 07, 2018 at 01:16:14AM +0100, Ramsay Jones wrote: > > Probably. We may want to go the same route as we did for perl in > > a0e0ec9f7d (t: provide a perl() function which uses $PERL_PATH, > > 2013-10-28) so that test writers don't have to remember this. > > > > That said, I wonder if it would be hard to simply do the python bits > > here in perl. This is the first use of python in our test scripts (and > > Hmm, not quite the _first_ use: > > $ git grep PYTHON_PATH -- t > t/lib-git-p4.sh:(cd / && "$PYTHON_PATH" -c 'import time; > print(int(time.time()))') > t/lib-git-p4.sh:"$PYTHON_PATH" "$TRASH_DIRECTORY/marshal-dump.py" > t/t9020-remote-svn.sh:export PATH PYTHON_PATH GIT_BUILD_DIR > t/t9020-remote-svn.sh:exec "$PYTHON_PATH" > "$GIT_BUILD_DIR/contrib/svn-fe/svnrdump_sim.py" "$@" > t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" > "$TRASH_DIRECTORY/k_smush.py" <"$cli/k-text-k" >cli-k-text-k-smush && > t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" > "$TRASH_DIRECTORY/ko_smush.py" <"$cli/k-text-ko" >cli-k-text-ko-smush && > t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" > "$TRASH_DIRECTORY/gendouble.py" >%double.png && > t/t9810-git-p4-rcs.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/scrub_k.py" > <"$git/$file" >"$scrub" && > t/t9810-git-p4-rcs.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/scrub_ko.py" > <"$git/$file" >"$scrub" && > $ OK, the first for a feature that is not already written in python (leading into my second claim that python is used only for fringe commands ;) ). Though maybe I am wrong that the remote-svn stuff requires python. I thought it did, but poking around, it looks like it's all C, and just the "svnrdump_sim" helper is python. At any rate, I think the point still stands that perl is our main scripting language. I'd rather keep us to that unless there's a good reason not to. -Peff
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
Ramsay Jones wrote: [...] > I don't run the p4 or svn tests, so ... :-D Heh, lucky you. :) I try to run them all as part of the fedora builds since they cover much more than I'd ever use. That's the main reason I noticed the bare python. That would trip me up when it came time to build on a near-future fedora where python isn't installed by default and I only wanted to require python3 for the build/runtime scripts. > On 06/06/18 22:03, Jeff King wrote: >> really the only user in the whole code base outside of a few fringe >> commands). Leaving aside any perl vs python flame-war, I think there's >> value in keeping the number of languages limited when there's not a >> compelling reason to do otherwise. > > I agree that fewer languages is (generally) a good idea. Yep, that's certainly even better and if Jeff H. can use perl relatively easily (or one of the many perl folks here can help with that part of the test), that's great. The best way to solve many problems is avoid having them. :) Thanks, -- Todd ~~ Chaos, panic, and disorder - my job is done here.
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On 06/06/18 22:03, Jeff King wrote: > On Wed, Jun 06, 2018 at 01:10:52PM -0400, Todd Zullinger wrote: > >> g...@jeffhostetler.com wrote: >>> +# As a sanity check, ask Python to parse our generated JSON. Let Python >>> +# recursively dump the resulting dictionary in sorted order. Confirm that >>> +# that matches our expectations. >>> +test_expect_success PYTHON 'parse JSON using Python' ' >> [...] >>> + python "$TEST_DIRECTORY"/t0019/parse_json_1.py actual && >> >> Would this be better using $PYTHON_PATH rather than >> hard-coding python as the command? > > Probably. We may want to go the same route as we did for perl in > a0e0ec9f7d (t: provide a perl() function which uses $PERL_PATH, > 2013-10-28) so that test writers don't have to remember this. > > That said, I wonder if it would be hard to simply do the python bits > here in perl. This is the first use of python in our test scripts (and Hmm, not quite the _first_ use: $ git grep PYTHON_PATH -- t t/lib-git-p4.sh:(cd / && "$PYTHON_PATH" -c 'import time; print(int(time.time()))') t/lib-git-p4.sh:"$PYTHON_PATH" "$TRASH_DIRECTORY/marshal-dump.py" t/t9020-remote-svn.sh:export PATH PYTHON_PATH GIT_BUILD_DIR t/t9020-remote-svn.sh:exec "$PYTHON_PATH" "$GIT_BUILD_DIR/contrib/svn-fe/svnrdump_sim.py" "$@" t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/k_smush.py" <"$cli/k-text-k" >cli-k-text-k-smush && t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/ko_smush.py" <"$cli/k-text-ko" >cli-k-text-ko-smush && t/t9802-git-p4-filetype.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/gendouble.py" >%double.png && t/t9810-git-p4-rcs.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/scrub_k.py" <"$git/$file" >"$scrub" && t/t9810-git-p4-rcs.sh: "$PYTHON_PATH" "$TRASH_DIRECTORY/scrub_ko.py" <"$git/$file" >"$scrub" && $ I don't run the p4 or svn tests, so ... :-D > really the only user in the whole code base outside of a few fringe > commands). Leaving aside any perl vs python flame-war, I think there's > value in keeping the number of languages limited when there's not a > compelling reason to do otherwise. I agree that fewer languages is (generally) a good idea. ATB, Ramsay Jones
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On 6/6/2018 5:03 PM, Jeff King wrote: On Wed, Jun 06, 2018 at 01:10:52PM -0400, Todd Zullinger wrote: g...@jeffhostetler.com wrote: +# As a sanity check, ask Python to parse our generated JSON. Let Python +# recursively dump the resulting dictionary in sorted order. Confirm that +# that matches our expectations. +test_expect_success PYTHON 'parse JSON using Python' ' [...] + python "$TEST_DIRECTORY"/t0019/parse_json_1.py actual && Would this be better using $PYTHON_PATH rather than hard-coding python as the command? Probably. We may want to go the same route as we did for perl in a0e0ec9f7d (t: provide a perl() function which uses $PERL_PATH, 2013-10-28) so that test writers don't have to remember this. That said, I wonder if it would be hard to simply do the python bits here in perl. This is the first use of python in our test scripts (and really the only user in the whole code base outside of a few fringe commands). Leaving aside any perl vs python flame-war, I think there's value in keeping the number of languages limited when there's not a compelling reason to do otherwise. Perl works too. (I don't know perl, but I'm told it does work for stuff like this. :-) I'll take a stab at it. Jeff
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On 6/6/2018 1:10 PM, Todd Zullinger wrote: g...@jeffhostetler.com wrote: +# As a sanity check, ask Python to parse our generated JSON. Let Python +# recursively dump the resulting dictionary in sorted order. Confirm that +# that matches our expectations. +test_expect_success PYTHON 'parse JSON using Python' ' [...] + python "$TEST_DIRECTORY"/t0019/parse_json_1.py actual && Would this be better using $PYTHON_PATH rather than hard-coding python as the command? good catch! thanks Jeff
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
On Wed, Jun 06, 2018 at 01:10:52PM -0400, Todd Zullinger wrote: > g...@jeffhostetler.com wrote: > > +# As a sanity check, ask Python to parse our generated JSON. Let Python > > +# recursively dump the resulting dictionary in sorted order. Confirm that > > +# that matches our expectations. > > +test_expect_success PYTHON 'parse JSON using Python' ' > [...] > > + python "$TEST_DIRECTORY"/t0019/parse_json_1.py actual && > > Would this be better using $PYTHON_PATH rather than > hard-coding python as the command? Probably. We may want to go the same route as we did for perl in a0e0ec9f7d (t: provide a perl() function which uses $PERL_PATH, 2013-10-28) so that test writers don't have to remember this. That said, I wonder if it would be hard to simply do the python bits here in perl. This is the first use of python in our test scripts (and really the only user in the whole code base outside of a few fringe commands). Leaving aside any perl vs python flame-war, I think there's value in keeping the number of languages limited when there's not a compelling reason to do otherwise. -Peff
Re: [PATCH v7 2/2] json-writer: t0019: add Python unit test
g...@jeffhostetler.com wrote: > +# As a sanity check, ask Python to parse our generated JSON. Let Python > +# recursively dump the resulting dictionary in sorted order. Confirm that > +# that matches our expectations. > +test_expect_success PYTHON 'parse JSON using Python' ' [...] > + python "$TEST_DIRECTORY"/t0019/parse_json_1.py actual && Would this be better using $PYTHON_PATH rather than hard-coding python as the command? -- Todd ~~ Sometimes I think I understand everything, then I regain consciousness.
[PATCH v7 2/2] json-writer: t0019: add Python unit test
From: Jeff Hostetler
Test json-writer output using Python.
Signed-off-by: Jeff Hostetler
---
t/t0019-json-writer.sh | 38 ++
t/t0019/parse_json_1.py | 35 +++
2 files changed, 73 insertions(+)
create mode 100644 t/t0019/parse_json_1.py
diff --git a/t/t0019-json-writer.sh b/t/t0019-json-writer.sh
index c9c2e23..951cd89 100755
--- a/t/t0019-json-writer.sh
+++ b/t/t0019-json-writer.sh
@@ -233,4 +233,42 @@ test_expect_success 'inline array with no members' '
test_cmp expect actual
'
+# As a sanity check, ask Python to parse our generated JSON. Let Python
+# recursively dump the resulting dictionary in sorted order. Confirm that
+# that matches our expectations.
+test_expect_success PYTHON 'parse JSON using Python' '
+ cat >expect <<-\EOF &&
+ a abc
+ b 42
+ sub1 dict
+ sub1.c 3.14
+ sub1.d True
+ sub1.sub2 list
+ sub1.sub2[0] False
+ sub1.sub2[1] dict
+ sub1.sub2[1].g 0
+ sub1.sub2[1].h 1
+ sub1.sub2[2] None
+ EOF
+ test-json-writer >output.json \
+ @object \
+ @object-string a abc \
+ @object-int b 42 \
+ @object-object "sub1" \
+ @object-double c 2 3.140 \
+ @object-true d \
+ @object-array "sub2" \
+ @array-false \
+ @array-object \
+ @object-int g 0 \
+ @object-int h 1 \
+ @end \
+ @array-null \
+ @end \
+ @end \
+ @end &&
+ python "$TEST_DIRECTORY"/t0019/parse_json_1.py actual &&
+ test_cmp expect actual
+'
+
test_done
diff --git a/t/t0019/parse_json_1.py b/t/t0019/parse_json_1.py
new file mode 100644
index 000..9d928a3
--- /dev/null
+++ b/t/t0019/parse_json_1.py
@@ -0,0 +1,35 @@
+import os
+import sys
+import json
+
+def dump_item(label_input, v):
+if type(v) is dict:
+print("%s dict" % (label_input))
+dump_dict(label_input, v)
+elif type(v) is list:
+print("%s list" % (label_input))
+dump_list(label_input, v)
+else:
+print("%s %s" % (label_input, v))
+
+def dump_list(label_input, list_input):
+ix = 0
+for v in list_input:
+label = ("%s[%d]" % (label_input, ix))
+dump_item(label, v)
+ix += 1
+return
+
+def dump_dict(label_input, dict_input):
+for k in sorted(dict_input.iterkeys()):
+v = dict_input[k]
+if (len(label_input) > 0):
+label = ("%s.%s" % (label_input, k))
+else:
+label = k
+dump_item(label, v)
+return
+
+for line in sys.stdin:
+data = json.loads(line)
+dump_dict("", data)
--
2.9.3
