On Fri, Jan 19 2024, [email protected] wrote: > On 19.01.2024 13:50, Ihor Radchenko wrote: >> [email protected] writes: >> >>> This patch addresses a number of glitches discovered during >>> interactive >>> testing and contains possible fixes (sub-patch test-ob-maxima fixes >>> batch testing on my system, but not interactive testing). >> >> Thanks for your interest in improving Org mode tests! >> >>> The test-ob-maxima patch may be due to a different Maxima version, but >>> I do not think so. In batch testing test-ob-maxima passes contrary to >>> interactive mode. >> >> On my side, Maxima tests pass both interactively and non-interactively. >> I use Maxima 5.47.0 >> What is your version? > The same: maxima --version > Maxima 5.47.0 > > In batch testing without the patch I get: > > Test ob-maxima/batch+verbatim condition: > (ert-test-failed > ((should > (equal (org-babel-execute-src-block) > "(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n > gamma(z + 1)")) > :form > (equal > "(linenum:0,\n(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n > gamma(z + 1)" > "(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n > gamma(z + 1)") > :value nil :explanation > (arrays-of-different-length 104 92 > "(linenum:0,\n(assume(z > > 0),integrate(exp(-t)*t^z,t,0,inf))\n > gamma(z + 1)" > "(assume(z > > 0),integrate(exp(-t)*t^z,t,0,inf))\n > gamma(z + 1)" > first-mismatch-at 1))) > FAILED 162/1252 ob-maxima/batch+verbatim (0.694916 sec) at > ../lisp/test-ob > -maxima.el:71 > > I do not know where the difference comes from. I am not a Maxima user > and not inclined to track this down unless you tell me it is important. >> >>> The test-ob-python patch unsets org-src-preserve-indentation which I >>> have set. >>> >>> The test-ob patch also unsets org-src-preserve-indentation. >> >>> The test-org patch also unsets org-src-preserve-indentation. This >>> reduces the number of failures to 2 while testing test-org >>> interactively. >> >> It is generally not expected that Org tests will pass when you try to >> run them on Emacs instance with your personal config. This is because >> you can have arbitrary customizations, hooks, or advices that change >> the >> Org mode defaults. >> >> let-binding a couple of variables cannot fix this general problem for >> all scenarios. So, I do not see any reason to accept patches that make >> the tests run with personal config of a single user. >> > > Before I made the patch, I had reverted changes that worked > interactively > in my personal config but caused failures with make test. > > Because of the Maxima test failure (also in batch), I was not sure that > all > tests pass on a reference system. > >> What might be useful, however, is making sure that tests do not fail >> interactively with a clean Emacs config (make repro). >> Check out testing/README file. >> > > I have a super-weird system where make repro opens Emacs with > as default-directory my home directory. So I have to make symbolic > links from my home to the testing and lisp directories. > But make test works. > > Because of testing my > 0001-org-babel-demarcate-block-split-using-element-API.patch > interactively in my personal config (before make test which takes > quite some time), I figured out that it would guess to suppress false > failures. >> >>> (ert-deftest ob-maxima/string-input () >>> "Test of string input" >>> (org-test-at-id "b5842ed4-8e8b-4b18-a1c9-cef006b6a6c8" >>> (org-babel-next-src-block 2) >>> - (should (equal "- sin(x)" (org-babel-execute-src-block))))) >>> + (should (equal '(("(linenum:0," "") ("-" "sin(x)")) >>> + (org-babel-execute-src-block))))) >> >> The test in question is testing the output of >> >> #+begin_src maxima :var fun="sin(x)" :var q=2 :results silent >> print(diff(fun, x, q))$ >> #+end_src >> >> It is certainly not expected that such code block returns "linenum:0". > > I would say that it is an unexplained difference, I still get (with > "linenum:0") : > > #+header: :wrap "src text -n" > #+name: lst:maxima-example > > #+begin_src maxima -n :batch batch :cmdline --quiet :exports both > :results raw > rat(1/(x+1) + x/(x-1)); > #+end_src > > #+name: lst:maxima-example-results > #+results: lst:maxima-example > > #+begin_src text -n > (%i1) (linenum:0, > (%i1) rat(1/(x+1)+x/(x-1)) > 2 > x + 2 x - 1 > (%o1)/R/ ------------ > 2 > x - 1 > #+end_src > >> >> Tests in Org mode are verifying that Org mode behaves correctly. It >> makes no sense to change the test when Org mode result is not correct - >> we should instead fix Org mode. >> >> In the majority of cases, if the test is failing, it is not something >> wrong with the test, but something wrong with Org mode code that test >> is >> verifying. >> >>> (ert-deftest test-ob/does-not-replace-a-block-with-the-results () >>> (org-test-with-temp-text "#+NAME: foo >>> diff --git a/testing/lisp/test-org-element.el >>> b/testing/lisp/test-org-element.el >>> index ca7d77e28..f0958ff59 100644 >>> --- a/testing/lisp/test-org-element.el >>> +++ b/testing/lisp/test-org-element.el >>> @@ -4457,7 +4457,7 @@ Text >>> '(paragraph nil " Two spaces\n\n \n Two spaces")) >>> '(paragraph nil "Two spaces\n\n\nTwo spaces"))) >>> (should >>> - (equal >>> + (equal ;; Fails in test. Why? (equal ...) form works in ielm. >> >> It would help if you provided the error ERT displayed when the test >> failed. > > I cannot reproduce it. Now it works in interactive testing (personal > config > as when I first saw this). > > If you ask me, I am still willing to collect the 2 sub-patches with the > let bindings which improve testing interactively in personal configs > somewhat. If not, it is OK (I understand now that it is not important). > > Regards -- Gerard
Re: test-ob-maxima.el There is a long thread from Dec 2024 where the problem with the fragment "(linenum," was discussed and resolved (see [1], which is in the middle of the thread). When I look at the patch to test-ob-maxima.el sent by Gerard in the current thread, it is clear the patch is trying to accommodate the appearance of that unwanted fragment. That was fixed in commit a628d46712 [2]. Thanks to Morgan Smith for bringing the current thread to my attention [3]. Leo [1] https://list.orgmode.org/orgmode/[email protected]/ [2] https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a628d46712 [3] https://list.orgmode.org/ch3pr84mb3424a1b8a69e3770ab9739ecc5...@ch3pr84mb3424.namprd84.prod.outlook.com/
