Achim Gratz <strom...@nexgo.de> writes: > Eric Schulte <eric.schu...@gmx.com> writes: >> Moving forward on this point, many of the existing tests explicitly >> `require' new Org-mode functionality (mainly language support for >> testing code blocks execution). I do not think that tests should ever >> be activating new packages changing a users global environment. > > Thanks. > > I've had to deactivate the inclusion of ./testing/lisp/contrib into the > test flow due to a recent removal of said directory by Nicolas (he's > probably doing that patch himself) and now get this: >
Yes, I talked with Nicolas this morning, and at my urging he moved everything from ./testing/lisp/contrib into ./testing/lisp and removed the contrib directory. I think moving forward we are best served by a single test directory, especially given that we already have good methods of conditionally loading only those tests appropriate for the users environment. > > lisp/org-mode> make test-dirty |& grep failed > failed 5/109 ob-exp/export-from-a-temp-buffer > failed 17/109 org-missing-dependency/test-ob-C > failed 18/109 org-missing-dependency/test-ob-R > failed 19/109 org-missing-dependency/test-ob-awk > failed 20/109 org-missing-dependency/test-ob-fortran > failed 21/109 org-missing-dependency/test-ob-lilypond > failed 22/109 org-missing-dependency/test-ob-maxima > failed 23/109 org-missing-dependency/test-ob-octave > failed 24/109 org-missing-dependency/test-ob-python > failed 25/109 org-missing-dependency/test-org-element > failed 26/109 org-missing-dependency/test-org-export > > This is expected, as my local configuration doesn't have any of these > packages activated. > Yes, I assume that the lower case "failed" means expected failure. > > lisp/org-mode> make test-dirty | & grep FAILED > FAILED 27/109 test-ob-exp/org-babel-exp-src-blocks/w-no-file > FAILED 62/109 test-org-babel/get-src-block-info-body > FAILED 63/109 test-org-babel/get-src-block-info-language > FAILED 64/109 test-org-babel/get-src-block-info-tangle > > This is unexpected and I don't really see where they are coming from. > The backtrace always starts at (regexp-quote org-test-file-ob-anchor). > My guess is that when I removed require statements for org-html and org-ascii from from test-org-html and test-org-exp, they removed some functionality not loaded by default in the test scripts. These should probably be added into the Makefile. > >> For this reason I have just pushed up a commit which changes all >> >> (require 'org-foo) >> >> to >> >> (unless (featurep 'org-foo) >> (signal 'missing-test-dependency "Org support for doing foo.")) >> >> so that those tests simple aren't run on the users system. Please let >> me know if anyone thinks this is a mistake and we can discuss. The only >> drawback I see is that batch-mode scripts will have to explicitly >> activate the features which they would like to test, by evaluating forms >> like (require 'org-foo) before the call to the test suite -- and I would >> argue that being explicit about such things is a benefit. > > Well, currently all testing runs under my user settings, which is not > ideal for testing anyway. I suppose it would not be difficult to just > inject a different (minimal) startup file specifically for testing to > separate these issues. > Yea, I also run most tests from my normal Emacs session, which is part of the motivation for removing these embedded require statements. Cheers, > > > Regards, > Achim. -- Eric Schulte http://cs.unm.edu/~eschulte/