Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-04 Thread Achim Gratz
Sebastien Vauban
wxhgmqzgw...@spammotel.com writes:
 I don't understand why once there are 192 tests, once 130. I thought that the
 second figure was the total number of tests, hence should be stable over the
 test runs?

The number of available tests depend on the Babel languages that are
activated.  One of those failures was with a test run that had all
languages (except R) activated, the other one with no languages (except
emacs-lisp, which is always active), but org-element and org-export made
active.  Those two also seem to pull in ob-sh and maybe other Babel
languages, I would have to check.


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-04 Thread Achim Gratz
Nicolas Goaziou n.goaz...@gmail.com writes:
 I cannot reproduce any of them, interactively or in batch mode[1].

This is the invocation (to be run in bash, like make would do):

TMPDIR=/tmp/tmp-orgtest /usr/local/bin/emacs -batch -Q \
-L lisp/ -L testing/ -L contrib/lisp \
--eval '(defconst org-release 7.8.03-Test)' -l testing/org-test.el \
--eval '(require '''org-element)' --eval '(require '''org-export)' \
--eval '(setq org-confirm-babel-evaluate nil)' -f org-test-run-batch-tests

[...]
2 unexpected results:
   FAILED  test-org-export/export-scope
   FAILED  test-org-footnote/normalize-outside-org

 I'm not sure where they could come from.

Since I run this with -Q, I suspect that you implicitly rely on some
customization someplace.


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-04 Thread Nicolas Goaziou
Hello,

Achim Gratz strom...@nexgo.de writes:

 Nicolas Goaziou n.goaz...@gmail.com writes:
 I cannot reproduce any of them, interactively or in batch mode[1].

 This is the invocation (to be run in bash, like make would do):

 TMPDIR=/tmp/tmp-orgtest /usr/local/bin/emacs -batch -Q \
 -L lisp/ -L testing/ -L contrib/lisp \
 --eval '(defconst org-release 7.8.03-Test)' -l testing/org-test.el \
 --eval '(require '''org-element)' --eval '(require '''org-export)' \
 --eval '(setq org-confirm-babel-evaluate nil)' -f org-test-run-batch-tests

 [...]
 2 unexpected results:
FAILED  test-org-export/export-scope
FAILED  test-org-footnote/normalize-outside-org

With your command, I was able to reproduce the first one (along with 24
unexpected results), for which I have pushed a fix. I still don't see
the second one.


Regards,

-- 
Nicolas Goaziou



Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-04 Thread Achim Gratz
Nicolas Goaziou n.goaz...@gmail.com writes:
 [...]
 2 unexpected results:
FAILED  test-org-export/export-scope
FAILED  test-org-footnote/normalize-outside-org

 With your command, I was able to reproduce the first one (along with 24
 unexpected results), for which I have pushed a fix. I still don't see
 the second one.

Well, that is likely because you run the test on uncompiled source,
while I am running it on compiled orgmode... maybe one of these pesky
byte-compiler warnings shouldn't have been ignored.  I can't reproduce
the 24 unexpected results, but they probably have a similar origin
(partially compiled orgmode?).


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-04 Thread Nicolas Goaziou
Achim Gratz strom...@nexgo.de writes:

 Well, that is likely because you run the test on uncompiled source,
 while I am running it on compiled orgmode... maybe one of these pesky
 byte-compiler warnings shouldn't have been ignored.  I can't reproduce
 the 24 unexpected results, but they probably have a similar origin
 (partially compiled orgmode?).

It's possible. I have two different Org installations. One compiled for
my work, and the other, uncompiled for development. Both are latest git
version.

When batch running tests on my compiled Org, I get:

--8---cut here---start-8---
Ran 130 tests, 105 results as expected, 25 unexpected (2012-03-04 15:42:25+0100)
9 expected failures

25 unexpected results:
   FAILED  ob-emacs-lisp/commented-last-block-line-no-var
   FAILED  ob-emacs-lisp/commented-last-block-line-with-var
   FAILED  ob-exp/exports-both
   FAILED  ob-exp/mixed-blocks-with-exports-both
   FAILED  ob-exp/noweb-no-export-and-exports-both
   FAILED  ob-exp/noweb-on-export
   FAILED  ob-exp/noweb-on-export-with-exports-results
   FAILED  ob-exp/noweb-strip-export-ensure-strips
   FAILED  test-ob-lob/do-not-eval-lob-lines-in-example-blocks-on-export
   FAILED  test-ob-sh/dont-error-on-empty-results
   FAILED  test-ob/commented-last-block-line-no-var
   FAILED  test-ob/commented-last-block-line-with-var
   FAILED  test-ob/org-babel-remove-result--results-default
   FAILED  test-org-babel/combining-scalar-and-raw-result-types
   FAILED  test-org-babel/elisp-in-header-arguments
   FAILED  test-org-babel/inline-src-blocks
   FAILED  test-org-babel/inline-src_blk-default-results-replace-line-1
   FAILED  test-org-babel/just-one-results-block
   FAILED  test-org-babel/multi-line-header-arguments
   FAILED  test-org-babel/parse-header-args
   FAILED  test-org-babel/parse-header-args2
   FAILED  test-org-babel/simple-named-code-block
   FAILED  test-org-babel/simple-variable-resolution
   FAILED  test-org-footnote/normalize-outside-org
   FAILED  test-org-table/simple-formula
--8---cut here---end---8---

So, indeed, the second failing test appears.  It should pass now (at
least it does even on my compiled Org installation).

I think the others come from a file-error (Opening output file).

Thanks for your help.


Regards,

-- 
Nicolas Goaziou



Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-04 Thread Achim Gratz
Nicolas Goaziou n.goaz...@gmail.com writes:
 So, indeed, the second failing test appears.  It should pass now (at
 least it does even on my compiled Org installation).

I confirm the fix, thank you very much.

 I think the others come from a file-error (Opening output file).

I can see now why this happens: the temporary directory for the test
should already exist — make usually takes care of that, but running it
by hand does not.  So these failures should go away if you mkdir
/tmp/tmp-orgtest before starting the tests and you should also get a
clean test run.


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-03 Thread Achim Gratz
Achim Gratz strom...@nexgo.de writes:
  It all works now when adding testing/ to the load path.  I'll add
 this to the default configuration in my Makefile fork.

I've improved the test invocation in my Makefile fork.  It now starts
from a clean Emacs invocation withoutsite startup and loads the
configured Babel languages and possibly extra packages before running
the tests.  All Babel languages are activated by default except R, since
R seems to depend on ess (i.e. even if you add it to the list, the tests
still fail due to that missing dependency).  Extra packages, for
instance from contrib, can be required by adding their names to
BTEST_EXTRA in local.mk and possibly prepending or appending load-path
elements or other command line options via BTEST_PRE and BTEST_POST.


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-03 Thread Achim Gratz
Two test failures in current HEAD:

--8---cut here---start-8---
Test test-org-footnote/normalize-outside-org backtrace:
  org-footnote-in-valid-context-p()
  org-footnote-at-reference-p()
  byte-code(\20\304\202 \305\n\306#\204\307\310\311\\21\20
  org-footnote-get-next-reference()
  org-footnote-normalize()
  (let ((major-mode (quote message-mode))) (org-footnote-normalize))
  (progn (insert Body[fn::def]\n-- \nFake signature\n-- \nSignature)
  (unwind-protect (progn (insert Body[fn::def]\n-- \nFake signature\n
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
  (with-current-buffer temp-buffer (unwind-protect (progn (insert Bod
  (let ((temp-buffer (generate-new-buffer  *temp*))) (with-current-b
  (with-temp-buffer (insert Body[fn::def]\n-- \nFake signature\n-- \n
  (let ((org-footnote-tag-for-non-org-mode-files nil) (message-signatu
  (lambda nil (let ((org-footnote-tag-for-non-org-mode-files nil)) (wi
  byte-code(\306\307!▒q\210\310\216\311 \312\216\313\314\315\316\3
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  byte-code(\306\307!\211▒r\310\311!q\210\312 d\313\223)L\210\314\216
  ert-run-test([cl-struct-ert-test test-org-footnote/normalize-outside
  ert-run-or-rerun-test([cl-struct-ert--stats \\(org\\|ob\\) [[cl-st
  ert-run-tests(\\(org\\|ob\\) #[(event-type rest event-args) \30
  ert-run-tests-batch(\\(org\\|ob\\))
  ert-run-tests-batch-and-exit(\\(org\\|ob\\))
  (let ((org-id-track-globally t) (org-id-locations-file (convert-stan
  org-test-run-batch-tests()
  call-interactively(org-test-run-batch-tests nil nil)
  command-execute(org-test-run-batch-tests)
  command-line-1((-L lisp/ -L testing/ --eval (defconst org
  command-line()
  normal-top-level()
Test test-org-footnote/normalize-outside-org condition:
(void-variable message-cite-prefix-regexp)
   FAILED  166/192  test-org-footnote/normalize-outside-org
--8---cut here---end---8---

--8---cut here---start-8---
Test test-org-export/export-scope backtrace:
  signal(ert-test-failed (((should (equal (org-export-as (quote test))
  ert-fail(((should (equal (org-export-as (quote test)) text\n)) :fo
  (if (unwind-protect (setq value-2005 (apply fn-2003 args-2004)) (set
  (unless (unwind-protect (setq value-2005 (apply fn-2003 args-2004)) 
  (let (form-description-2007) (unless (unwind-protect (setq value-200
  (let ((value-2005 (quote ert-form-evaluation-aborted-2006))) (let (f
  (let ((fn-2003 (function equal)) (args-2004 (list (org-export-as (qu
  (should (equal (org-export-as (quote test)) text\n))
  (progn (fset (quote org-test-center-block) (function* (lambda (obj c
  (unwind-protect (progn (fset (quote org-test-center-block) (function
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-center-block))) (
  (letf (((symbol-function (quote org-test-center-block)) (function* (
  (progn (fset (quote org-test-comment) (function* (lambda (obj conten
  (unwind-protect (progn (fset (quote org-test-comment) (function* (la
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-comment))) (--cl-
  (letf (((symbol-function (quote org-test-comment)) (function* (lambd
  (progn (fset (quote org-test-comment-block) (function* (lambda (obj 
  (unwind-protect (progn (fset (quote org-test-comment-block) (functio
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-comment-block))) 
  (letf (((symbol-function (quote org-test-comment-block)) (function* 
  (progn (fset (quote org-test-drawer) (function* (lambda (obj content
  (unwind-protect (progn (fset (quote org-test-drawer) (function* (lam
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-drawer))) (--cl-l
  (letf (((symbol-function (quote org-test-drawer)) (function* (lambda
  (progn (fset (quote org-test-dynamic-block) (function* (lambda (obj 
  (unwind-protect (progn (fset (quote org-test-dynamic-block) (functio
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-dynamic-block))) 
  (letf (((symbol-function (quote org-test-dynamic-block)) (function* 
  (progn (fset (quote org-test-example-block) (function* (lambda (obj 
  (unwind-protect (progn (fset (quote org-test-example-block) (functio
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-example-block))) 
  (letf (((symbol-function (quote org-test-example-block)) (function* 
  (progn (fset (quote org-test-export-block) (function* (lambda (obj c
  (unwind-protect (progn (fset (quote org-test-export-block) (function
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-export-block))) (
  (letf (((symbol-function (quote org-test-export-block)) (function* (
  (progn (fset (quote org-test-fixed-width) (function* (lambda (obj co
  (unwind-protect (progn (fset (quote org-test-fixed-width) (function*
  (let* ((--cl-letf-bound-- (fboundp (quote org-test-fixed-width))) (-
  (letf (((symbol-function (quote org-test-fixed-width)) (function* (l
  (progn (fset (quote org-test-footnote-definition) (function* 

Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-03 Thread Nicolas Goaziou
Hello,

Achim Gratz strom...@nexgo.de writes:

 Two test failures in current HEAD:

I cannot reproduce any of them, interactively or in batch mode[1].

I'm not sure where they could come from.


Regards,

[1] Though, I have 8 unexpected failures there, but not related to
those described here.

-- 
Nicolas Goaziou



Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-03 Thread Sebastien Vauban
Hello Achim,

Achim Gratz wrote:
 Two test failures in current HEAD:

 Test test-org-footnote/normalize-outside-org backtrace:
   org-footnote-in-valid-context-p()
   [...]
   normal-top-level()
 Test test-org-footnote/normalize-outside-org condition:
 (void-variable message-cite-prefix-regexp)
FAILED  166/192  test-org-footnote/normalize-outside-org

 Test test-org-export/export-scope backtrace:
   signal(ert-test-failed (((should (equal (org-export-as (quote test))
   [...]
   normal-top-level()
 Test test-org-export/export-scope condition:
 (ert-test-failed
  ((should
(equal
 (org-export-as ...)
 text\n))
   :form
   (equal * Head1\n** Head2\ntext\n*** Head3\n text\n)
   :value nil :explanation
   (arrays-of-different-length 32 5 * Head1\n** Head2\ntext\n*** Head3\n 
 text\n first-mismatch-at 0)))
FAILED   95/130  test-org-export/export-scope

I don't understand why once there are 192 tests, once 130. I thought that the
second figure was the total number of tests, hence should be stable over the
test runs?

Best regards,
  Seb

-- 
Sebastien Vauban




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-02 Thread Nicolas Goaziou
Hello,

Achim Gratz strom...@nexgo.de writes:

 There's another patch by Nicolas that added a few test, one of which
 seems broken:

This should be fixed now. Thank you for reporting it.


Regards,

-- 
Nicolas Goaziou



Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-02 Thread Achim Gratz
Nicolas Goaziou n.goaz...@gmail.com writes:
 This should be fixed now. Thank you for reporting it.

I confirm the fix, testing is clean again, thank you very much.


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-03-01 Thread Achim Gratz
Eric Schulte eric.schu...@gmx.com writes:
[...]

Thanks for your latest round of patches.  It all works now when adding
testing/ to the load path.  I'll add this to the default configuration
in my Makefile fork.  There's another patch by Nicolas that added a few
test, one of which seems broken:

OVERVIEW
Mark set
Test test-org-export/fuzzy-links backtrace:
  signal(error (No link found))
  error(No link found)
  org-open-at-point()
  (prog1 (goto-line 4) (org-open-at-point) (should (looking-at Test
  (progn (org-mode) (progn (insert #+TARGET: Test\n#+NAME: Test\n|a|b
  (unwind-protect (progn (org-mode) (progn (insert #+TARGET: Test\n#+
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
  (with-current-buffer temp-buffer (unwind-protect (progn (org-mode) (
  (let ((temp-buffer (generate-new-buffer  *temp*))) (with-current-b
  (with-temp-buffer (org-mode) (progn (insert #+TARGET: Test\n#+NAME:
  (org-test-with-temp-text #+TARGET: Test\n#+NAME: Test\n|a|b|\nTes
  (lambda nil (org-test-with-temp-text #+TARGET: Test\n#+NAME: Test\n
  byte-code(\306\307!▒q\210\310\216\311 \312\216\313\314\315\316\3
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  byte-code(\306\307!\211▒r\310\311!q\210\312 d\313\223)L\210\314\216
  ert-run-test([cl-struct-ert-test test-org-export/fuzzy-links Test f
  ert-run-or-rerun-test([cl-struct-ert--stats \\(org\\|ob\\) [[cl-st
  ert-run-tests(\\(org\\|ob\\) #[(event-type rest event-args) \30
  ert-run-tests-batch(\\(org\\|ob\\))
  ert-run-tests-batch-and-exit(\\(org\\|ob\\))
  (let ((org-id-track-globally t) (org-id-locations-file (convert-stan
  org-test-run-batch-tests()
  call-interactively(org-test-run-batch-tests nil nil)
  command-execute(org-test-run-batch-tests)
  command-line-1((-L testing/ -L lisp/ --eval (defconst org
  command-line()
  normal-top-level()
Test test-org-export/fuzzy-links condition:
(error No link found)
   FAILED   87/112  test-org-export/fuzzy-links




Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-02-29 Thread Eric Schulte
Moving forward on this point, many of the existing tests explicitly
`require' new Org-mode functionality (mainly language support for
testing code blocks execution).  I do not think that tests should ever
be activating new packages changing a users global environment.

For this reason I have just pushed up a commit which changes all

  (require 'org-foo)

to

  (unless (featurep 'org-foo)
(signal 'missing-test-dependency Org support for doing foo.))

so that those tests simple aren't run on the users system.  Please let
me know if anyone thinks this is a mistake and we can discuss.  The only
drawback I see is that batch-mode scripts will have to explicitly
activate the features which they would like to test, by evaluating forms
like (require 'org-foo) before the call to the test suite -- and I would
argue that being explicit about such things is a benefit.

Cheers,

Achim Gratz strom...@nexgo.de writes:

 Tests for experimental org features (e.g. from contrib/ ) should expect
 to fail when the user has not configured their inclusion into the
 current setup.  In other words, things like (require org-element)
 should not break the test run, but instead just note that this test has
 failed expectedly and continue testing.


 Regards,
 Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/



Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-02-29 Thread Achim Gratz
Eric Schulte eric.schu...@gmx.com writes:
 Moving forward on this point, many of the existing tests explicitly
 `require' new Org-mode functionality (mainly language support for
 testing code blocks execution).  I do not think that tests should ever
 be activating new packages changing a users global environment.

Thanks.

I've had to deactivate the inclusion of ./testing/lisp/contrib into the
test flow due to a recent removal of said directory by Nicolas (he's
probably doing that patch himself) and now get this:

lisp/org-mode make test-dirty | grep failed
   failed5/109  ob-exp/export-from-a-temp-buffer
   failed   17/109  org-missing-dependency/test-ob-C
   failed   18/109  org-missing-dependency/test-ob-R
   failed   19/109  org-missing-dependency/test-ob-awk
   failed   20/109  org-missing-dependency/test-ob-fortran
   failed   21/109  org-missing-dependency/test-ob-lilypond
   failed   22/109  org-missing-dependency/test-ob-maxima
   failed   23/109  org-missing-dependency/test-ob-octave
   failed   24/109  org-missing-dependency/test-ob-python
   failed   25/109  org-missing-dependency/test-org-element
   failed   26/109  org-missing-dependency/test-org-export

This is expected, as my local configuration doesn't have any of these
packages activated.

lisp/org-mode make test-dirty |  grep FAILED
   FAILED   27/109  test-ob-exp/org-babel-exp-src-blocks/w-no-file
   FAILED   62/109  test-org-babel/get-src-block-info-body
   FAILED   63/109  test-org-babel/get-src-block-info-language
   FAILED   64/109  test-org-babel/get-src-block-info-tangle

This is unexpected and I don't really see where they are coming from.
The backtrace always starts at (regexp-quote org-test-file-ob-anchor).

 For this reason I have just pushed up a commit which changes all

   (require 'org-foo)

 to

   (unless (featurep 'org-foo)
 (signal 'missing-test-dependency Org support for doing foo.))

 so that those tests simple aren't run on the users system.  Please let
 me know if anyone thinks this is a mistake and we can discuss.  The only
 drawback I see is that batch-mode scripts will have to explicitly
 activate the features which they would like to test, by evaluating forms
 like (require 'org-foo) before the call to the test suite -- and I would
 argue that being explicit about such things is a benefit.

Well, currently all testing runs under my user settings, which is not
ideal for testing anyway.  I suppose it would not be difficult to just
inject a different (minimal) startup file specifically for testing to
separate these issues.


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada




Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-02-29 Thread Eric Schulte
Achim Gratz strom...@nexgo.de writes:

 Eric Schulte eric.schu...@gmx.com writes:
 Moving forward on this point, many of the existing tests explicitly
 `require' new Org-mode functionality (mainly language support for
 testing code blocks execution).  I do not think that tests should ever
 be activating new packages changing a users global environment.

 Thanks.

 I've had to deactivate the inclusion of ./testing/lisp/contrib into the
 test flow due to a recent removal of said directory by Nicolas (he's
 probably doing that patch himself) and now get this:


Yes,

I talked with Nicolas this morning, and at my urging he moved everything
from ./testing/lisp/contrib into ./testing/lisp and removed the contrib
directory.  I think moving forward we are best served by a single test
directory, especially given that we already have good methods of
conditionally loading only those tests appropriate for the users
environment.


 lisp/org-mode make test-dirty | grep failed
failed5/109  ob-exp/export-from-a-temp-buffer
failed   17/109  org-missing-dependency/test-ob-C
failed   18/109  org-missing-dependency/test-ob-R
failed   19/109  org-missing-dependency/test-ob-awk
failed   20/109  org-missing-dependency/test-ob-fortran
failed   21/109  org-missing-dependency/test-ob-lilypond
failed   22/109  org-missing-dependency/test-ob-maxima
failed   23/109  org-missing-dependency/test-ob-octave
failed   24/109  org-missing-dependency/test-ob-python
failed   25/109  org-missing-dependency/test-org-element
failed   26/109  org-missing-dependency/test-org-export

 This is expected, as my local configuration doesn't have any of these
 packages activated.


Yes, I assume that the lower case failed means expected failure.


 lisp/org-mode make test-dirty |  grep FAILED
FAILED   27/109  test-ob-exp/org-babel-exp-src-blocks/w-no-file
FAILED   62/109  test-org-babel/get-src-block-info-body
FAILED   63/109  test-org-babel/get-src-block-info-language
FAILED   64/109  test-org-babel/get-src-block-info-tangle

 This is unexpected and I don't really see where they are coming from.
 The backtrace always starts at (regexp-quote org-test-file-ob-anchor).


My guess is that when I removed require statements for org-html and
org-ascii from from test-org-html and test-org-exp, they removed some
functionality not loaded by default in the test scripts.  These should
probably be added into the Makefile.


 For this reason I have just pushed up a commit which changes all

   (require 'org-foo)

 to

   (unless (featurep 'org-foo)
 (signal 'missing-test-dependency Org support for doing foo.))

 so that those tests simple aren't run on the users system.  Please let
 me know if anyone thinks this is a mistake and we can discuss.  The only
 drawback I see is that batch-mode scripts will have to explicitly
 activate the features which they would like to test, by evaluating forms
 like (require 'org-foo) before the call to the test suite -- and I would
 argue that being explicit about such things is a benefit.

 Well, currently all testing runs under my user settings, which is not
 ideal for testing anyway.  I suppose it would not be difficult to just
 inject a different (minimal) startup file specifically for testing to
 separate these issues.


Yea, I also run most tests from my normal Emacs session, which is part
of the motivation for removing these embedded require statements.

Cheers,



 Regards,
 Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/



Re: [O] [Bug] Tests for experimental org-features should expect to fail if not activated by the user

2012-02-29 Thread Achim Gratz
Eric Schulte eric.schu...@gmx.com writes:
 My guess is that when I removed require statements for org-html and
 org-ascii from from test-org-html and test-org-exp, they removed some
 functionality not loaded by default in the test scripts.  These should
 probably be added into the Makefile.

Unless someone can give me a hint, I'll have a look at this
later... it'll probably become weekend before I get to this.

 Yea, I also run most tests from my normal Emacs session, which is part
 of the motivation for removing these embedded require statements.

I'll see that I create a minimal config and use that just for testing
from the Makefile (in my Makefile fork of course).  If somebody has a
good template to work from, please post, otherwise I should be able to
scrape it together from the parts that have just been removed.  BTW,
where should that startup file go, directly under testing?


Regards,
Achim.
-- 
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra