Re: ctest fails standard exports of a number of examples and templates files

[Guenter Milde]

On 2017-02-20, Scott Kostyshak wrote:

> [-- Type: text/plain, Encoding: quoted-printable --]

> On Mon, Feb 20, 2017 at 06:40:26PM +0100, Jean-Pierre Chrétien wrote:

>> Sure, but that means a lot of test to be moved to unreliable:nonstandard.

Not moved but added. In contrast to "invert", adding tests to "unreliable"
has no side-effect besides them getting a tag.

The tag is there to tell developers that these are "optional" tests.

The test suite is OK, if all reliable tests pass, i.e. if

   ctests -LE unreliable
   
does not report failures.


Günter   

Re: ctest fails standard exports of a number of examples and templates files

[Jean-Pierre Chrétien]

Le 21/02/2017 à 00:10, Kornel Benko a écrit :
.



OK, but that was before I imported the necessary packages from the publishers
who are not in TeXLive 2016.


It is even better now. I was in the process to install iucr package, and in the 
process
I also install data from the iucr/utilities directory. This dir contains _old_ 
float.sty which led
to errors like not having floatstyle "plaintop".


I do not have this subdir:

$ ls /usr/local/texlive/texmf-local/tex/latex/IUCr/
abstemplate.ltxdocumentation.ltx  fig1.psiucr.cls 
README template.ltx


and my float.sty is tha standard one:

$ kpsewhich float.sty
/usr/local/texlive/2016/texmf-dist/tex/latex/float/float.sty

Aha, I did not mention that I run ctest on branch, not on master.



OK. Looking into my texmf-local/tex/latex dir, I have 35 extra packages outside 
of texlive installed. And also plenty of fonts.
From them 17 files have been used today in export test. All of them from 
'culmus'.


I will see this, maybe it is the reason for the pdf2 hebrew failures.

--
Jean-Pierre

Re: ctest fails standard exports of a number of examples and templates files

[Kornel Benko]

Am Montag, 20. Februar 2017 um 23:27:26, schrieb Jean-Pierre Chrétien 

> Le 20/02/2017 à 20:13, Kornel Benko a écrit :
> > Am Montag, 20. Februar 2017 um 18:40:26, schrieb Jean-Pierre Chrétien 
> > 
> > ...
> >>>   Here are the resources which should be installed to
> >>>   avoid failure of tests with the "unreliable:nonstandard" label:
> >>>
> >>> Do you agree?
> >>
> >> Sure, but that means a lot of test to be moved to unreliable:nonstandard.
> >> To make things clearer,  attached is a file with the ctest command I ran 
> >> and the
> >> list of failures I got.
> >>
> >> The command runs ctest excluding export tests other than pdf and pdf2 on a 
> >> list
> >> of files which contain one or more maths option 0 (i. e. do not load by 
> >> default).
> >
> > That is a big list.
> > Using the same ctest command here I see only 21 fails. One of them
> > actually never ends (export/templates/poster-a0poster-colored._pdf2).
> > What’s more, my list does not match any of elements in your list.
> >
...

> OK, but that was before I imported the necessary packages from the publishers 
> who are not in TeXLive 2016.

It is even better now. I was in the process to install iucr package, and in the 
process
I also install data from the iucr/utilities directory. This dir contains _old_ 
float.sty which led
to errors like not having floatstyle "plaintop".

Now all my exports compile without problems.

> After importing wha is needed I get this:
> 
> 306:export/doc/Additional_pdf
> 307:export/doc/Additional_pdf2
> 737:export/doc/de/Additional_pdf
> 738:export/doc/de/Additional_pdf2
> 917:export/doc/es/Additional_pdf
> 918:export/doc/es/Additional_pdf2
> 1112:export/doc/fr/Additiontaeal_pdf
> 1113:export/doc/fr/Additional_pdf2
> 1307:export/doc/he/Intro_pdf
> 1308:export/doc/he/Intro_pdf2
> 1322:export/doc/he/Tutorial_pdf
> 1323:export/doc/he/Tutorial_pdf2
> 1897:export/examples/Literate_lyx21
> 1904:export/examples/Literate_pdf
> 1905:export/examples/Literate_pdf2
> 2111:export/examples/docbook_article_pdf
> 2176:export/examples/knitr_lyx16
> 2177:export/examples/knitr_lyx21
> 2184:export/examples/knitr_pdf
> 2185:export/examples/knitr_pdf2
> 2206:export/examples/lilypond_lyx16
> 2207:export/examples/lilypond_lyx21
> 2214:export/examples/lilypond_pdf
> 2215:export/examples/lilypond_pdf2
> 2236:export/examples/listerrors_lyx16
> 2237:export/examples/listerrors_lyx21
> 2241:export/examples/listerrors_pdf
> 2278:export/examples/noweb2lyx_lyx21
> 2285:export/examples/noweb2lyx_pdf
> 2286:export/examples/noweb2lyx_pdf2
> 2769:export/examples/sweave_lyx16
> 2770:export/examples/sweave_lyx21
> 2777:export/examples/sweave_pdf
> 2778:export/examples/sweave_pdf2
> 3460:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf
> 3461:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf2
> 3754:export/examples/he/example_lyxified_pdf
> 3755:export/examples/he/example_lyxified_pdf2
> 3769:export/examples/he/example_raw_pdf
> 3770:export/examples/he/example_raw_pdf2
> 3784:export/examples/he/splash_pdf
> 3785:export/examples/he/splash_pdf2
> 3916:export/examples/ja/knitr_lyx16
> 3917:export/examples/ja/knitr_lyx21
> 3923:export/examples/ja/lilypond_lyx16
> 3924:export/examples/ja/lilypond_lyx21
> 3943:export/examples/ja/sweave_lyx16
> 3944:export/examples/ja/sweave_lyx21
> 4337:export/templates/AGUTeXtae_pdf
> 4338:export/templates/AGUTeX_pdf2
> 4379:export/templates/DocBook_article_pdf
> 4533:export/templates/RJournal_lyx16
> 4534:export/templates/RJournal_lyx21
> 4541:export/templates/RJournal_pdf
> 4542:export/templates/RJournal_pdf2
> 4795:UNRELIABLE.NONSTANDARD_export/templates/ja_beamer-conference-ornate-20min_pdf2
> 

Still many ...

> So
>   - you can export hebrew docs in pdf2, I can't (I need system fonts);
>   - you can export the various literate programming examples and templates in 
> pdf2, I can't either (I can only in xhtml, which is the sole export method 
> allowed if I try by hand).
> 
> I will investigate later, I'll be away for a while.
> 

OK. Looking into my texmf-local/tex/latex dir, I have 35 extra packages outside 
of texlive installed. And also plenty of fonts.
From them 17 files have been used today in export test. All of them from 
'culmus'.

Kornel

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: ctest fails standard exports of a number of examples and templates files

[Scott Kostyshak]

On Mon, Feb 20, 2017 at 06:40:26PM +0100, Jean-Pierre Chrétien wrote:

> Sure, but that means a lot of test to be moved to unreliable:nonstandard.

Agreed. Or changing the definition of unreliable:nonstandard. But I
personally like the definition.

This is only slightly related, but if you do want all of the
dependencies, you can run this script:

  https://github.com/scottkosty/lyx-tester

The script installs all necessary dependencies and runs the tests.
There should be 0 failing tests after running that script.

I only recommend running it on a virtual box image, or after a fresh
installation of Ubuntu. If you do either of those and want to run the
script, let me know.

Scott


signature.asc
Description: PGP signature

Re: ctest fails standard exports of a number of examples and templates files

[Jean-Pierre Chrétien]

Le 20/02/2017 à 20:13, Kornel Benko a écrit :

Am Montag, 20. Februar 2017 um 18:40:26, schrieb Jean-Pierre Chrétien 

...

  Here are the resources which should be installed to
  avoid failure of tests with the "unreliable:nonstandard" label:

Do you agree?


Sure, but that means a lot of test to be moved to unreliable:nonstandard.
To make things clearer,  attached is a file with the ctest command I ran and the
list of failures I got.

The command runs ctest excluding export tests other than pdf and pdf2 on a list
of files which contain one or more maths option 0 (i. e. do not load by 
default).


That is a big list.
Using the same ctest command here I see only 21 fails. One of them
actually never ends (export/templates/poster-a0poster-colored._pdf2).
What’s more, my list does not match any of elements in your list.

1474 - export/doc/EmbeddedObjects_pdf (Failed)
1475 - export/doc/EmbeddedObjects_pdf2 (Failed)
1909 - export/doc/de/EmbeddedObjects_pdf (Failed)
1910 - export/doc/de/EmbeddedObjects_pdf2 (Failed)
2101 - export/doc/es/EmbeddedObjects_pdf (Failed)
2102 - export/doc/es/EmbeddedObjects_pdf2 (Failed)
2309 - export/doc/fr/EmbeddedObjects_pdf (Failed)
2310 - export/doc/fr/EmbeddedObjects_pdf2 (Failed)
3191 - export/examples/achemso_pdf (Failed)
3192 - export/examples/achemso_pdf2 (Failed)
3444 - export/examples/linguistics_pdf (Failed)
3445 - export/examples/linguistics_pdf2 (Failed)
4447 - export/examples/de/linguistics_pdf (Failed)
4448 - export/examples/de/linguistics_pdf2 (Failed)
4630 - export/examples/es/linguistics_pdf (Failed)
4631 - export/examples/es/linguistics_pdf2 (Failed)
4924 - export/examples/fr/linguistics_pdf (Failed)
4925 - export/examples/fr/linguistics_pdf2 (Failed)
5249 - export/examptaeles/ja/linguistics_pdf (Failed)
6253 - export/templates/poster-a0poster-colored_pdf2 (Failed)
6301 - export/templates/poster-sciposter_pdf2 (Failed)


OK, but that was before I imported the necessary packages from the publishers 
who are not in TeXLive 2016. After importing wha is needed I get this:


306:export/doc/Additional_pdf
307:export/doc/Additional_pdf2
737:export/doc/de/Additional_pdf
738:export/doc/de/Additional_pdf2
917:export/doc/es/Additional_pdf
918:export/doc/es/Additional_pdf2
1112:export/doc/fr/Additiontaeal_pdf
1113:export/doc/fr/Additional_pdf2
1307:export/doc/he/Intro_pdf
1308:export/doc/he/Intro_pdf2
1322:export/doc/he/Tutorial_pdf
1323:export/doc/he/Tutorial_pdf2
1897:export/examples/Literate_lyx21
1904:export/examples/Literate_pdf
1905:export/examples/Literate_pdf2
2111:export/examples/docbook_article_pdf
2176:export/examples/knitr_lyx16
2177:export/examples/knitr_lyx21
2184:export/examples/knitr_pdf
2185:export/examples/knitr_pdf2
2206:export/examples/lilypond_lyx16
2207:export/examples/lilypond_lyx21
2214:export/examples/lilypond_pdf
2215:export/examples/lilypond_pdf2
2236:export/examples/listerrors_lyx16
2237:export/examples/listerrors_lyx21
2241:export/examples/listerrors_pdf
2278:export/examples/noweb2lyx_lyx21
2285:export/examples/noweb2lyx_pdf
2286:export/examples/noweb2lyx_pdf2
2769:export/examples/sweave_lyx16
2770:export/examples/sweave_lyx21
2777:export/examples/sweave_pdf
2778:export/examples/sweave_pdf2
3460:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf
3461:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf2
3754:export/examples/he/example_lyxified_pdf
3755:export/examples/he/example_lyxified_pdf2
3769:export/examples/he/example_raw_pdf
3770:export/examples/he/example_raw_pdf2
3784:export/examples/he/splash_pdf
3785:export/examples/he/splash_pdf2
3916:export/examples/ja/knitr_lyx16
3917:export/examples/ja/knitr_lyx21
3923:export/examples/ja/lilypond_lyx16
3924:export/examples/ja/lilypond_lyx21
3943:export/examples/ja/sweave_lyx16
3944:export/examples/ja/sweave_lyx21
4337:export/templates/AGUTeXtae_pdf
4338:export/templates/AGUTeX_pdf2
4379:export/templates/DocBook_article_pdf
4533:export/templates/RJournal_lyx16
4534:export/templates/RJournal_lyx21
4541:export/templates/RJournal_pdf
4542:export/templates/RJournal_pdf2
4795:UNRELIABLE.NONSTANDARD_export/templates/ja_beamer-conference-ornate-20min_pdf2


So
 - you can export hebrew docs in pdf2, I can't (I need system fonts);
 - you can export the various literate programming examples and templates in 
pdf2, I can't either (I can only in xhtml, which is the sole export method 
allowed if I try by hand).


I will investigate later, I'll be away for a while.

--
Jean-Pierre

Re: ctest fails standard exports of a number of examples and templates files

[Kornel Benko]

Am Montag, 20. Februar 2017 um 18:40:26, schrieb Jean-Pierre Chrétien 

...
> >   Here are the resources which should be installed to
> >   avoid failure of tests with the "unreliable:nonstandard" label:
> >
> > Do you agree?
> 
> Sure, but that means a lot of test to be moved to unreliable:nonstandard.
> To make things clearer,  attached is a file with the ctest command I ran and 
> the 
> list of failures I got.
> 
> The command runs ctest excluding export tests other than pdf and pdf2 on a 
> list 
> of files which contain one or more maths option 0 (i. e. do not load by 
> default).

That is a big list.
Using the same ctest command here I see only 21 fails. One of them
actually never ends (export/templates/poster-a0poster-colored._pdf2).
What’s more, my list does not match any of elements in your list.

1474 - export/doc/EmbeddedObjects_pdf (Failed)
1475 - export/doc/EmbeddedObjects_pdf2 (Failed)
1909 - export/doc/de/EmbeddedObjects_pdf (Failed)
1910 - export/doc/de/EmbeddedObjects_pdf2 (Failed)
2101 - export/doc/es/EmbeddedObjects_pdf (Failed)
2102 - export/doc/es/EmbeddedObjects_pdf2 (Failed)
2309 - export/doc/fr/EmbeddedObjects_pdf (Failed)
2310 - export/doc/fr/EmbeddedObjects_pdf2 (Failed)
3191 - export/examples/achemso_pdf (Failed)
3192 - export/examples/achemso_pdf2 (Failed)
3444 - export/examples/linguistics_pdf (Failed)
3445 - export/examples/linguistics_pdf2 (Failed)
4447 - export/examples/de/linguistics_pdf (Failed)
4448 - export/examples/de/linguistics_pdf2 (Failed)
4630 - export/examples/es/linguistics_pdf (Failed)
4631 - export/examples/es/linguistics_pdf2 (Failed)
4924 - export/examples/fr/linguistics_pdf (Failed)
4925 - export/examples/fr/linguistics_pdf2 (Failed)
5249 - export/examples/ja/linguistics_pdf (Failed)
6253 - export/templates/poster-a0poster-colored_pdf2 (Failed)
6301 - export/templates/poster-sciposter_pdf2 (Failed)

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: ctest fails standard exports of a number of examples and templates files

[Jean-Pierre Chrétien]

Le 15/02/2017 à 02:09, Scott Kostyshak a écrit :

On Tue, Feb 14, 2017 at 04:31:22PM +0100, Jean-Pierre Chrétien wrote:

Hello,

I discovered that quite a number of examples and templates files depend on
resources  which do not come with TeXLive 2016. This is normal with external
applications (R, lilypond, etc.) but there are a number of LaTeX packages
from editors which are not in the appropriate collection of TeXLIve 2016.

I made an addition to Development.lyx to record what is needed to avoid
ctest failures on files which are labeled as standard exports, see
attachment.


Thanks for taking a look at the tests! And I agree, it's good to specify
all the dependencies. I have some comments below.

I think there are tests that should be labeled "nonstandard" but
currently are not. Once these are corrected, we would then change the
following sentence you added:

  With TeXLive 2016, here are the resources which should be installed to
  avoid standard tests failure:

to something like:

  Here are the resources which should be installed to
  avoid failure of tests with the "unreliable:nonstandard" label:

Do you agree?


Sure, but that means a lot of test to be moved to unreliable:nonstandard.
To make things clearer,  attached is a file with the ctest command I ran and the 
list of failures I got.


The command runs ctest excluding export tests other than pdf and pdf2 on a list 
of files which contain one or more maths option 0 (i. e. do not load by default).


The failures mostly come from packages not distributed by TeXLive, which I 
collected in the add-on I proposed to Development.lyx.


As you may see, there are only 4 files marked as unreliable:nonstandard :

3460:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf2
4292:UNRELIABLE.NONSTANDARD_export/templates/ACM-siggraph_pdf2
4499:UNRELIABLE.NONSTANDARD_export/templates/IUCr-article_pdf
4795:UNRELIABLE.NONSTANDARD_export/templates/ja_beamer-conference-ornate-20min_pdf2

That means that, omitting the literate programming files (which should be 
exorted to xhtml if I understand correctly) and the Hebrew files (which export 
correctly with XeTeX or LuaTeX), we should mark as unreliable:nonstandard


In examples: Foils, aa_sample, landslide, lilypond, spreadsheet, all the 
springer files,


In templates : ACM-SIGS, ACM-sigplan, AEA, AGUTeX, IOP-article, JASA, aa, 
ectaart, svjour3


As far as AGUTeX is concerned, I detected some erratic update of the LaTeX 
packages distributed by AGU: they seem to have stabilized now to a class named 
agujournal.cls. I began to create a agujournal.lyx template in branch, but I 
understand that I should do it in master.




For Development.lyx, the section
now reads:

To run all tests, in the build directory simply run the command ctest. A
full, up-to-date TeXLive installation is recommended to run the tests.
Otherwise, some tests will fail. Tests with additional requirements are
labeled "unreliable:nonstandard" [footnote: With an up-to-date Texlive 2016
installation, there are standard tests which will fail because of
missing packages or external applications, see below for a list of those
which should be installed to avoid these failures.]

I think this could be made more concise. Also, we need to be sure not to
confuse two things: (1) a *full* (I think we should mention this,
right?) up-to-date TeXLive installation is assumed for tests without the
"unreliable:nonstandard" label; and (2) for tests with the
"unreliable:nonstandard" label, there are requirements in addition to a
full TeXLive installation.

Instead of "see below", I would prefer a reference to a label.

Currently the list of dependencies you added is under the section
"4.3.1.7 What action should you take if a test fails?".
I think it would make more sense to add a section before e.g.
"Configuring the tests" that is dedicated to "Test dependencies".
What do you think?


Sure, that would be better. I think also to put in trac a tds of the packages I 
had to add in my texmf-local tree, that I would of course keep up to date.


--
Jean-Pierre



   --- command applied -
   
ctest  -j8 -E 'attic|check_load|check_layout|systemF|texF|pdf3|dvi' -R 

Re: ctest fails standard exports of a number of examples and templates files

[Scott Kostyshak]

On Wed, Feb 15, 2017 at 01:26:09PM +, Guenter Milde wrote:

> IMO, it is simpler to add the additional nonstandard dependencies in the
> comments in "unreliableTests" only and reword the start of 4.3.1.4
>   
>   To run all tests, in the build directory simply run the command ctest.
>   A full, up-to-date TeXLive installation is recommended to run the
>   tests. Otherwise, some tests will fail. Tests with additional
>   requirements are labeled “unreliable:nonstandard”. 
>   
> to something like
> 
>   A full, up-to-date TeXLive installation is required to run the
>   tests. Otherwise, some tests will fail. 
>   Tests with the label “unreliable:nonstandard” have additional
>   requirements (see development/autotests/unreliableTests for details).
>   To exclude these tests, run `ctest -LE nonstandard`.
>   To run all tests, in the build directory simply run the command ctest.

That looks good.

> This would safe us some work updating requirements and prevent
> desynchronisation of the requirement lists.

Good point. I wonder if an outdated list would be better than no list.

Scott


signature.asc
Description: PGP signature

Re: ctest fails standard exports of a number of examples and templates files

[Guenter Milde]

Dear Jean-Pierre, Scott, and Kornel, dear others,

On 2017-02-15, Scott Kostyshak wrote:
> On Tue, Feb 14, 2017 at 04:31:22PM +0100, Jean-Pierre Chrétien wrote:
>> Hello,

>> I discovered that quite a number of examples and templates files
>> depend on resources which do not come with TeXLive 2016. This is
>> normal with external applications (R, lilypond, etc.) but there are a
>> number of LaTeX packages from editors which are not in the appropriate
>> collection of TeXLIve 2016.

>> I made an addition to Development.lyx to record what is needed to avoid
>> ctest failures on files which are labeled as standard exports, see
>> attachment.

> Thanks for taking a look at the tests! And I agree, it's good to specify
> all the dependencies. I have some comments below.

> I think there are tests that should be labeled "nonstandard" but
> currently are not.

So do I. 
All tests that require more than the agreed "standard requirements"
should get the label "unreliable:nonstandard".

> Once these are corrected, we would then change the
> following sentence you added:

>   With TeXLive 2016, here are the resources which should be installed to
>   avoid standard tests failure:

> to something like:

>   Here are the resources which should be installed to
>   avoid failure of tests with the "unreliable:nonstandard" label:

> Do you agree?

> For Development.lyx, the section
> now reads:

> To run all tests, in the build directory simply run the command ctest. A
> full, up-to-date TeXLive installation is recommended to run the tests.
> Otherwise, some tests will fail. Tests with additional requirements are
> labeled "unreliable:nonstandard" [footnote: With an up-to-date Texlive 2016
> installation, there are standard tests which will fail because of
> missing packages or external applications, see below for a list of those
> which should be installed to avoid these failures.]

The footnote would no longer be required (i.e. replaced by adding the tests
from the list below to the "unreliableTests" filter file.

> I think this could be made more concise. Also, we need to be sure not to
> confuse two things: (1) a *full* (I think we should mention this,
> right?) up-to-date TeXLive installation is assumed for tests without the
> "unreliable:nonstandard" label; and (2) for tests with the
> "unreliable:nonstandard" label, there are requirements in addition to a
> full TeXLive installation.

> Instead of "see below", I would prefer a reference to a label.

> Currently the list of dependencies you added is under the section
> "4.3.1.7 What action should you take if a test fails?".
> I think it would make more sense to add a section before e.g.
> "Configuring the tests" that is dedicated to "Test dependencies".
> What do you think?

IMO, it is simpler to add the additional nonstandard dependencies in the
comments in "unreliableTests" only and reword the start of 4.3.1.4
  
  To run all tests, in the build directory simply run the command ctest.
  A full, up-to-date TeXLive installation is recommended to run the
  tests. Otherwise, some tests will fail. Tests with additional
  requirements are labeled “unreliable:nonstandard”. 
  
to something like

  A full, up-to-date TeXLive installation is required to run the
  tests. Otherwise, some tests will fail. 
  Tests with the label “unreliable:nonstandard” have additional
  requirements (see development/autotests/unreliableTests for details).
  To exclude these tests, run `ctest -LE nonstandard`.
  To run all tests, in the build directory simply run the command ctest.

This would safe us some work updating requirements and prevent
desynchronisation of the requirement lists.

Günter

Re: ctest fails standard exports of a number of examples and templates files

[Scott Kostyshak]

On Tue, Feb 14, 2017 at 04:31:22PM +0100, Jean-Pierre Chrétien wrote:
> Hello,
> 
> I discovered that quite a number of examples and templates files depend on
> resources  which do not come with TeXLive 2016. This is normal with external
> applications (R, lilypond, etc.) but there are a number of LaTeX packages
> from editors which are not in the appropriate collection of TeXLIve 2016.
> 
> I made an addition to Development.lyx to record what is needed to avoid
> ctest failures on files which are labeled as standard exports, see
> attachment.

Thanks for taking a look at the tests! And I agree, it's good to specify
all the dependencies. I have some comments below.

I think there are tests that should be labeled "nonstandard" but
currently are not. Once these are corrected, we would then change the
following sentence you added:

  With TeXLive 2016, here are the resources which should be installed to
  avoid standard tests failure:

to something like:

  Here are the resources which should be installed to
  avoid failure of tests with the "unreliable:nonstandard" label:

Do you agree?

For Development.lyx, the section
now reads:

To run all tests, in the build directory simply run the command ctest. A
full, up-to-date TeXLive installation is recommended to run the tests.
Otherwise, some tests will fail. Tests with additional requirements are
labeled "unreliable:nonstandard" [footnote: With an up-to-date Texlive 2016
installation, there are standard tests which will fail because of
missing packages or external applications, see below for a list of those
which should be installed to avoid these failures.]

I think this could be made more concise. Also, we need to be sure not to
confuse two things: (1) a *full* (I think we should mention this,
right?) up-to-date TeXLive installation is assumed for tests without the
"unreliable:nonstandard" label; and (2) for tests with the
"unreliable:nonstandard" label, there are requirements in addition to a
full TeXLive installation.

Instead of "see below", I would prefer a reference to a label.

Currently the list of dependencies you added is under the section
"4.3.1.7 What action should you take if a test fails?".
I think it would make more sense to add a section before e.g.
"Configuring the tests" that is dedicated to "Test dependencies".
What do you think?

Scott


signature.asc
Description: PGP signature