Re: ctest fails standard exports of a number of examples and templates files

[Jean-Pierre Chrétien]

Le 21/02/2017 à 00:10, Kornel Benko a écrit :
.



OK, but that was before I imported the necessary packages from the publishers
who are not in TeXLive 2016.


It is even better now. I was in the process to install iucr package, and in the 
process
I also install data from the iucr/utilities directory. This dir contains _old_ 
float.sty which led
to errors like not having floatstyle "plaintop".


I do not have this subdir:

$ ls /usr/local/texlive/texmf-local/tex/latex/IUCr/
abstemplate.ltxdocumentation.ltx  fig1.psiucr.cls 
README template.ltx


and my float.sty is tha standard one:

$ kpsewhich float.sty
/usr/local/texlive/2016/texmf-dist/tex/latex/float/float.sty

Aha, I did not mention that I run ctest on branch, not on master.



OK. Looking into my texmf-local/tex/latex dir, I have 35 extra packages outside 
of texlive installed. And also plenty of fonts.
From them 17 files have been used today in export test. All of them from 
'culmus'.


I will see this, maybe it is the reason for the pdf2 hebrew failures.

--
Jean-Pierre

Re: Should LaTeX output from command line be equivalent to GUI?

[Scott Kostyshak]

On Tue, Feb 21, 2017 at 02:07:13AM -0500, Richard Heck wrote:
> On 02/20/2017 09:41 PM, Scott Kostyshak wrote:
> > Is it a bug if our LaTeX output is different when exporting from the
> > command line, versus exporting from the GUI? That is, are there some
> > cases where we expect them to be different, and if so, are these
> > documented?
> 
> This is definitely a bug, I'd say.

I get different output for the Spanish UserGuide.lyx.

# I can only reproduce with the Spanish version
cd lib/doc/es
mylyx master -E luatex UserGuide_CL.tex UserGuide.lyx
mylyx master -x "command-sequence buffer-export luatex" UserGuide.lyx
# exit LyX.
diff UserGuide_CL.tex UserGuide.tex

It gives me the following output:

8548c8548
< \settowidth{\nomlabelwidth}{"@, "|, "!, ""}
---
> \settowidth{\nomlabelwidth}{May�sculas}

Thinking more generally, I wonder if it would be useful to automate this
to check all of our documents and formats (more tests?). Or
alternatively, is this a one-time bug that is very unlikely to surface
again?

Scott


signature.asc
Description: PGP signature

Re: [LyX/master] ctests: update inversion patterns.

[Scott Kostyshak]

On Tue, Feb 21, 2017 at 08:00:45AM +0100, Jürgen Spitzmüller wrote:
> Am Montag, den 20.02.2017, 20:45 -0500 schrieb Scott Kostyshak:

> At the moment, option 2 seems the preferable workaround to me.

Sounds good to me.

> > Perhaps we have to do load the tipa package if \textipa is not
> > defined?
> > This might work on both old and new TeX Live.
> 
> No, TIPA does not work with fontspec.

I'm glad you are around to stop me from experimenting like this without
knowing what I'm doing.

Thank you for this detailed explanation! We should link to this email in
our ctests (although hopefully it will be fixed and we can remove the
inversion).

Scott


signature.asc
Description: PGP signature

Re: Should LaTeX output from command line be equivalent to GUI?

[Richard Heck]

On 02/20/2017 09:41 PM, Scott Kostyshak wrote:
> Is it a bug if our LaTeX output is different when exporting from the
> command line, versus exporting from the GUI? That is, are there some
> cases where we expect them to be different, and if so, are these
> documented?

This is definitely a bug, I'd say.

Richard

Re: [LyX/master] ctests: update inversion patterns.

[Jürgen Spitzmüller]

Am Montag, den 20.02.2017, 20:45 -0500 schrieb Scott Kostyshak:
> On Wed, Feb 15, 2017 at 09:23:19AM +, Guenter Milde wrote:
> 
> > Missing \textipa could be intentional -- maybe this is "outsourced"
> > to a
> > Luatex package (for XeTeX, \textipa support is in xunicode.sty). 
> 
> Jürgen, do you have an idea? To reproduce on an updated TeX
> Live, compile the linguistics manual with LuaTeX and system fonts.

The reason is this. Up to v. 2.6, fontspec internally loaded xunicode
which defines the tipa environments. Although xunicode is only written
for XeTeX and terminates with a warning if XeTeX is not used, fontspec
pretended that XeTeX is used also for LuaTeX, by cheating on xunicode's
test (it just tests whether \XeTeXpicfile is defined. If it is, XeTeX
is assumed).

As of fontspec 2.6 and the introduction of the new TU font encodings,
xunicode is no longer loaded. Why does TIPA work nevertheless for us
with XeTeX? Since we (LyX) load xunicode ourselves for XeTeX+fontspec
(which we probably should not do anymore).

Workarounds:

1. Load fontspec with the "euenc" option. This loads the old
experimental unicode encodings (EU1 for XeTeX, EU2 for LuaTeX). This
also loads fontspec. The drawback is that the new TU encodings are not
used.

2. Load xunicode if TIPA is required. For LuaTeX, just do what fontspec
did -- cheat xunicode:

\def\XeTeXpicfile{}
\usepackage{xunicode}

This might break at some point (I don't know whether and where TU
encoding and xunicode will clash).

At the moment, option 2 seems the preferable workaround to me.

A real fix needs to be done upstream. I suppose a package specifically
for tipa with fontspec (derived from xunicode) or, even better, an
integration into the tipa package, would be desired. However, tipa has
not seen updates for a long time (it does not include all glyphs from
recent IPA revisions), so it is not very likely.

Note also that xunicode's tipa support has some flaws. For instance
\textbottomtiebar does not really work, as seen in the linguistics
manual (first IPA chunk).

> Perhaps we have to do load the tipa package if \textipa is not
> defined?
> This might work on both old and new TeX Live.

No, TIPA does not work with fontspec.

Jürgen

> 
> Scott

signature.asc
Description: This is a digitally signed message part

Re: Memory leak in master?

[Jean-Marc Lasgouttes]

Le 21/02/2017 à 00:08, Guillaume Munch a écrit :


Could you do the backport to stable? :)


What about that? Please check especially the code related to 
LYX_USE_CXX11 in Cache.h. For the caching, I re-read the code to make 
sure that my hand-merging was correct, I hope I did not miss anything.


BTW, why don't you use Cache::contains in getLayout like you do for 
other cache uses?


JMarc



From 30b62647ba2dc111aeb4425292e92994cf96f931 Mon Sep 17 00:00:00 2001
From: Guillaume Munch 
Date: Mon, 20 Feb 2017 23:59:24 +0100
Subject: [PATCH] Introduce support/Cache.h

Useful to cache copies of objects, including shared_ptrs. No risks of dangling
pointer, and avoid naked pointers in the source.

Fix memory leak when compiling with Qt5.

As part as the backport to stable, this code has been change to work
with C++98.

(cherry picked from commit 33b696c8acf2e64b44d449180781de6dbc203709)
(cherry picked from commit e04079aa528ecbf4a8e39ed2b19c3cb50174e151)
(cherry picked from commit 5211ca52cac2ad7a6669d15c39f2cee172d18323)
---
 src/frontends/qt4/GuiFontMetrics.cpp |  126 +++---
 src/frontends/qt4/GuiFontMetrics.h   |   46 +++--
 src/frontends/qt4/GuiPainter.cpp |3 +-
 src/support/Cache.h  |   88 
 src/support/Makefile.am  |1 +
 5 files changed, 159 insertions(+), 105 deletions(-)
 create mode 100644 src/support/Cache.h

diff --git a/src/frontends/qt4/GuiFontMetrics.cpp b/src/frontends/qt4/GuiFontMetrics.cpp
index 5bb99aa..80ff819 100644
--- a/src/frontends/qt4/GuiFontMetrics.cpp
+++ b/src/frontends/qt4/GuiFontMetrics.cpp
@@ -24,14 +24,11 @@
 #include "support/convert.h"
 #include "support/lassert.h"
 
-#ifdef CACHE_SOME_METRICS
 #include 
-#endif
 
 using namespace std;
 using namespace lyx::support;
 
-#ifdef CACHE_SOME_METRICS
 namespace std {
 
 /*
@@ -46,11 +43,28 @@ uint qHash(lyx::docstring const & s)
 }
 
 }
-#endif
 
 namespace lyx {
 namespace frontend {
 
+
+/*
+ * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
+ * Limit qtextlayout_cache_ size to 500 elements (we do not know the
+ * size of the QTextLayout objects anyway).
+ * Note that all these numbers are arbitrary.
+ * Also, setting size to 0 is tantamount to disabling the cache.
+ */
+int cache_metrics_width_size = 1 << 19;
+int cache_metrics_breakat_size = 1 << 19;
+// Qt 5.x already has its own caching of QTextLayout objects
+#if (QT_VERSION < 0x05)
+int cache_metrics_qtextlayout_size = 500;
+#else
+int cache_metrics_qtextlayout_size = 0;
+#endif
+
+
 namespace {
 /**
  * Convert a UCS4 character into a QChar.
@@ -73,23 +87,11 @@ inline QChar const ucs4_to_qchar(char_type const ucs4)
 } // anon namespace
 
 
-/*
- * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
- * Limit qtextlayout_cache_ size to 500 elements (we do not know the
- * size of the QTextLayout objects anyway).
- * Note that all these numbers are arbitrary.
- */
 GuiFontMetrics::GuiFontMetrics(QFont const & font)
-	: font_(font), metrics_(font, 0)
-#ifdef CACHE_METRICS_WIDTH
-	, strwidth_cache_(1 << 19)
-#endif
-#ifdef CACHE_METRICS_BREAKAT
-	, breakat_cache_(1 << 19)
-#endif
-#ifdef CACHE_METRICS_QTEXTLAYOUT
-	,  qtextlayout_cache_(500)
-#endif
+	: font_(font), metrics_(font, 0),
+	  strwidth_cache_(cache_metrics_width_size),
+	  breakat_cache_(cache_metrics_breakat_size),
+	  qtextlayout_cache_(cache_metrics_qtextlayout_size)
 {
 }
 
@@ -174,11 +176,8 @@ int GuiFontMetrics::rbearing(char_type c) const
 
 int GuiFontMetrics::width(docstring const & s) const
 {
-#ifdef CACHE_METRICS_WIDTH
-	int * pw = strwidth_cache_[s];
-	if (pw)
-		return *pw;
-#endif
+	if (strwidth_cache_.contains(s))
+		return strwidth_cache_[s];
 	/* For some reason QMetrics::width returns a wrong value with Qt5
 	 * with some arabic text. OTOH, QTextLayout is broken for single
 	 * characters with null width (like \not in mathed). Also, as a
@@ -200,9 +199,7 @@ int GuiFontMetrics::width(docstring const & s) const
 		tl.endLayout();
 		w = int(line.naturalTextWidth());
 	}
-#ifdef CACHE_METRICS_WIDTH
-	strwidth_cache_.insert(s, new int(w), s.size() * sizeof(char_type));
-#endif
+	strwidth_cache_.insert(s, w, s.size() * sizeof(char_type));
 	return w;
 }
 
@@ -225,31 +222,26 @@ int GuiFontMetrics::signedWidth(docstring const & s) const
 }
 
 
-QTextLayout const *
+shared_ptr
 GuiFontMetrics::getTextLayout(docstring const & s, bool const rtl,
   double const wordspacing) const
 {
-	QTextLayout * ptl;
-#ifdef CACHE_METRICS_QTEXTLAYOUT
-	docstring const s_cache = s + (rtl ? "r" : "l") + convert(wordspacing);
-	ptl = qtextlayout_cache_[s_cache];
-	if (!ptl) {
-#endif
-		ptl = new QTextLayout();
-		ptl->setCacheEnabled(true);
-		ptl->setText(toqstr(s));
-		QFont copy = font_;
-		copy.setWordSpacing(wordspacing);
-		ptl->setFont(copy);
-		// Note that both setFlags and the enums are undocumented
-		ptl->setFlags(rtl ? 

Re: lyx errors with lualatex and non-tex fonts

[Scott Kostyshak]

On Tue, Feb 14, 2017 at 09:34:38PM +, Guenter Milde wrote:

> > 2196 - export/doc/es/UserGuide_dvi3_systemF (Failed)
> > 2203 - export/doc/es/UserGuide_pdf5_systemF (Failed)
> 
> These two have the known problem to produce wrong output, e.g.
> 
>   ÃŊndice alfabÃľtico

Can you send me the PDF that you get and tell me which page you see this
problem? I do not see it on the latest TeX Live.

> From unreliableTests:
> 
> # Babel-Spanish uses Babel's "strings" feature to define
> # separate auto-strings using UTF-8 literals.
> # Babel uses the "unicode" strings if it detects XeTeX or LuaTeX.
> # This is wrong for Xe/Lua with 8-bit TeX-fonts.
> # set inputenc to utf8?
> # (Changing the default in lib/languages requires more tests for utf8 first.)
> export/examples/es/linguistics_pdf4_texF
> export/doc/es/.*_(pdf[45]|dvi3)_texF

Yes, I do get an error with TeX fonts.

> With the latest TeXLive update, this also leads to:
> 
> ! Undefined control sequence.
>  ...art {link}{\Hy@tocdestname }{\nonumberline Í
>   ndice alfabético}\hyper...
> l.393 ... Índice alfabético}{175}{chapter*.10}
> 
> Inverting.

I do not get that error with the latest TeX Live.

Are you sure that you tested with system fonts?

The attached patch fixes things for me. OK to go in?

Scott
From c814a1d7840505c2b1bd0d378e5f7fcecf5f Mon Sep 17 00:00:00 2001
From: Scott Kostyshak 
Date: Tue, 21 Feb 2017 00:46:20 -0500
Subject: [PATCH] Fix es/UserGuide.lyx LuaTeX + sysfonts on TL

Remove a \textipa that snuck into es/UserGuide.lyx. This empty inset
caused problems only on the newest TeX Live (see b863fb07).
---
 development/autotests/invertedTests | 11 ---
 lib/doc/es/UserGuide.lyx|  6 +-
 2 files changed, 1 insertion(+), 16 deletions(-)

diff --git a/development/autotests/invertedTests 
b/development/autotests/invertedTests
index b02b0a8..9b43523 100644
--- a/development/autotests/invertedTests
+++ b/development/autotests/invertedTests
@@ -185,17 +185,6 @@ export/.*/he/.*pdf4_texF
 export/doc/ru/(Intro|Tutorial)_(dvi3|pdf[45])_texF
 export/examples/ru/splash_(dvi3|pdf[45])_texF
 
-# Babel-Spanish uses  UTF-8 for auto-strings if it detects Xe/LuaTeX.
-# This leads to wrong output (see unreliableTests) and fails in some cases
-# unless the inputencoding is set to utf-8, too.
-#
-# ! Undefined control sequence.
-#  ...art {link}{\Hy@tocdestname }{\nonumberline Í
-#   ndice alfabético}\hyper...
-# l.393 ... Índice alfabético}{175}{chapter*.10}
-export/doc/es/UserGuide_(dvi3|pdf5)_systemF
-
-
 # Gives the following error:
 # # ! Package hpstatement Error: H or P statement "H251"
 # # (hpstatement)is not defined.
diff --git a/lib/doc/es/UserGuide.lyx b/lib/doc/es/UserGuide.lyx
index 64a4857..6424627 100644
--- a/lib/doc/es/UserGuide.lyx
+++ b/lib/doc/es/UserGuide.lyx
@@ -37044,11 +37044,7 @@ article (Elsevier)
 \begin_inset Quotes frd
 \end_inset
 
- con tres recuadros
-\begin_inset Formula $\text{\textipa{}}$
-\end_inset
-
- personalizados.
+ con tres recuadros personalizados.
  La sección 
 \emph on
 Recuadros flexibles y formato del recuadro
-- 
2.7.4



signature.asc
Description: PGP signature

Re: Memory leak in master?

[Jean-Marc Lasgouttes]

Le 21/02/2017 à 00:08, Guillaume Munch a écrit :

Le 20/02/2017 à 18:27, Jean-Marc Lasgouttes a écrit :

Le 13/02/2017 à 20:55, Jean-Marc Lasgouttes a écrit :

What I mean is that you can easily force
QCache into shared ownership by replacing

QCache

with

QCache

and create the appropriate wrappers for insertion and retrieval.


Could you have a go at this solution?



Could you do the backport to stable? :)


You mean the version that does not require C++11? Mmm, I can always have 
a go at it.


JMarc

Should LaTeX output from command line be equivalent to GUI?

[Scott Kostyshak]

Is it a bug if our LaTeX output is different when exporting from the
command line, versus exporting from the GUI? That is, are there some
cases where we expect them to be different, and if so, are these
documented?

Scott


signature.asc
Description: PGP signature

Re: [LyX/master] ctests: update inversion patterns.

[Scott Kostyshak]

On Wed, Feb 15, 2017 at 09:23:19AM +, Guenter Milde wrote:

> Missing \textipa could be intentional -- maybe this is "outsourced" to a
> Luatex package (for XeTeX, \textipa support is in xunicode.sty). 

Jürgen, do you have an idea? To reproduce on an updated TeX
Live, compile the linguistics manual with LuaTeX and system fonts.

Perhaps we have to do load the tipa package if \textipa is not defined?
This might work on both old and new TeX Live.

Scott


signature.asc
Description: PGP signature

Re: ctest fails standard exports of a number of examples and templates files

[Kornel Benko]

Am Montag, 20. Februar 2017 um 23:27:26, schrieb Jean-Pierre Chrétien 

> Le 20/02/2017 à 20:13, Kornel Benko a écrit :
> > Am Montag, 20. Februar 2017 um 18:40:26, schrieb Jean-Pierre Chrétien 
> > 
> > ...
> >>>   Here are the resources which should be installed to
> >>>   avoid failure of tests with the "unreliable:nonstandard" label:
> >>>
> >>> Do you agree?
> >>
> >> Sure, but that means a lot of test to be moved to unreliable:nonstandard.
> >> To make things clearer,  attached is a file with the ctest command I ran 
> >> and the
> >> list of failures I got.
> >>
> >> The command runs ctest excluding export tests other than pdf and pdf2 on a 
> >> list
> >> of files which contain one or more maths option 0 (i. e. do not load by 
> >> default).
> >
> > That is a big list.
> > Using the same ctest command here I see only 21 fails. One of them
> > actually never ends (export/templates/poster-a0poster-colored._pdf2).
> > What’s more, my list does not match any of elements in your list.
> >
...

> OK, but that was before I imported the necessary packages from the publishers 
> who are not in TeXLive 2016.

It is even better now. I was in the process to install iucr package, and in the 
process
I also install data from the iucr/utilities directory. This dir contains _old_ 
float.sty which led
to errors like not having floatstyle "plaintop".

Now all my exports compile without problems.

> After importing wha is needed I get this:
> 
> 306:export/doc/Additional_pdf
> 307:export/doc/Additional_pdf2
> 737:export/doc/de/Additional_pdf
> 738:export/doc/de/Additional_pdf2
> 917:export/doc/es/Additional_pdf
> 918:export/doc/es/Additional_pdf2
> 1112:export/doc/fr/Additiontaeal_pdf
> 1113:export/doc/fr/Additional_pdf2
> 1307:export/doc/he/Intro_pdf
> 1308:export/doc/he/Intro_pdf2
> 1322:export/doc/he/Tutorial_pdf
> 1323:export/doc/he/Tutorial_pdf2
> 1897:export/examples/Literate_lyx21
> 1904:export/examples/Literate_pdf
> 1905:export/examples/Literate_pdf2
> 2111:export/examples/docbook_article_pdf
> 2176:export/examples/knitr_lyx16
> 2177:export/examples/knitr_lyx21
> 2184:export/examples/knitr_pdf
> 2185:export/examples/knitr_pdf2
> 2206:export/examples/lilypond_lyx16
> 2207:export/examples/lilypond_lyx21
> 2214:export/examples/lilypond_pdf
> 2215:export/examples/lilypond_pdf2
> 2236:export/examples/listerrors_lyx16
> 2237:export/examples/listerrors_lyx21
> 2241:export/examples/listerrors_pdf
> 2278:export/examples/noweb2lyx_lyx21
> 2285:export/examples/noweb2lyx_pdf
> 2286:export/examples/noweb2lyx_pdf2
> 2769:export/examples/sweave_lyx16
> 2770:export/examples/sweave_lyx21
> 2777:export/examples/sweave_pdf
> 2778:export/examples/sweave_pdf2
> 3460:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf
> 3461:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf2
> 3754:export/examples/he/example_lyxified_pdf
> 3755:export/examples/he/example_lyxified_pdf2
> 3769:export/examples/he/example_raw_pdf
> 3770:export/examples/he/example_raw_pdf2
> 3784:export/examples/he/splash_pdf
> 3785:export/examples/he/splash_pdf2
> 3916:export/examples/ja/knitr_lyx16
> 3917:export/examples/ja/knitr_lyx21
> 3923:export/examples/ja/lilypond_lyx16
> 3924:export/examples/ja/lilypond_lyx21
> 3943:export/examples/ja/sweave_lyx16
> 3944:export/examples/ja/sweave_lyx21
> 4337:export/templates/AGUTeXtae_pdf
> 4338:export/templates/AGUTeX_pdf2
> 4379:export/templates/DocBook_article_pdf
> 4533:export/templates/RJournal_lyx16
> 4534:export/templates/RJournal_lyx21
> 4541:export/templates/RJournal_pdf
> 4542:export/templates/RJournal_pdf2
> 4795:UNRELIABLE.NONSTANDARD_export/templates/ja_beamer-conference-ornate-20min_pdf2
> 

Still many ...

> So
>   - you can export hebrew docs in pdf2, I can't (I need system fonts);
>   - you can export the various literate programming examples and templates in 
> pdf2, I can't either (I can only in xhtml, which is the sole export method 
> allowed if I try by hand).
> 
> I will investigate later, I'll be away for a while.
> 

OK. Looking into my texmf-local/tex/latex dir, I have 35 extra packages outside 
of texlive installed. And also plenty of fonts.
From them 17 files have been used today in export test. All of them from 
'culmus'.

Kornel

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Memory leak in master?

[Guillaume Munch]

Le 20/02/2017 à 18:27, Jean-Marc Lasgouttes a écrit :

Le 13/02/2017 à 20:55, Jean-Marc Lasgouttes a écrit :

What I mean is that you can easily force
QCache into shared ownership by replacing

QCache

with

QCache

and create the appropriate wrappers for insertion and retrieval.


Could you have a go at this solution?



Could you do the backport to stable? :)

Re: ctest fails standard exports of a number of examples and templates files

[Scott Kostyshak]

On Mon, Feb 20, 2017 at 06:40:26PM +0100, Jean-Pierre Chrétien wrote:

> Sure, but that means a lot of test to be moved to unreliable:nonstandard.

Agreed. Or changing the definition of unreliable:nonstandard. But I
personally like the definition.

This is only slightly related, but if you do want all of the
dependencies, you can run this script:

  https://github.com/scottkosty/lyx-tester

The script installs all necessary dependencies and runs the tests.
There should be 0 failing tests after running that script.

I only recommend running it on a virtual box image, or after a fresh
installation of Ubuntu. If you do either of those and want to run the
script, let me know.

Scott


signature.asc
Description: PGP signature

Re: [patch] Theorem environment: set NextNoIndent to 0

[Scott Kostyshak]

On Mon, Feb 20, 2017 at 05:34:14PM +0100, Jean-Pierre Chrétien wrote:
> Le 20/02/2017 à 00:40, Scott Kostyshak a écrit :
> 
> > 
> > As noted in the commit log, hollywood and broadway are strange cases. I
> > attach some sample PDF output from them. I don't plan on thinking of a
> > fix for these.
> 
> These classes are useful to write plays or movie scripts and the use of
> theorems should not happen with them, so I guess no fix is required.

OK good.

> In addition, they are quite old (hollywood is dated 2001) and apparently
> only LyX knows about them as they are in the tex/ subdir of the LyX
> distribution (they cannot be found in CTAN).

Good to know. We should probably deprecate some old examples/templates
at some point to make the list cleaner, but it's hard to know who still
uses them.

Scott


signature.asc
Description: PGP signature

Re: ctest fails standard exports of a number of examples and templates files

[Jean-Pierre Chrétien]

Le 20/02/2017 à 20:13, Kornel Benko a écrit :

Am Montag, 20. Februar 2017 um 18:40:26, schrieb Jean-Pierre Chrétien 

...

  Here are the resources which should be installed to
  avoid failure of tests with the "unreliable:nonstandard" label:

Do you agree?


Sure, but that means a lot of test to be moved to unreliable:nonstandard.
To make things clearer,  attached is a file with the ctest command I ran and the
list of failures I got.

The command runs ctest excluding export tests other than pdf and pdf2 on a list
of files which contain one or more maths option 0 (i. e. do not load by 
default).


That is a big list.
Using the same ctest command here I see only 21 fails. One of them
actually never ends (export/templates/poster-a0poster-colored._pdf2).
What’s more, my list does not match any of elements in your list.

1474 - export/doc/EmbeddedObjects_pdf (Failed)
1475 - export/doc/EmbeddedObjects_pdf2 (Failed)
1909 - export/doc/de/EmbeddedObjects_pdf (Failed)
1910 - export/doc/de/EmbeddedObjects_pdf2 (Failed)
2101 - export/doc/es/EmbeddedObjects_pdf (Failed)
2102 - export/doc/es/EmbeddedObjects_pdf2 (Failed)
2309 - export/doc/fr/EmbeddedObjects_pdf (Failed)
2310 - export/doc/fr/EmbeddedObjects_pdf2 (Failed)
3191 - export/examples/achemso_pdf (Failed)
3192 - export/examples/achemso_pdf2 (Failed)
3444 - export/examples/linguistics_pdf (Failed)
3445 - export/examples/linguistics_pdf2 (Failed)
4447 - export/examples/de/linguistics_pdf (Failed)
4448 - export/examples/de/linguistics_pdf2 (Failed)
4630 - export/examples/es/linguistics_pdf (Failed)
4631 - export/examples/es/linguistics_pdf2 (Failed)
4924 - export/examples/fr/linguistics_pdf (Failed)
4925 - export/examples/fr/linguistics_pdf2 (Failed)
5249 - export/examptaeles/ja/linguistics_pdf (Failed)
6253 - export/templates/poster-a0poster-colored_pdf2 (Failed)
6301 - export/templates/poster-sciposter_pdf2 (Failed)


OK, but that was before I imported the necessary packages from the publishers 
who are not in TeXLive 2016. After importing wha is needed I get this:


306:export/doc/Additional_pdf
307:export/doc/Additional_pdf2
737:export/doc/de/Additional_pdf
738:export/doc/de/Additional_pdf2
917:export/doc/es/Additional_pdf
918:export/doc/es/Additional_pdf2
1112:export/doc/fr/Additiontaeal_pdf
1113:export/doc/fr/Additional_pdf2
1307:export/doc/he/Intro_pdf
1308:export/doc/he/Intro_pdf2
1322:export/doc/he/Tutorial_pdf
1323:export/doc/he/Tutorial_pdf2
1897:export/examples/Literate_lyx21
1904:export/examples/Literate_pdf
1905:export/examples/Literate_pdf2
2111:export/examples/docbook_article_pdf
2176:export/examples/knitr_lyx16
2177:export/examples/knitr_lyx21
2184:export/examples/knitr_pdf
2185:export/examples/knitr_pdf2
2206:export/examples/lilypond_lyx16
2207:export/examples/lilypond_lyx21
2214:export/examples/lilypond_pdf
2215:export/examples/lilypond_pdf2
2236:export/examples/listerrors_lyx16
2237:export/examples/listerrors_lyx21
2241:export/examples/listerrors_pdf
2278:export/examples/noweb2lyx_lyx21
2285:export/examples/noweb2lyx_pdf
2286:export/examples/noweb2lyx_pdf2
2769:export/examples/sweave_lyx16
2770:export/examples/sweave_lyx21
2777:export/examples/sweave_pdf
2778:export/examples/sweave_pdf2
3460:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf
3461:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf2
3754:export/examples/he/example_lyxified_pdf
3755:export/examples/he/example_lyxified_pdf2
3769:export/examples/he/example_raw_pdf
3770:export/examples/he/example_raw_pdf2
3784:export/examples/he/splash_pdf
3785:export/examples/he/splash_pdf2
3916:export/examples/ja/knitr_lyx16
3917:export/examples/ja/knitr_lyx21
3923:export/examples/ja/lilypond_lyx16
3924:export/examples/ja/lilypond_lyx21
3943:export/examples/ja/sweave_lyx16
3944:export/examples/ja/sweave_lyx21
4337:export/templates/AGUTeXtae_pdf
4338:export/templates/AGUTeX_pdf2
4379:export/templates/DocBook_article_pdf
4533:export/templates/RJournal_lyx16
4534:export/templates/RJournal_lyx21
4541:export/templates/RJournal_pdf
4542:export/templates/RJournal_pdf2
4795:UNRELIABLE.NONSTANDARD_export/templates/ja_beamer-conference-ornate-20min_pdf2


So
 - you can export hebrew docs in pdf2, I can't (I need system fonts);
 - you can export the various literate programming examples and templates in 
pdf2, I can't either (I can only in xhtml, which is the sole export method 
allowed if I try by hand).


I will investigate later, I'll be away for a while.

--
Jean-Pierre

Re: ctest fails standard exports of a number of examples and templates files

[Kornel Benko]

Am Montag, 20. Februar 2017 um 18:40:26, schrieb Jean-Pierre Chrétien 

...
> >   Here are the resources which should be installed to
> >   avoid failure of tests with the "unreliable:nonstandard" label:
> >
> > Do you agree?
> 
> Sure, but that means a lot of test to be moved to unreliable:nonstandard.
> To make things clearer,  attached is a file with the ctest command I ran and 
> the 
> list of failures I got.
> 
> The command runs ctest excluding export tests other than pdf and pdf2 on a 
> list 
> of files which contain one or more maths option 0 (i. e. do not load by 
> default).

That is a big list.
Using the same ctest command here I see only 21 fails. One of them
actually never ends (export/templates/poster-a0poster-colored._pdf2).
What’s more, my list does not match any of elements in your list.

1474 - export/doc/EmbeddedObjects_pdf (Failed)
1475 - export/doc/EmbeddedObjects_pdf2 (Failed)
1909 - export/doc/de/EmbeddedObjects_pdf (Failed)
1910 - export/doc/de/EmbeddedObjects_pdf2 (Failed)
2101 - export/doc/es/EmbeddedObjects_pdf (Failed)
2102 - export/doc/es/EmbeddedObjects_pdf2 (Failed)
2309 - export/doc/fr/EmbeddedObjects_pdf (Failed)
2310 - export/doc/fr/EmbeddedObjects_pdf2 (Failed)
3191 - export/examples/achemso_pdf (Failed)
3192 - export/examples/achemso_pdf2 (Failed)
3444 - export/examples/linguistics_pdf (Failed)
3445 - export/examples/linguistics_pdf2 (Failed)
4447 - export/examples/de/linguistics_pdf (Failed)
4448 - export/examples/de/linguistics_pdf2 (Failed)
4630 - export/examples/es/linguistics_pdf (Failed)
4631 - export/examples/es/linguistics_pdf2 (Failed)
4924 - export/examples/fr/linguistics_pdf (Failed)
4925 - export/examples/fr/linguistics_pdf2 (Failed)
5249 - export/examples/ja/linguistics_pdf (Failed)
6253 - export/templates/poster-a0poster-colored_pdf2 (Failed)
6301 - export/templates/poster-sciposter_pdf2 (Failed)

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: ctest fails standard exports of a number of examples and templates files

[Jean-Pierre Chrétien]

Le 15/02/2017 à 02:09, Scott Kostyshak a écrit :

On Tue, Feb 14, 2017 at 04:31:22PM +0100, Jean-Pierre Chrétien wrote:

Hello,

I discovered that quite a number of examples and templates files depend on
resources  which do not come with TeXLive 2016. This is normal with external
applications (R, lilypond, etc.) but there are a number of LaTeX packages
from editors which are not in the appropriate collection of TeXLIve 2016.

I made an addition to Development.lyx to record what is needed to avoid
ctest failures on files which are labeled as standard exports, see
attachment.


Thanks for taking a look at the tests! And I agree, it's good to specify
all the dependencies. I have some comments below.

I think there are tests that should be labeled "nonstandard" but
currently are not. Once these are corrected, we would then change the
following sentence you added:

  With TeXLive 2016, here are the resources which should be installed to
  avoid standard tests failure:

to something like:

  Here are the resources which should be installed to
  avoid failure of tests with the "unreliable:nonstandard" label:

Do you agree?


Sure, but that means a lot of test to be moved to unreliable:nonstandard.
To make things clearer,  attached is a file with the ctest command I ran and the 
list of failures I got.


The command runs ctest excluding export tests other than pdf and pdf2 on a list 
of files which contain one or more maths option 0 (i. e. do not load by default).


The failures mostly come from packages not distributed by TeXLive, which I 
collected in the add-on I proposed to Development.lyx.


As you may see, there are only 4 files marked as unreliable:nonstandard :

3460:UNRELIABLE.NONSTANDARD_export/examples/fa/splash_pdf2
4292:UNRELIABLE.NONSTANDARD_export/templates/ACM-siggraph_pdf2
4499:UNRELIABLE.NONSTANDARD_export/templates/IUCr-article_pdf
4795:UNRELIABLE.NONSTANDARD_export/templates/ja_beamer-conference-ornate-20min_pdf2

That means that, omitting the literate programming files (which should be 
exorted to xhtml if I understand correctly) and the Hebrew files (which export 
correctly with XeTeX or LuaTeX), we should mark as unreliable:nonstandard


In examples: Foils, aa_sample, landslide, lilypond, spreadsheet, all the 
springer files,


In templates : ACM-SIGS, ACM-sigplan, AEA, AGUTeX, IOP-article, JASA, aa, 
ectaart, svjour3


As far as AGUTeX is concerned, I detected some erratic update of the LaTeX 
packages distributed by AGU: they seem to have stabilized now to a class named 
agujournal.cls. I began to create a agujournal.lyx template in branch, but I 
understand that I should do it in master.




For Development.lyx, the section
now reads:

To run all tests, in the build directory simply run the command ctest. A
full, up-to-date TeXLive installation is recommended to run the tests.
Otherwise, some tests will fail. Tests with additional requirements are
labeled "unreliable:nonstandard" [footnote: With an up-to-date Texlive 2016
installation, there are standard tests which will fail because of
missing packages or external applications, see below for a list of those
which should be installed to avoid these failures.]

I think this could be made more concise. Also, we need to be sure not to
confuse two things: (1) a *full* (I think we should mention this,
right?) up-to-date TeXLive installation is assumed for tests without the
"unreliable:nonstandard" label; and (2) for tests with the
"unreliable:nonstandard" label, there are requirements in addition to a
full TeXLive installation.

Instead of "see below", I would prefer a reference to a label.

Currently the list of dependencies you added is under the section
"4.3.1.7 What action should you take if a test fails?".
I think it would make more sense to add a section before e.g.
"Configuring the tests" that is dedicated to "Test dependencies".
What do you think?


Sure, that would be better. I think also to put in trac a tds of the packages I 
had to add in my texmf-local tree, that I would of course keep up to date.


--
Jean-Pierre



   --- command applied -
   
ctest  -j8 -E 'attic|check_load|check_layout|systemF|texF|pdf3|dvi' -R 

Re: Memory leak in master?

[Jean-Marc Lasgouttes]

Le 13/02/2017 à 20:55, Jean-Marc Lasgouttes a écrit :

What I mean is that you can easily force
QCache into shared ownership by replacing

QCache

with

QCache

and create the appropriate wrappers for insertion and retrieval.


Could you have a go at this solution?

JMarc

Re: Memory leak in master?

[Jean-Marc Lasgouttes]

Le 10/02/2017 à 17:58, Guillaume Munch a écrit :

There's more data, but I am not sure how to interpret the results that
massif-visualizer shows.


If you send the file or make it available, I can take a look.


Here it is. But if it is like callgrind, it might lack the symbols.


Interestingly, the massif output that you sent also points to 
MathMacro::clone(). Besides the obvious getTextLayout leak that needs to 
be plugged, would you have an idea on why it is specifically macros that 
appear here (with an increasing memory use, like the other)? I do not 
see other math objects doing that.


Note that there are other sources that point to MathAtom::MathAtom, as 
the drop in percentage shows.


JMarc

->09.75% (18,139,776B) 0x8DA8B5: 
lyx::MathMacro::MathMacro(lyx::MathMacro const&) (MathMacro.cpp:299)
| ->09.75% (18,139,776B) 0x8DB03F: lyx::MathMacro::clone() const 
(MathMacro.cpp:394)
|   ->09.75% (18,139,776B) 0x8AD26D: 
lyx::MathAtom::MathAtom(lyx::MathAtom const&) (MathAtom.cpp:35)
| ->03.42% (6,356,216B) 0x86ADFE: 
std::vector::operator=(std::vector const&) 
(stl_construct.h:75)
| | ->01.16% (2,158,056B) 0x8EF837: lyx::(anonymous 
namespace)::Parser::parse(lyx::MathData&, unsigned int, 
lyx::Inset::mode_type) (vector:111)
| | | ->01.16% (2,158,056B) 0x8EFC99: 
lyx::mathed_parse_cell(lyx::MathData&, lyx::docstring const&, 
lyx::Parse::flags) (MathParser.cpp:2150)
| | | | ->01.16% (2,158,056B) 0x9079EB: lyx::asArray(lyx::docstring 
const&, lyx::MathData&, lyx::Parse::flags) (MathSupport.cpp:985)
| | | |   ->01.16% (2,158,056B) 0x8DBBE8: 
lyx::MathMacro::updateRepresentation(lyx::Cursor*, lyx::MacroContext 
const&, lyx::UpdateType, int) (MathMacro.cpp:699)

Re: [patch] Theorem environment: set NextNoIndent to 0

[Jean-Pierre Chrétien]

Le 20/02/2017 à 00:40, Scott Kostyshak a écrit :



As noted in the commit log, hollywood and broadway are strange cases. I
attach some sample PDF output from them. I don't plan on thinking of a
fix for these.


These classes are useful to write plays or movie scripts and the use of theorems 
should not happen with them, so I guess no fix is required.


In addition, they are quite old (hollywood is dated 2001) and apparently only 
LyX knows about them as they are in the tex/ subdir of the LyX distribution 
(they cannot be found in CTAN).


--
Jean-Pierre