Re: Unit testing: The Small Plan

[Elmar Hinz]

Hello Cyrille,

On Wed, May 8, 2013 at 1:28 AM, Cyrille Artho c.ar...@aist.go.jp wrote:

 Hi Elmar,
 I think your plan covers the question HOW do we want to unit test the
 software well. However, we have not thought much about the WHAT do we
 want to test? question. Essentially, we need to think about which
 classes/functions to test first.

 I think it is not realistic to aim for a high test coverage with software
 as large as LyX. Unit tests make sense in certain cases and less in others.
 So we should identify these use cases first, and start with a few selective
 unit tests. We can then grow them as we see fit.


Yes, I share this vision of let the tests grow. That's the reason, why it's
only a small plan. A plan can't tell the people what they want to test.
It's the decision of the people themself, driven by different motivations.
A plan can only leverage.

There are at least two questions:

* What do I want to test?
* What can I test at all, at reasonable costs.

When I write new software in other languages, I write code and tests side
by side, resulting in a test coverage of approximately 100%, without extra
costs, as I need to run the code anyway.

Writing tests for existing software, is much more difficult. In the
beginning that are extra costs. They pay on the long run by fewer bugs. I
have to weight.


 For user-driven actions (LFUNs), LyX already has a test framework.
 However, these tests do not cover internals of LyX.


Right, unit testing is only one class of testing. That's why it has it's
own path tests/unit.

Gui testing needs completely different approaches and more support by
testing framewoks resulting in tests/gui or similar.

Probably there are also tools to cover memory leaks or performance.


 Which internals do not require a complex sequence of actions (or a complex
 internal state) to be tested? (Those that do are maybe better covered with
 other types of tests.) Where has the code changed often in the past, and is
 expected to change often in the future? What kinds of (internal) functions
 are often covered by bug reports and thus warrant better test automation?


Likely the most error-prone corners are the haredest to tests, because of
the same reasons.

To get started, I would need to do some more simple cases first.



 I hope some of the veteran developers can help answer these questions.


\Elmar

-- 
Elmar Hinz
Freiherr-vom-Stein-Str. 1
33014 Bad Driburg

TYPO3 community contact: t.3.e.l.m.a...@.g.m.a.i.l.dot.c.o.m
personal contact: e.l.m.a.r.dot.h.i.n...@.g.m.a.i.l.dot.c.o.m

Re: [2.0.x] compatibility with automake 1.13

[Richard Heck]

On 05/07/2013 04:05 PM, Jean-Marc Lasgouttes wrote:

Le 07/05/13 22:04, Jean-Marc Lasgouttes a écrit :

Richard, what about this for 2.0.x?


Yes, that's fine.

rh

Re: Donations via Flattr

[Alessandro Di Federico]

On Wed, 2013-05-08 at 15:11 +0200, Christian Staudt wrote:
 as a regular user I just wanted to let you know that if the LyX
 project would accept donations via Flattr, I would have a
 subscription. 

+1, except for the fact that the cheapest way to use Flattr is through
MoneyBookers/Skrill, which is a company that sells your data to
spammers. Sadly, first person experience.

However, having a Flattr button is good idea IMO.

-- 
Ale

Regarding Interface for Lyx

[Abhishek Sharma]

Good day to you

I'm working on the project Horizontal scrollbar for Lyx

I'm using the Lyx document processor and studying the interface for Lyx.
And for this I have studied the tutorial given in Lyx, is there any
other source I can use to enhance my knowledge for interface of Lyx.

Thankyou
Abhishek

[RFC][PATCH] Implement native .mo file reading

[Jean-Marc Lasgouttes]

Dear all,

Here is a first go at reading directly .mo files (and thus getting rid
of the code in intl/). The code is even simpler than I thought it would
be. There remains several things to do, but I am rather pleased with the
result in general.

Note that currently, the two code paths are active, and Messages::get()
compares the outputs of the two codes.

I post it for comments, I will continue tomorrow in Milano. The main
advantages of this approach are:

  * get rid of code that is difficult to update because of portability
issues
  * get rid of code that provides an API that is so bad that we have to
work around it (play with environment variables...)
  * Probably reduce memory usage (previously memory allocated by
gettext + std::map used as a cache; now only one std::map)
  * better control on the code we use.

JMarc

From 07edb10fba2700b107ad61a880e15b2158763f88 Mon Sep 17 00:00:00 2001
From: Jean-Marc Lasgouttes lasgout...@lyx.org
Date: Wed, 8 May 2013 18:50:38 +0200
Subject: [PATCH]  Implement native reading of mo files. PROTOTYPE.

Known limitations:
   * encoding is supposed to be UTF-8 (the charset parameter is not honored);
 Only cs.po uses something ese currently.
   * context is not handled (implemented differently in LyX);
   * plural forms not implemented (not used for now in LyX);.
   * tThe byte endianness of the machine on which the .mo file have been
 built is expected to be the same as the one of the machine where this
 code is run.

What remains to be done:
   * use Qt to detect the current locale (this was done by gettext)
   * remove the old code in Message.*
   * remove the code in intl/
   * remove the corresponding gettext code
   * update documentation
   * check whether the code in po/ can be simplified.
---
 src/support/Messages.cpp | 204 ++-
 src/support/Messages.h   |   9 +++
 2 files changed, 210 insertions(+), 3 deletions(-)

diff --git a/src/support/Messages.cpp b/src/support/Messages.cpp
index ea6ec0d..92b6fd4 100644
--- a/src/support/Messages.cpp
+++ b/src/support/Messages.cpp
@@ -20,11 +20,19 @@
 
 #include support/lassert.h
 
+#include boost/cstdint.hpp
+
 #include cerrno
+#include fstream
+
+#ifdef HAVE_SYS_STAT_H
+# include sys/stat.h
+#endif
 
 #  define N_(str) (str)  // for marking strings to be translated
 
 using namespace std;
+using boost::uint32_t;
 
 namespace lyx {
 
@@ -40,9 +48,11 @@ void cleanTranslation(docstring  trans)
  bogus messages. If we are unable to honour the request we
  just return what we got in.
*/
-   size_t const pos1 = trans.find(from_ascii([[));
+   static docstring const ctx_start = from_ascii([[);
+   static docstring const ctx_end = from_ascii(]]);
+   size_t const pos1 = trans.find(ctx_start);
if (pos1 != docstring::npos) {
-   size_t const pos2 = trans.find(from_ascii(]]), pos1);
+   size_t const pos2 = trans.find(ctx_end, pos1);
if (pos2 != docstring::npos) 
trans.erase(pos1, pos2 - pos1 + 2);
}
@@ -75,6 +85,8 @@ Messages::Messages(string const  l)
size_t i = lang_.find(.);
lang_ = lang_.substr(0, i);
LYXERR(Debug::LOCALE, language(  lang_  ));
+
+   readMoFile();
 }
 
 
@@ -127,7 +139,6 @@ bool Messages::available(string const  c)
else return false;
}
return false;
-
 }
 
 namespace {
@@ -165,6 +176,18 @@ docstring const getText(string const  m)
 
 docstring const Messages::get(string const  m) const
 {
+   docstring s1 = intlGet(m);
+   docstring s2 = nativeGet(m);
+
+   if (s1 != s2)
+   lyxerr  Two translations for \  m  \: \n s1 
 \n s2  endl;
+
+   return s1;
+}
+
+
+docstring const Messages::intlGet(string const  m) const
+{
if (m.empty())
return docstring();
 
@@ -209,6 +232,181 @@ docstring const Messages::get(string const  m) const
return result.first-second;
 }
 
+/*
+  This is a limited parser for gettext's po files. Several features are
+  not handled for now:
+   * encoding is supposed to be UTF-8 (the charset parameter is not honored)
+   * context is not handled (implemented differently in LyX)
+   * plural forms not implemented (not used for now in LyX).
+   * The byte endianness of the machine on which the .mo file have been
+ built is expected to be the same as the one of the machine where this
+ code is run.
+
+  The data is loaded in a std::map object for simplicity.
+ */
+
+/*
+  Format of a MO file. Source: 
http://www.gnu.org/software/gettext/manual/html_node/MO-Files.html
+
+ byte
+  +--+
+   0  | magic number = 0x950412de|
+  |  |
+   4  | file format revision = 0 |
+  |  

Re: Regarding Interface for Lyx

[Jacob Bishop]

On Wed, May 8, 2013 at 11:37 AM, Abhishek Sharma abhios...@gmail.comwrote:

 I'm using the Lyx document processor and studying the interface for Lyx.
 And for this I have studied the tutorial given in Lyx, is there any
 other source I can use to enhance my knowledge for interface of Lyx.


It's not a tutorial per se, but my suggestion is to use LyX to prepare a
complete document. Include a table of contents and a table of figures,
include several equations and images (referenced using labels), and
citations with a BibTeX bibliography. That will probably really help.

Since you are working on the scrollbar project, my suggestion is to also be
sure to add several insets to the document. One option is to add an ERT
inset and fill it with text (beyond the length of the screen). Another is
to add a large figure with multiple sub-figures so that it extends beyond
the screen in all directions.

When I do this, it suggests to me that what LyX could use is not merely a
global horizontal scrollbar to go with the existing vertical one, but also
inset-specific scrollbars. Take a large horizontal table, for example.
Ideally, I think the table width would be limited to some fraction of the
screen width, and would have a horizontal scrollbar that appears within the
inset (my tables usually live inside insets). I also think it would be
extremely useful to have a vertical scrollbar within an inset that is very
tall. For example, I sometimes use ERT to insert a tikz diagram. I wish I
could scroll vertically through the entire document without having to
scroll through multiple screens full of ERT. Of course, I can collapse the
inset, and often do. Another thing this would help solve for me is the
frustration when trying to select all the text within the inset. If I
accidentally scroll one line too far, I have to scroll back to the top of
the inset and try again. This would be much easier to manage if an inset
were never larger than one screen.

Please understand that these are just ideas and opinions from a user's
perspective. Of course, you and the project mentor(s) will have to
determine whether or not to take on any of these other issues, but from my
(possibly naïve) point of view, it seems like once you get the scrollbar
code working, making it inset-specific would be both straightforward and
desirable (if done in a modular way).

Jacob

XML For LyX

[Richard Heck]

I have started to think seriously about moving to XML for LyX's native 
file format. I doubt that we will want to do this for 2.1, as it is too 
late, really, so I am thinking about doing it for some time early in the 
2.2 cycle, which means starting now.


My plan is first to write routines that will output a pure XML version 
of a LyX document and then to worry about the read routines once that is 
working. I think it will be fairly easy to get that much done, by 
working off the XHTML stuff. Some of that will prove re-usable.


Thinking ahead, however: Should we use some SAX library to read the XML? 
Or should we just adapt the Lexer for this purpose?


Richard

Re: XML For LyX

[José Matos]

On Wednesday 08 May 2013 17:43:41 Richard Heck wrote:
 Thinking ahead, however: Should we use some SAX library to read the XML? 
 Or should we just adapt the Lexer for this purpose?
 
 Richard

Lars had that working for a previous version of lyx with lexer. His branches 
are still available in git, I think...

-- 
José Abílio

Re: XML For LyX

[Nico Williams]

Reading will be easier, I think, for the reasons I've described before.
 Also, you could use lyx2xml to write so you can test the read path, but I
don't know of an xml2lyx tool you could use for the reverse.

Just my 2c.

Re: Unit testing: The Small Plan

[Elmar Hinz]

Hello Cyrille,

On Wed, May 8, 2013 at 1:28 AM, Cyrille Artho  wrote:

> Hi Elmar,
> I think your plan covers the question "HOW do we want to unit test the
> software" well. However, we have not thought much about the "WHAT do we
> want to test?" question. Essentially, we need to think about which
> classes/functions to test first.
>
> I think it is not realistic to aim for a high test coverage with software
> as large as LyX. Unit tests make sense in certain cases and less in others.
> So we should identify these use cases first, and start with a few selective
> unit tests. We can then grow them as we see fit.
>
>
Yes, I share this vision of let the tests grow. That's the reason, why it's
only a small plan. A plan can't tell the people what they want to test.
It's the decision of the people themself, driven by different motivations.
A plan can only leverage.

There are at least two questions:

* What do I want to test?
* What can I test at all, at reasonable costs.

When I write new software in other languages, I write code and tests side
by side, resulting in a test coverage of approximately 100%, without extra
costs, as I need to run the code anyway.

Writing tests for existing software, is much more difficult. In the
beginning that are extra costs. They pay on the long run by fewer bugs. I
have to weight.


> For user-driven actions (LFUNs), LyX already has a test framework.
> However, these tests do not cover internals of LyX.
>
>
Right, unit testing is only one class of testing. That's why it has it's
own path "tests/unit".

Gui testing needs completely different approaches and more support by
testing framewoks resulting in "tests/gui" or similar.

Probably there are also tools to cover memory leaks or performance.


> Which internals do not require a complex sequence of actions (or a complex
> internal state) to be tested? (Those that do are maybe better covered with
> other types of tests.) Where has the code changed often in the past, and is
> expected to change often in the future? What kinds of (internal) functions
> are often covered by bug reports and thus warrant better test automation?
>

Likely the most error-prone corners are the haredest to tests, because of
the same reasons.

To get started, I would need to do some more simple cases first.


>
> I hope some of the veteran developers can help answer these questions.
>
>
\Elmar

-- 
Elmar Hinz
Freiherr-vom-Stein-Str. 1
33014 Bad Driburg

TYPO3 community contact: t.3.e.l.m.a...@.g.m.a.i.l.dot.c.o.m
personal contact: e.l.m.a.r.dot.h.i.n...@.g.m.a.i.l.dot.c.o.m

Re: [2.0.x] compatibility with automake 1.13

[Richard Heck]

On 05/07/2013 04:05 PM, Jean-Marc Lasgouttes wrote:

Le 07/05/13 22:04, Jean-Marc Lasgouttes a écrit :

Richard, what about this for 2.0.x?


Yes, that's fine.

rh

Re: Donations via Flattr

[Alessandro Di Federico]

On Wed, 2013-05-08 at 15:11 +0200, Christian Staudt wrote:
> as a regular user I just wanted to let you know that if the LyX
> project would accept donations via Flattr, I would have a
> subscription. 

+1, except for the fact that the cheapest way to use Flattr is through
MoneyBookers/Skrill, which is a company that sells your data to
spammers. Sadly, first person experience.

However, having a Flattr button is good idea IMO.

-- 
Ale

Regarding Interface for Lyx

[Abhishek Sharma]

Good day to you

I'm working on the project "Horizontal scrollbar for Lyx"

I'm using the Lyx document processor and studying the interface for Lyx.
And for this I have studied the tutorial given in Lyx, is there any
other source I can use to enhance my knowledge for interface of Lyx.

Thankyou
Abhishek

[RFC][PATCH] Implement native .mo file reading

[Jean-Marc Lasgouttes]

Dear all,

Here is a first go at reading directly .mo files (and thus getting rid
of the code in intl/). The code is even simpler than I thought it would
be. There remains several things to do, but I am rather pleased with the
result in general.

Note that currently, the two code paths are active, and Messages::get()
compares the outputs of the two codes.

I post it for comments, I will continue tomorrow in Milano. The main
advantages of this approach are:

  * get rid of code that is difficult to update because of portability
issues
  * get rid of code that provides an API that is so bad that we have to
work around it (play with environment variables...)
  * Probably reduce memory usage (previously memory allocated by
gettext + std::map used as a cache; now only one std::map)
  * better control on the code we use.

JMarc

>From 07edb10fba2700b107ad61a880e15b2158763f88 Mon Sep 17 00:00:00 2001
From: Jean-Marc Lasgouttes 
Date: Wed, 8 May 2013 18:50:38 +0200
Subject: [PATCH]  Implement native reading of mo files. PROTOTYPE.

Known limitations:
   * encoding is supposed to be UTF-8 (the charset parameter is not honored);
 Only cs.po uses something ese currently.
   * context is not handled (implemented differently in LyX);
   * plural forms not implemented (not used for now in LyX);.
   * tThe byte endianness of the machine on which the .mo file have been
 built is expected to be the same as the one of the machine where this
 code is run.

What remains to be done:
   * use Qt to detect the current locale (this was done by gettext)
   * remove the old code in Message.*
   * remove the code in intl/
   * remove the corresponding gettext code
   * update documentation
   * check whether the code in po/ can be simplified.
---
 src/support/Messages.cpp | 204 ++-
 src/support/Messages.h   |   9 +++
 2 files changed, 210 insertions(+), 3 deletions(-)

diff --git a/src/support/Messages.cpp b/src/support/Messages.cpp
index ea6ec0d..92b6fd4 100644
--- a/src/support/Messages.cpp
+++ b/src/support/Messages.cpp
@@ -20,11 +20,19 @@
 
 #include "support/lassert.h"
 
+#include 
+
 #include 
+#include 
+
+#ifdef HAVE_SYS_STAT_H
+# include 
+#endif
 
 #  define N_(str) (str)  // for marking strings to be translated
 
 using namespace std;
+using boost::uint32_t;
 
 namespace lyx {
 
@@ -40,9 +48,11 @@ void cleanTranslation(docstring & trans)
  bogus messages. If we are unable to honour the request we
  just return what we got in.
*/
-   size_t const pos1 = trans.find(from_ascii("[["));
+   static docstring const ctx_start = from_ascii("[[");
+   static docstring const ctx_end = from_ascii("]]");
+   size_t const pos1 = trans.find(ctx_start);
if (pos1 != docstring::npos) {
-   size_t const pos2 = trans.find(from_ascii("]]"), pos1);
+   size_t const pos2 = trans.find(ctx_end, pos1);
if (pos2 != docstring::npos) 
trans.erase(pos1, pos2 - pos1 + 2);
}
@@ -75,6 +85,8 @@ Messages::Messages(string const & l)
size_t i = lang_.find(".");
lang_ = lang_.substr(0, i);
LYXERR(Debug::LOCALE, "language(" << lang_ << ")");
+
+   readMoFile();
 }
 
 
@@ -127,7 +139,6 @@ bool Messages::available(string const & c)
else return false;
}
return false;
-
 }
 
 namespace {
@@ -165,6 +176,18 @@ docstring const getText(string const & m)
 
 docstring const Messages::get(string const & m) const
 {
+   docstring s1 = intlGet(m);
+   docstring s2 = nativeGet(m);
+
+   if (s1 != s2)
+   lyxerr << "Two translations for \"" << m << "\": \n   " << s1 
<< "\n   " << s2 << endl;
+
+   return s1;
+}
+
+
+docstring const Messages::intlGet(string const & m) const
+{
if (m.empty())
return docstring();
 
@@ -209,6 +232,181 @@ docstring const Messages::get(string const & m) const
return result.first->second;
 }
 
+/*
+  This is a limited parser for gettext's po files. Several features are
+  not handled for now:
+   * encoding is supposed to be UTF-8 (the charset parameter is not honored)
+   * context is not handled (implemented differently in LyX)
+   * plural forms not implemented (not used for now in LyX).
+   * The byte endianness of the machine on which the .mo file have been
+ built is expected to be the same as the one of the machine where this
+ code is run.
+
+  The data is loaded in a std::map object for simplicity.
+ */
+
+/*
+  Format of a MO file. Source: 
http://www.gnu.org/software/gettext/manual/html_node/MO-Files.html
+
+ byte
+  +--+
+   0  | magic number = 0x950412de|
+  |  |
+   4  | file format revision = 0 |
+

Re: Regarding Interface for Lyx

[Jacob Bishop]

On Wed, May 8, 2013 at 11:37 AM, Abhishek Sharma wrote:

> I'm using the Lyx document processor and studying the interface for Lyx.
> And for this I have studied the tutorial given in Lyx, is there any
> other source I can use to enhance my knowledge for interface of Lyx.
>

It's not a tutorial per se, but my suggestion is to use LyX to prepare a
complete document. Include a table of contents and a table of figures,
include several equations and images (referenced using labels), and
citations with a BibTeX bibliography. That will probably really help.

Since you are working on the scrollbar project, my suggestion is to also be
sure to add several insets to the document. One option is to add an ERT
inset and fill it with text (beyond the length of the screen). Another is
to add a large figure with multiple sub-figures so that it extends beyond
the screen in all directions.

When I do this, it suggests to me that what LyX could use is not merely a
global horizontal scrollbar to go with the existing vertical one, but also
inset-specific scrollbars. Take a large horizontal table, for example.
Ideally, I think the table width would be limited to some fraction of the
screen width, and would have a horizontal scrollbar that appears within the
inset (my tables usually live inside insets). I also think it would be
extremely useful to have a vertical scrollbar within an inset that is very
tall. For example, I sometimes use ERT to insert a tikz diagram. I wish I
could scroll vertically through the entire document without having to
scroll through multiple screens full of ERT. Of course, I can collapse the
inset, and often do. Another thing this would help solve for me is the
frustration when trying to select all the text within the inset. If I
accidentally scroll one line too far, I have to scroll back to the top of
the inset and try again. This would be much easier to manage if an inset
were never larger than one screen.

Please understand that these are just ideas and opinions from a user's
perspective. Of course, you and the project mentor(s) will have to
determine whether or not to take on any of these other issues, but from my
(possibly naïve) point of view, it seems like once you get the scrollbar
code working, making it inset-specific would be both straightforward and
desirable (if done in a modular way).

Jacob

XML For LyX

[Richard Heck]

I have started to think seriously about moving to XML for LyX's native 
file format. I doubt that we will want to do this for 2.1, as it is too 
late, really, so I am thinking about doing it for some time early in the 
2.2 cycle, which means starting now.


My plan is first to write routines that will output a pure XML version 
of a LyX document and then to worry about the read routines once that is 
working. I think it will be fairly easy to get that much done, by 
working off the XHTML stuff. Some of that will prove re-usable.


Thinking ahead, however: Should we use some SAX library to read the XML? 
Or should we just adapt the Lexer for this purpose?


Richard

Re: XML For LyX

[José Matos]

On Wednesday 08 May 2013 17:43:41 Richard Heck wrote:
> Thinking ahead, however: Should we use some SAX library to read the XML? 
> Or should we just adapt the Lexer for this purpose?
> 
> Richard

Lars had that working for a previous version of lyx with lexer. His branches 
are still available in git, I think...

-- 
José Abílio

Re: XML For LyX

[Nico Williams]

Reading will be easier, I think, for the reasons I've described before.
 Also, you could use lyx2xml to write so you can test the read path, but I
don't know of an xml2lyx tool you could use for the reverse.

Just my 2c.