Re: lyx2lyx problems (nothing to do with '--' ligatures)

[Guenter Milde]

On 2018-02-16, Scott Kostyshak wrote:
> On Fri, Feb 16, 2018 at 03:27:30PM +, Kornel Benko wrote:

>> > >>   Is there a chance, to test the lyx2lyx output with LyX 2.2, 2.1
>> > >>   and maybe even 1.6?
>> > > 
>> > > I don't think so. Remember, we only test with executables just created.
>> > 
>> > This is a pity. We should, at the very least test all 2.2-exported files
>> > with 2.2 before releasing 2.3!
>> > 
>> > Maybe we can add a simple script that opens all lyx22 files in
>> > autotests/out-home with lyx2.2 and reports failures (affected file(s) +
>> > eventual error/warning message).

>> Yes.

> I agree these tests could be useful and I have done this manually in the
> past (script to export to 2.2.x and then just copy those over to the
> 2.2.x directory and run ctests there). If I remember correctly, they did
> indeed find errors in lyx2lyx that the round-trip did not catch. What
> interface do you have in mind?

If I understand the system right, the ctests leave generated files in
"/autotests/out-home/", so instead of exporting via a script, we can
select the test documents with a pattern (e.g. the regexp
"/autotests/out-home/.*22.lyx" for LyX 2.2 documents) and then 

* feed them to LyX 2.2 in a script or
* copy them to a directory that is tested with lyx22 and ctest.

Maybe this is already sufficient. You should get an error for
lyx_2_3_test.22.lyx if everything is right because of the baselineskip
problem (unless it is reverted in the 2.2 test suite, too).


> Should we set the path to the LyX 2.2.x binary with an environment
> variable?

> If these tests are included in the ctest framework, we should not enable
> them by default because of the huge dependency. But it might be nice if
> we can figure out an easy way to put them in our ctest framework because
> that would make them easy to run. Maybe something like

>-DLYX_CTESTS_22X=/path/to/lyx22x/binary

> to enable them? Otherwise (by default) they are not turned on. We could
> similarly have -DLYX_CTESTS_21X. I suppose that the ctests would have to
> take care of configuring with a separate user directory and all that.
> Seems complicated to me, but might be worth it?

> Thoughts?

Maybe just test for a binary lyx2.2, lyx2.1, and lyx1.6 executable in the ctest
set-up with cmake. This assumes that if a user has installed several LyX
verions with version suffix, he/she also cared for different user dirs etc.

Then, these test could run by default if the conditions are met.
If we give them a label, it would be easy to turn them off.


Whatever the final (or intermediate) outcome, it should be documented in
Development.lyx.

Thanks,

Günter

Re: lyx2lyx problems (nothing to do with '--' ligatures)

[Scott Kostyshak]

On Fri, Feb 16, 2018 at 03:27:30PM +, Kornel Benko wrote:

> > >>   Is there a chance, to test the lyx2lyx output with LyX 2.2, 2.1 and
> > >>   maybe
> > >>   even 1.6?
> > > 
> > > I don't think so. Remember, we only test with executables just created.
> > 
> > This is a pity. We should, at the very least test all 2.2-exported files
> > with 2.2 before releasing 2.3!
> > 
> > Maybe we can add a simple script that opens all lyx22 files in
> > autotests/out-home with lyx2.2 and reports failures (affected file(s) +
> > eventual error/warning message).
> > 
> 
> Yes.

I agree these tests could be useful and I have done this manually in the
past (script to export to 2.2.x and then just copy those over to the
2.2.x directory and run ctests there). If I remember correctly, they did
indeed find errors in lyx2lyx that the round-trip did not catch. What
interface do you have in mind? Should we set the path to the LyX 2.2.x
binary with an environment variable?

If these tests are included in the ctest framework, we should not enable
them by default because of the huge dependency. But it might be nice if
we can figure out an easy way to put them in our ctest framework because
that would make them easy to run. Maybe something like

   -DLYX_CTESTS_22X=/path/to/lyx22x/binary

to enable them? Otherwise (by default) they are not turned on. We could
similarly have -DLYX_CTESTS_21X. I suppose that the ctests would have to
take care of configuring with a separate user directory and all that.
Seems complicated to me, but might be worth it?

Thoughts?

Scott


signature.asc
Description: PGP signature

Re: lyx2lyx problems (nothing to do with '--' ligatures)

[Kornel Benko]

Am Freitag, 16. Februar 2018 09:32:39 CET schrieb Guenter Milde 
:
> On 2018-02-15, Kornel Benko wrote:
> > Am Mittwoch, 14. Februar 2018 17:26:47 CET schrieb Guenter Milde
> > 
> >> We found and fixed some problems with the new test.
> >> However, there are still problems that go undetected:
> >> 
> >> * We currently do not test, whether a backported file can be opened by
> >> the
> >> 
> >>   target LyX version. This means that instances of new constructs not
> >>   handled by lyx2lyx go undetected. (An example are "baseskip%" lengths
> >>   in
> >>   box insets.)
> >>   
> >>   Is there a chance, to test the lyx2lyx output with LyX 2.2, 2.1 and
> >>   maybe
> >>   even 1.6?
> > 
> > I don't think so. Remember, we only test with executables just created.
> 
> This is a pity. We should, at the very least test all 2.2-exported files
> with 2.2 before releasing 2.3!
> 
> Maybe we can add a simple script that opens all lyx22 files in
> autotests/out-home with lyx2.2 and reports failures (affected file(s) +
> eventual error/warning message).
> 

Yes.

> ...
> 
> >> * Sometimes, the back-converted file will load in LyX but fail to compile
> >> 
> >>   due to errors in the generated LaTeX. Could we also test this?
> > 
> > Should be doable. Something like export to pdflatex as a last step in the
> > process.
> 
> This would be nice. It should suffice to do this on the last "round-trip
> version". Instead of "pdflatex" I suggest testing with default output as set
> in the document.

Goor idea. The attached works for me.

> Thanks,
> Günter

Korneldiff --git a/development/autotests/export.cmake b/development/autotests/export.cmake
index 3800ef4..e60e2ea 100755
--- a/development/autotests/export.cmake
+++ b/development/autotests/export.cmake
@@ -134,6 +134,12 @@ if (extension MATCHES "\\.lyx$")
 # Check if result file identical to source file
 if(result_md5sum STREQUAL ${source_md5sum})
   message(STATUS "Source(${LYX_SOURCE}) and dest(${result_file_name}) are equal")
+  message(STATUS "Now try to export the lyx2lyx created file")
+  message(STATUS "Executing ${lyx} -userdir \"${LYX_TESTS_USERDIR}\" -E default \"${result_file_name}.default\" \"${result_file_name}\"")
+  execute_process(
+COMMAND ${lyx} -userdir "${LYX_TESTS_USERDIR}" -E default "${result_file_name}.default" "${result_file_name}"
+RESULT_VARIABLE _err
+ERROR_VARIABLE lyxerr)
   break()
 else()
   message(STATUS "Source(${LYX_SOURCE}) and dest(${result_file_name}) are still different")

Re: lyx2lyx problems (nothing to do with '--' ligatures)

[Guenter Milde]

On 2018-02-15, Kornel Benko wrote:
> Am Mittwoch, 14. Februar 2018 17:26:47 CET schrieb Guenter Milde 

>> We found and fixed some problems with the new test.
>> However, there are still problems that go undetected:

>> * We currently do not test, whether a backported file can be opened by the
>>   target LyX version. This means that instances of new constructs not
>>   handled by lyx2lyx go undetected. (An example are "baseskip%" lengths in
>>   box insets.)

>>   Is there a chance, to test the lyx2lyx output with LyX 2.2, 2.1 and maybe
>>   even 1.6?

> I don't think so. Remember, we only test with executables just created.

This is a pity. We should, at the very least test all 2.2-exported files
with 2.2 before releasing 2.3! 

Maybe we can add a simple script that opens all lyx22 files in
autotests/out-home with lyx2.2 and reports failures (affected file(s) +
eventual error/warning message).


...

>> * Sometimes, the back-converted file will load in LyX but fail to compile
>>   due to errors in the generated LaTeX. Could we also test this?

> Should be doable. Something like export to pdflatex as a last step in the 
> process.

This would be nice. It should suffice to do this on the last "round-trip
version". Instead of "pdflatex" I suggest testing with default output as set
in the document.

Thanks,
Günter

Re: lyx2lyx problems (nothing to do with '--' ligatures)

[Kornel Benko]

Am Mittwoch, 14. Februar 2018 17:26:47 CET schrieb Guenter Milde 
:
> On 2018-01-22, Kornel Benko wrote:
> > I expanded the lyx2lyx test so, the an error is emitted at 10th
> > repeated run to export lyx16x.
> > 
> > If two consecutive created lyx-files are identical, the test stops
> > without error. if a created lyx-file is not loadable, test stops with
> > error.
> 
> We found and fixed some problems with the new test.
> However, there are still problems that go undetected:
> 
> * We currently do not test, whether a backported file can be opened by the
>   target LyX version. This means that instances of new constructs not
>   handled by lyx2lyx go undetected. (An example are "baseskip%" lengths in
>   box insets.)
> 
>   Is there a chance, to test the lyx2lyx output with LyX 2.2, 2.1 and maybe
>   even 1.6?

I don't think so. Remember, we only test with executables just created.

> * LyX does accept some variability in the input files and normalizes them on
> output. Examples are newlines or the order of settings in the document
> header.
> 
>   - Do we want lyx2lyx to produce output that is unchanged when opened and
> saved by the target LyX version?
> What is the price we are willing to pay for this in terms of
> - added complexity of the lyx2lyx functions
> - ly2lyx run time

Good question.

>   Examples:
> 
> * Reverting Quote insets to literal quotes adds newlines (or rather does
> not remove them).
> 
> * The Developers guide says we should add new settings in lyx2lyx but
>   this can also be taken care of by LyX itself: a missing setting is
> just set to the default value already.
> 
> * Sorting the settings to the place where LyX will save them can be done
> in lyx2lyx but makes the conversion take longer and the lyx2lyx script have
> 2 more lines. It is not necessary, though (or am I missing something here)?
> 
> 
> * Sometimes, the back-converted file will load in LyX but fail to compile
>   due to errors in the generated LaTeX. Could we also test this?
> 
> Günter

Should be doable. Something like export to pdflatex as a last step in the 
process.

Kornel

Re: lyx2lyx problems (nothing to do with '--' ligatures)

[Guenter Milde]

On 2018-01-22, Kornel Benko wrote:

> I expanded the lyx2lyx test so, the an error is emitted at 10th
> repeated run to export lyx16x.

> If two consecutive created lyx-files are identical, the test stops
> without error. if a created lyx-file is not loadable, test stops with
> error.

We found and fixed some problems with the new test.
However, there are still problems that go undetected:

* We currently do not test, whether a backported file can be opened by the
  target LyX version. This means that instances of new constructs not
  handled by lyx2lyx go undetected. (An example are "baseskip%" lengths in
  box insets.)

  Is there a chance, to test the lyx2lyx output with LyX 2.2, 2.1 and maybe
  even 1.6?

* LyX does accept some variability in the input files and normalizes them on
  output. Examples are newlines or the order of settings in the document
  header.

  - Do we want lyx2lyx to produce output that is unchanged when opened and
saved by the target LyX version?
What is the price we are willing to pay for this in terms of
- added complexity of the lyx2lyx functions
- ly2lyx run time

  Examples:

* Reverting Quote insets to literal quotes adds newlines (or rather does
  not remove them).

* The Developers guide says we should add new settings in lyx2lyx but
  this can also be taken care of by LyX itself: a missing setting is just
  set to the default value already.
  
* Sorting the settings to the place where LyX will save them can be done
  in lyx2lyx but makes the conversion take longer and the lyx2lyx script
  have 2 more lines. It is not necessary, though (or am I missing
  something here)?
  

* Sometimes, the back-converted file will load in LyX but fail to compile
  due to errors in the generated LaTeX. Could we also test this?
  
Günter