Re: Many tex2lyx failing now on master
Am 26.10.2015 um 21:36 schrieb Vincent van Ravesteijn: Well yes, there is no "env" command on Windows. Thanks to Kornel this is now fixed. Besides that, running is a nightmare: - Somehow tex2lyx often deadlocks when outputting errors, Indeed. When I re-run, it works. - I need to have python in the path, Thanks, this was important to know. I updated Development.lyx accordingly. - It complains about not finding git to run "git ls-files" - I need ssconvert for Gnumeric Gnumeric dropped the Windows support so we can ignore this. - It complains that I cannot convert fig formats to pdftex - and more. If it is possible, tests that require stuff not available should be disabled on Windows. I don't know if this is possible. if not, I can life with these errors. All in all the feature works now for me and i just used it for a simple file format change. many thanks again Vincent and Kornel Uwe
Re: Many tex2lyx failing now on master
Am Donnerstag, 29. Oktober 2015 um 20:28:00, schrieb Vincent van Ravesteijn
> Op 29-10-2015 om 18:33 schreef Kornel Benko:
> > The problem was, that cmake accumulated messages from runtests.py.
> > Now runtests.py is called for each file separately.
> > Vincent, have you checked if that is OK to you?
> >
> > Kornel
>
> It indeed works much better.
>
> I just don't understand what the following is supposed to do:
>
> string(REGEX REPLACE "\\." "_" _f "LyxTestFiles${_fl}")
>
> _fl is just the file names like "test.ltx", so the name of the test
> becomes "LyxTestFilestest.ltx". This name is does not look so nice, and
> there is never ever something replaced by the line above.
Sure, it is only used as a dependency marker that test.ltx is processed.
And the name is "LyxTestFilestest_ltx", (without '.')
And there is no such test (e.g. not inserted with add_test()).
The only target here is still "updatetex2lyxtests"
> Vincent
Kornel
signature.asc
Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Kornel Benko wrote: > I see. In my installation there is no uniconvertor either. I have it installed, but it is not called by LyX directly (at least seraching for it in the complete 2.2 git folder results in 0 hits). > It is 'convert' calling this, so maybe it is not 'convert' from > image-magick? Or maybe image-magick knows that it is not good at vector graphics and delegates this, but without checking whether the command really exists. Georg
Re: Many tex2lyx failing now on master
Op 29-10-2015 om 18:33 schreef Kornel Benko:
The problem was, that cmake accumulated messages from runtests.py.
Now runtests.py is called for each file separately.
Vincent, have you checked if that is OK to you?
Kornel
It indeed works much better.
I just don't understand what the following is supposed to do:
string(REGEX REPLACE "\\." "_" _f "LyxTestFiles${_fl}")
_fl is just the file names like "test.ltx", so the name of the test
becomes "LyxTestFilestest.ltx". This name is does not look so nice, and
there is never ever something replaced by the line above.
Vincent
Re: Many tex2lyx failing now on master
Vincent van Ravesteijn wrote: > While running updatetex2lyxtest: > > algo2e.tex": >Ignoring options 'boxed' of package algorithm2e. This should be fixed one day hopefully. > test-insets.tex: >warning : Could not find included file 'Dummy Document.tex'. (2x) >warning : Could not find included file 'Dummy"Document.tex'. >warning : Could not find included file 'foo'. (2x) > ..\..\..\source\gitlyx\src\output_latex.cpp (1193): Error in > latexParagraphs: You should not mix title layouts with normal ones. This is not a problem. LyX is stoo stupid to notice that a title layout, then a comment, and then another title layout is perfectly fine (also in the exported .tex). Uwe once changed the .tex input file to get rid of this warning, but this particular construct was introduced to test a previous tex2lyx bug which caused a wrong .lyx file to be generated, so that removal was reverted. > test-insets-basic.tex: >warning : Could not find included file 'Dummy Document.tex'. (2x) We should probably disable this warning in roundtrip mode. We test here (on purpose) that tex2lyx can translate .tex files which include non-existing files. > Besides these I get a few messages saying foo.pdf already exists, Do > you want to overwrite ? Shouldn't this be written automatically given > -f flag ? Yes. Georg
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 23:35:43, schrieb Kornel Benko > Am Mittwoch, 28. Oktober 2015 um 23:29:30, schrieb Kornel Benko > > > Am Mittwoch, 28. Oktober 2015 um 23:02:08, schrieb Stephan Witt > > > > > Am 28.10.2015 um 22:49 schrieb Vincent van Ravesteijn : > > > > > > > > > It is only a bit > > > > > > annoying that I have to wait for a long time without getting any > > > > > > feedback on the progress. > > > > > > > > > > > > Vincent > > > > > > > > > > > > > > > > I don't feel guilty. I don't remember anyone saying anything about > > > > > the 'env' command before. > > > > > > > > > > What do you mean with feedback about progress? > > > > > > > > > > Probably memory fading. > > > > > > > > > > Kornel > > > > > > > > I don't mean you. > > > > > > > > I mean that I don't get any response while running the > > > > updatetex2lyxtests target, which causes me to wonder whether it is > > > > doing anything. Msvc thingie probably. > > > > > > No, I don't think so. On my Mac it is the same. Since there were no visual > > > CPU usage I suspect now it is some test for X11 availability? > > > > > > Stephan > > > > I am ready in some minutes, let me test first. > > > > Is that OK? > The problem was, that cmake accumulated messages from runtests.py. > Now runtests.py is called for each file separately. Vincent, have you checked if that is OK to you? Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 22:59:15, schrieb Stephan Witt > Am 28.10.2015 um 19:55 schrieb Kornel Benko : > > > Am Mittwoch, 28. Oktober 2015 um 18:56:39, schrieb Stephan Witt > > > >> Am 28.10.2015 um 17:19 schrieb Scott Kostyshak : > >> ... > > 2.) target updatetex2lyxtests, e.g. "make updatetex2lyxtests" > > On my Mac there is no uniconvertor command available. Neither as standard nor > from MacPorts. > > I'll attach the log output as file. I see. In my installation there is no uniconvertor either. It is 'convert' calling this, so maybe it is not 'convert' from image-magick? Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 23:29:30, schrieb Kornel Benko
> Am Mittwoch, 28. Oktober 2015 um 23:02:08, schrieb Stephan Witt
>
> > Am 28.10.2015 um 22:49 schrieb Vincent van Ravesteijn :
> >
> > > > > It is only a bit
> > > > > annoying that I have to wait for a long time without getting any
> > > > > feedback on the progress.
> > > > >
> > > > > Vincent
> > > > >
> > > >
> > > > I don't feel guilty. I don't remember anyone saying anything about the
> > > > 'env' command before.
> > > >
> > > > What do you mean with feedback about progress?
> > > >
> > > > Probably memory fading.
> > > >
> > > > Kornel
> > >
> > > I don't mean you.
> > >
> > > I mean that I don't get any response while running the updatetex2lyxtests
> > > target, which causes me to wonder whether it is doing anything. Msvc
> > > thingie probably.
> >
> > No, I don't think so. On my Mac it is the same. Since there were no visual
> > CPU usage I suspect now it is some test for X11 availability?
> >
> > Stephan
>
> I am ready in some minutes, let me test first.
>
Is that OK?
The problem was, that cmake accumulated messages from runtests.py.
Now runtests.py is called for each file separately.
Korneldiff --git a/src/tex2lyx/test/CMakeLists.txt b/src/tex2lyx/test/CMakeLists.txt
index 304e162..81cbe02 100644
--- a/src/tex2lyx/test/CMakeLists.txt
+++ b/src/tex2lyx/test/CMakeLists.txt
@@ -61,17 +61,23 @@ endforeach()
add_dependencies(lyx_run_tests ${_tex2lyx} ${_lyx})
-add_custom_command(
- OUTPUT LyxTestFiles
- COMMAND ${CMAKE_COMMAND} -E touch LyxTestFiles
- COMMAND ${CMAKE_COMMAND}
--DENVIRON="${LYX_USERDIR_VER}=${LYX_TESTS_USERDIR}"
--DCOMMAND="${LYX_PYTHON_EXECUTABLE}"
--DPARAMS="${runtestsScript}'$'${scriptDir}'${CMAKE_CURRENT_SOURCE_DIR}"
--P ${TOP_SRC_DIR}/src/tex2lyx/test/env_custom_command.cmake
- WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
- DEPENDS ${_tex2lyx} ${_lyx}
-)
+set(LyxTestFiles)
+foreach(_fl ${_tex_tests})
+ string(REGEX REPLACE "\\." "_" _f "LyxTestFiles${_fl}")
+ list(APPEND LyxTestFiles ${_f})
+
+ add_custom_command(
+OUTPUT ${_f}
+COMMAND ${CMAKE_COMMAND} -E touch ${_f}
+COMMAND ${CMAKE_COMMAND}
+ -DENVIRON="${LYX_USERDIR_VER}=${LYX_TESTS_USERDIR}"
+ -DCOMMAND="${LYX_PYTHON_EXECUTABLE}"
+ -DPARAMS="${runtestsScript}'$'${scriptDir}'${CMAKE_CURRENT_SOURCE_DIR}'${_fl}"
+ -P ${TOP_SRC_DIR}/src/tex2lyx/test/env_custom_command.cmake
+WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
+DEPENDS ${_tex2lyx} ${_lyx}
+ )
+endforeach()
-add_custom_target(updatetex2lyxtests DEPENDS LyxTestFiles)
+add_custom_target(updatetex2lyxtests DEPENDS ${LyxTestFiles})
set_target_properties(updatetex2lyxtests PROPERTIES FOLDER "tests/tex2lyx")
signature.asc
Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 23:02:08, schrieb Stephan Witt > Am 28.10.2015 um 22:49 schrieb Vincent van Ravesteijn : > > > > > It is only a bit > > > > annoying that I have to wait for a long time without getting any > > > > feedback on the progress. > > > > > > > > Vincent > > > > > > > > > > I don't feel guilty. I don't remember anyone saying anything about the > > > 'env' command before. > > > > > > What do you mean with feedback about progress? > > > > > > Probably memory fading. > > > > > > Kornel > > > > I don't mean you. > > > > I mean that I don't get any response while running the updatetex2lyxtests > > target, which causes me to wonder whether it is doing anything. Msvc > > thingie probably. > > No, I don't think so. On my Mac it is the same. Since there were no visual > CPU usage I suspect now it is some test for X11 availability? > > Stephan I am ready in some minutes, let me test first. Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am 28.10.2015 um 22:49 schrieb Vincent van Ravesteijn : > > > It is only a bit > > > annoying that I have to wait for a long time without getting any > > > feedback on the progress. > > > > > > Vincent > > > > > > > I don't feel guilty. I don't remember anyone saying anything about the > > 'env' command before. > > > > What do you mean with feedback about progress? > > > > Probably memory fading. > > > > Kornel > > I don't mean you. > > I mean that I don't get any response while running the updatetex2lyxtests > target, which causes me to wonder whether it is doing anything. Msvc thingie > probably. No, I don't think so. On my Mac it is the same. Since there were no visual CPU usage I suspect now it is some test for X11 availability? Stephan
Re: Many tex2lyx failing now on master
Am 28.10.2015 um 19:55 schrieb Kornel Benko : > Am Mittwoch, 28. Oktober 2015 um 18:56:39, schrieb Stephan Witt > >> Am 28.10.2015 um 17:19 schrieb Scott Kostyshak : >> >>> On Wed, Oct 28, 2015 at 05:02:33PM +0100, Kornel Benko wrote: Am Dienstag, 27. Oktober 2015 um 14:33:59, schrieb Kornel Benko > Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko > >> Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van >> Ravesteijn > > Well yes, there is no "env" command on Windows. > > ... > > I tried to remove 'env'. The attached still works for me. > >>> Vincent >> It looks like the relevant code to set environment for tests added by add_test() command is in the cmake-sources at lest since february 2009. Searching the net, there are posts mentioning cmake2.6 with this feature. So I cannot imagine, why the patch should break anything on linux. But it may help for Windows. (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) >>> >>> And for Mac now that Stephan has taken a crack at running them, >>> hopefully he will let us know if this commit changes anything (but as >>> you say, this is not expected). >> >> No, it doesn't change anything on Mac. With git master at >> e430586ec36e2bb1646191682f434e45fb871d26 I have: >> >> $ ctest -C Debug -R 'export/doc/.*_texF$' -j8 >> ... >> 77% tests passed, 70 tests failed out of 302 >> >> Label Time Summary: >> export = 8710.41 sec >> reverted= 935.73 sec >> >> Total Test time (real) = 1119.05 sec >> >> Stephan > > Stephan, the change should effect only > 1.) ctest -R check_layout 100% tests passed, 0 tests failed out of 163 Label Time Summary: layout= 80.18 sec module= 8.74 sec Total Test time (real) = 90.10 sec > 2.) target updatetex2lyxtests, e.g. "make updatetex2lyxtests" On my Mac there is no uniconvertor command available. Neither as standard nor from MacPorts. I'll attach the log output as file. Stephan PS: The "normal" TeX distribution on a Mac is MacTeX 2015 - similar to a TeXlive 2015 ++ The available packages are listed in the configure output. updatetex2lyxtests.log Description: Binary data
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 22:49:27, schrieb Vincent van Ravesteijn > > > It is only a bit > > > annoying that I have to wait for a long time without getting any > > > feedback on the progress. > > > > > > Vincent > > > > > > > I don't feel guilty. I don't remember anyone saying anything about the > 'env' command before. > > > > What do you mean with feedback about progress? > > > > Probably memory fading. > > > > Kornel > > I don't mean you. > > I mean that I don't get any response while running the updatetex2lyxtests > target, which causes me to wonder whether it is doing anything. Msvc > thingie probably. I see. I will look into this. > Last I removed the git ls-files warning from the source as these error > messages seem to cause the deadlock in tex2lyx. > > Vincent Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
> > It is only a bit > > annoying that I have to wait for a long time without getting any > > feedback on the progress. > > > > Vincent > > > > I don't feel guilty. I don't remember anyone saying anything about the 'env' command before. > > What do you mean with feedback about progress? > > Probably memory fading. > > Kornel I don't mean you. I mean that I don't get any response while running the updatetex2lyxtests target, which causes me to wonder whether it is doing anything. Msvc thingie probably. Last I removed the git ls-files warning from the source as these error messages seem to cause the deadlock in tex2lyx. Vincent
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 21:28:29, schrieb Vincent van Ravesteijn > Op 28-10-2015 om 17:02 schreef Kornel Benko: > > It looks like the relevant code to set environment for tests added by > > add_test() command > > is in the cmake-sources at lest since february 2009. Searching the net, > > there are posts > > mentioning cmake2.6 with this feature. > > > > So I cannot imagine, why the patch should break anything on linux. But it > > may help for Windows. > > (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) > > > > > > It seems to work for me. > > While running updatetex2lyxtest: > > algo2e.tex": >Ignoring options 'boxed' of package algorithm2e. > > test-insets.tex: >warning : Could not find included file 'Dummy Document.tex'. (2x) >warning : Could not find included file 'Dummy"Document.tex'. >warning : Could not find included file 'foo'. (2x) > ..\..\..\source\gitlyx\src\output_latex.cpp (1193): Error in > latexParagraphs: You should not mix title layouts with normal ones. This is the same as on linux ... 'Dummy Document.tex' was removed from the repo. See commit b188e74cd6b089be3d63d905bf9db532e53f54c5 > test-insets-basic.tex: >warning : Could not find included file 'Dummy Document.tex'. (2x) > > Besides these I get a few messages saying foo.pdf already exists, Do > you want to overwrite ? Shouldn't this be written automatically given > -f flag ? Unfortunately the used script writes into the repo directory also files which are not under git control. Cmake uses the same script as autoconf > Except for the gnumeric stuff, it all seems to work. It is only a bit > annoying that I have to wait for a long time without getting any > feedback on the progress. > > Vincent > I don't feel guilty. I don't remember anyone saying anything about the 'env' command before. What do you mean with feedback about progress? Probably memory fading. Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Op 28-10-2015 om 17:02 schreef Kornel Benko: It looks like the relevant code to set environment for tests added by add_test() command is in the cmake-sources at lest since february 2009. Searching the net, there are posts mentioning cmake2.6 with this feature. So I cannot imagine, why the patch should break anything on linux. But it may help for Windows. (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) It seems to work for me. While running updatetex2lyxtest: algo2e.tex": Ignoring options 'boxed' of package algorithm2e. test-insets.tex: warning : Could not find included file 'Dummy Document.tex'. (2x) warning : Could not find included file 'Dummy"Document.tex'. warning : Could not find included file 'foo'. (2x) ..\..\..\source\gitlyx\src\output_latex.cpp (1193): Error in latexParagraphs: You should not mix title layouts with normal ones. test-insets-basic.tex: warning : Could not find included file 'Dummy Document.tex'. (2x) Besides these I get a few messages saying foo.pdf already exists, Do you want to overwrite ? Shouldn't this be written automatically given -f flag ? Except for the gnumeric stuff, it all seems to work. It is only a bit annoying that I have to wait for a long time without getting any feedback on the progress. Vincent
Re: Many tex2lyx failing now on master
On Wed, Oct 28, 2015 at 06:56:39PM +0100, Stephan Witt wrote: > Am 28.10.2015 um 17:19 schrieb Scott Kostyshak : > > > On Wed, Oct 28, 2015 at 05:02:33PM +0100, Kornel Benko wrote: > >> Am Dienstag, 27. Oktober 2015 um 14:33:59, schrieb Kornel Benko > >> > >>> Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko > >>> > Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van > Ravesteijn > >>> > >>> Well yes, there is no "env" command on Windows. > >>> > >>> ... > >>> > >>> I tried to remove 'env'. The attached still works for me. > >>> > > Vincent > > >> > >> It looks like the relevant code to set environment for tests added by > >> add_test() command > >> is in the cmake-sources at lest since february 2009. Searching the net, > >> there are posts > >> mentioning cmake2.6 with this feature. > >> > >> So I cannot imagine, why the patch should break anything on linux. But it > >> may help for Windows. > >> (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) > > > > And for Mac now that Stephan has taken a crack at running them, > > hopefully he will let us know if this commit changes anything (but as > > you say, this is not expected). > > No, it doesn't change anything on Mac. With git master at > e430586ec36e2bb1646191682f434e45fb871d26 I have: > > $ ctest -C Debug -R 'export/doc/.*_texF$' -j8 > ... > 77% tests passed, 70 tests failed out of 302 > > Label Time Summary: > export = 8710.41 sec > reverted= 935.73 sec > > Total Test time (real) = 1119.05 sec Great thanks for checking this! Scott
Re: Many tex2lyx failing now on master
Am Mittwoch, 28. Oktober 2015 um 18:56:39, schrieb Stephan Witt > Am 28.10.2015 um 17:19 schrieb Scott Kostyshak : > > > On Wed, Oct 28, 2015 at 05:02:33PM +0100, Kornel Benko wrote: > >> Am Dienstag, 27. Oktober 2015 um 14:33:59, schrieb Kornel Benko > >> > >>> Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko > >>> > Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van > Ravesteijn > >>> > >>> Well yes, there is no "env" command on Windows. > >>> > >>> ... > >>> > >>> I tried to remove 'env'. The attached still works for me. > >>> > > Vincent > > >> > >> It looks like the relevant code to set environment for tests added by > >> add_test() command > >> is in the cmake-sources at lest since february 2009. Searching the net, > >> there are posts > >> mentioning cmake2.6 with this feature. > >> > >> So I cannot imagine, why the patch should break anything on linux. But it > >> may help for Windows. > >> (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) > > > > And for Mac now that Stephan has taken a crack at running them, > > hopefully he will let us know if this commit changes anything (but as > > you say, this is not expected). > > No, it doesn't change anything on Mac. With git master at > e430586ec36e2bb1646191682f434e45fb871d26 I have: > > $ ctest -C Debug -R 'export/doc/.*_texF$' -j8 > ... > 77% tests passed, 70 tests failed out of 302 > > Label Time Summary: > export = 8710.41 sec > reverted= 935.73 sec > > Total Test time (real) = 1119.05 sec > > Stephan Stephan, the change should effect only 1.) ctest -R check_layout 2.) target updatetex2lyxtests, e.g. "make updatetex2lyxtests" so I don't wonder too much :) Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am 28.10.2015 um 17:19 schrieb Scott Kostyshak : > On Wed, Oct 28, 2015 at 05:02:33PM +0100, Kornel Benko wrote: >> Am Dienstag, 27. Oktober 2015 um 14:33:59, schrieb Kornel Benko >> >>> Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko >>> Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van Ravesteijn >>> >>> Well yes, there is no "env" command on Windows. >>> >>> ... >>> >>> I tried to remove 'env'. The attached still works for me. >>> > Vincent >> >> It looks like the relevant code to set environment for tests added by >> add_test() command >> is in the cmake-sources at lest since february 2009. Searching the net, >> there are posts >> mentioning cmake2.6 with this feature. >> >> So I cannot imagine, why the patch should break anything on linux. But it >> may help for Windows. >> (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) > > And for Mac now that Stephan has taken a crack at running them, > hopefully he will let us know if this commit changes anything (but as > you say, this is not expected). No, it doesn't change anything on Mac. With git master at e430586ec36e2bb1646191682f434e45fb871d26 I have: $ ctest -C Debug -R 'export/doc/.*_texF$' -j8 ... 77% tests passed, 70 tests failed out of 302 Label Time Summary: export = 8710.41 sec reverted= 935.73 sec Total Test time (real) = 1119.05 sec Stephan
Re: Many tex2lyx failing now on master
On Wed, Oct 28, 2015 at 05:02:33PM +0100, Kornel Benko wrote: > Am Dienstag, 27. Oktober 2015 um 14:33:59, schrieb Kornel Benko > > > Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko > > > > > Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van Ravesteijn > > > > > > > >> > > > > >> Well yes, there is no "env" command on Windows. > > > > ... > > > > I tried to remove 'env'. The attached still works for me. > > > > > > Vincent > > > > > It looks like the relevant code to set environment for tests added by > add_test() command > is in the cmake-sources at lest since february 2009. Searching the net, there > are posts > mentioning cmake2.6 with this feature. > > So I cannot imagine, why the patch should break anything on linux. But it may > help for Windows. > (new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir) And for Mac now that Stephan has taken a crack at running them, hopefully he will let us know if this commit changes anything (but as you say, this is not expected). > Scott, OK to commit? Yes. Scott
Re: Many tex2lyx failing now on master
Am Dienstag, 27. Oktober 2015 um 14:33:59, schrieb Kornel Benko
> Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko
>
> > Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van Ravesteijn
> >
> > > >>
> > > >> Well yes, there is no "env" command on Windows.
>
> ...
>
> I tried to remove 'env'. The attached still works for me.
>
> > > Vincent
> >
It looks like the relevant code to set environment for tests added by
add_test() command
is in the cmake-sources at lest since february 2009. Searching the net, there
are posts
mentioning cmake2.6 with this feature.
So I cannot imagine, why the patch should break anything on linux. But it may
help for Windows.
(new file env_custom_command.cmake belongs to src/tex2lyx/test/ dir)
Scott, OK to commit?
Kornel# This file is part of LyX, the document processor.
# Licence details can be found in the file COPYING.
#
# Copyright (c) 2015 Kornel Benko
#
# Environ and parameter strings are separated by "'"
# Script should be called like:
# add_custom_command(
# COMMAND ${CMAKE_COMMAND}
# -DENVIRON="${LYX_USERDIR_VER}=${LYX_TESTS_USERDIR}" \
# -DCOMMAND="${LYX_PYTHON_EXECUTABLE}" \
# -DPARAMS="${runtestsScript}'$'${scriptDir}'${CMAKE_CURRENT_SOURCE_DIR}"
# -P ${TOP_SRC_DIR}/src/tex2lyx/test/env_custom_command.cmake
# WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
# DEPENDS ...
# )
string(REGEX MATCHALL "[^']+" ENVIRON_LIST "${ENVIRON}")
foreach(_env ${ENVIRON_LIST})
if (_env MATCHES "^\([^=]+\)=\(.*\)$")
set(ENV{${CMAKE_MATCH_1}} ${CMAKE_MATCH_2})
#message(STATUS "setting env ${CMAKE_MATCH_1}=${CMAKE_MATCH_2}")
else()
message(FATAL_ERROR "Wrong environment string \"${_env}\"")
endif()
endforeach()
string(REGEX MATCHALL "[^']+" parameters "${PARAMS}")
execute_process(COMMAND "${COMMAND}" ${parameters}
RESULT_VARIABLE _err
OUTPUT_VARIABLE _out)
message(STATUS ${_out})
#message(STATUS "Error output of \"${COMMAND}\" " ${parameters} " = ${_err}")
string(COMPARE NOTEQUAL ${_err} 0 _erg)
if(_erg)
message(FATAL_ERROR "\"${COMMAND}\" ${parameters} failed")
endif()
diff --git a/src/tests/CMakeLists.txt b/src/tests/CMakeLists.txt
index bfbcdfd..afcceb5 100644
--- a/src/tests/CMakeLists.txt
+++ b/src/tests/CMakeLists.txt
@@ -38,11 +38,11 @@ foreach(bns ${layout_files} ${module_files})
if("${_checktype}" STREQUAL "module")
file(WRITE "${LYX_TESTS_USERDIR}/layouts/test${bn}.layout" "#% Do not delete the line below; configure depends on this\n")
file(APPEND "${LYX_TESTS_USERDIR}/layouts/test${bn}.layout" "# \\DeclareLaTeXClass{test${bn}}\n")
-
file(APPEND "${LYX_TESTS_USERDIR}/layouts/test${bn}.layout" "Format 55\n")
file(APPEND "${LYX_TESTS_USERDIR}/layouts/test${bn}.layout" "Input ${bns}\n")
add_test(NAME "check_layout/${bns}"
- COMMAND env LYX_USERDIR_22x=${LYX_TESTS_USERDIR} $ "${LYX_TESTS_USERDIR}/layouts/test${bn}" "${CMAKE_CURRENT_BINARY_DIR}/${bn}.out${_checktype}")
+ COMMAND $ "${LYX_TESTS_USERDIR}/layouts/test${bn}" "${CMAKE_CURRENT_BINARY_DIR}/${bn}.out${_checktype}")
+set_tests_properties("check_layout/${bns}" PROPERTIES ENVIRONMENT ${LYX_USERDIR_VER}=${LYX_TESTS_USERDIR})
else()
add_test(NAME "check_layout/${bns}"
COMMAND check_layout "${TOP_SRC_DIR}/lib/layouts/${bn}" "${CMAKE_CURRENT_BINARY_DIR}/${bn}.out${_checktype}")
diff --git a/src/tex2lyx/test/CMakeLists.txt b/src/tex2lyx/test/CMakeLists.txt
index c513d6e..304e162 100644
--- a/src/tex2lyx/test/CMakeLists.txt
+++ b/src/tex2lyx/test/CMakeLists.txt
@@ -64,8 +64,11 @@ add_dependencies(lyx_run_tests ${_tex2lyx} ${_lyx})
add_custom_command(
OUTPUT LyxTestFiles
COMMAND ${CMAKE_COMMAND} -E touch LyxTestFiles
- COMMAND env LYX_USERDIR_VER=${LYX_USERDIR_VER}
-${LYX_PYTHON_EXECUTABLE} "${runtestsScript}" $ "${scriptDir}" "${CMAKE_CURRENT_SOURCE_DIR}"
+ COMMAND ${CMAKE_COMMAND}
+-DENVIRON="${LYX_USERDIR_VER}=${LYX_TESTS_USERDIR}"
+-DCOMMAND="${LYX_PYTHON_EXECUTABLE}"
+-DPARAMS="${runtestsScript}'$'${scriptDir}'${CMAKE_CURRENT_SOURCE_DIR}"
+-P ${TOP_SRC_DIR}/src/tex2lyx/test/env_custom_command.cmake
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
DEPENDS ${_tex2lyx} ${_lyx}
)
signature.asc
Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Vincent van Ravesteijn wrote: > Well yes, there is no "env" command on Windows. > > This is used in both "src/tests/CMakeLists.txt" and > "src/tex2lyx/CMakeLists.txt". > > Besides that, running is a nightmare: > - Somehow tex2lyx often deadlocks when outputting errors, This would be a bug in tex2lyx which needs to be investigated. I would do it, but of course I don't see it on linux:-( > - I need to have python in the path, This is indeed a requirement right now because of our home made test driver. Maybe python can be found by some cmake trickery, but I don't know. In the long run we need a better test driver, but as a short term solution I suggest to create a batch file for running the tests that adds python to the path temporarily. > - It complains about not finding git to run "git ls-files" > - I need ssconvert for Gnumeric > - It complains that I cannot convert fig formats to pdftex These three come from LyX, when it is called to re-export the .lyx files created by tex2lyx. You are completely right that these dependencies are not needed for testing tex2lyx, and we should get rid of them. I'll put this on my list, but up to now nobody noticed since everybody who ran the tests had these things installed. Getting rid of these dependencies is easy, we simply need to instruct LyX on the command line that we are only interested in the pure .tex export, and do not want a LaTeX-reday export with all included objects converted to the correct format. Georg
Re: Many tex2lyx failing now on master
Am Dienstag, 27. Oktober 2015 um 12:34:47, schrieb Kornel Benko
> Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van Ravesteijn
>
> > >>
> > >> Well yes, there is no "env" command on Windows.
...
I tried to remove 'env'. The attached still works for me.
> > Vincent
>
Korneldiff --git a/development/autotests/ExportTests.cmake b/development/autotests/ExportTests.cmake
index 50b4aa4..e241997 100644
--- a/development/autotests/ExportTests.cmake
+++ b/development/autotests/ExportTests.cmake
@@ -138,7 +138,9 @@ endmacro()
loadTestList(revertedTests revertedTests)
loadTestList(ignoredTests ignoredTests)
-foreach(libsubfolder doc examples templates)
+foreach(libsubfolderx lib/doc lib/examples lib/templates development/mathmacros)
+ set(LIBSUB_SRC_DIR "${TOP_SRC_DIR}/${libsubfolderx}")
+ string(REGEX REPLACE "^(lib|development)/" "" libsubfolder "${libsubfolderx}")
set(LIBSUB_SRC_DIR "${TOP_SRC_DIR}/lib/${libsubfolder}")
file(GLOB_RECURSE lyx_files RELATIVE "${LIBSUB_SRC_DIR}" "${LIBSUB_SRC_DIR}/*.lyx")
list(SORT lyx_files)
diff --git a/src/tests/CMakeLists.txt b/src/tests/CMakeLists.txt
index bfbcdfd..3090222 100644
--- a/src/tests/CMakeLists.txt
+++ b/src/tests/CMakeLists.txt
@@ -42,7 +42,8 @@ foreach(bns ${layout_files} ${module_files})
file(APPEND "${LYX_TESTS_USERDIR}/layouts/test${bn}.layout" "Format 55\n")
file(APPEND "${LYX_TESTS_USERDIR}/layouts/test${bn}.layout" "Input ${bns}\n")
add_test(NAME "check_layout/${bns}"
- COMMAND env LYX_USERDIR_22x=${LYX_TESTS_USERDIR} $ "${LYX_TESTS_USERDIR}/layouts/test${bn}" "${CMAKE_CURRENT_BINARY_DIR}/${bn}.out${_checktype}")
+ COMMAND $ "${LYX_TESTS_USERDIR}/layouts/test${bn}" "${CMAKE_CURRENT_BINARY_DIR}/${bn}.out${_checktype}")
+set_tests_properties("check_layout/${bns}" PROPERTIES ENVIRONMENT LYX_USERDIR_22x=${LYX_TESTS_USERDIR})
else()
add_test(NAME "check_layout/${bns}"
COMMAND check_layout "${TOP_SRC_DIR}/lib/layouts/${bn}" "${CMAKE_CURRENT_BINARY_DIR}/${bn}.out${_checktype}")
diff --git a/src/tex2lyx/test/CMakeLists.txt b/src/tex2lyx/test/CMakeLists.txt
index c513d6e..7c62c63 100644
--- a/src/tex2lyx/test/CMakeLists.txt
+++ b/src/tex2lyx/test/CMakeLists.txt
@@ -64,8 +64,16 @@ add_dependencies(lyx_run_tests ${_tex2lyx} ${_lyx})
add_custom_command(
OUTPUT LyxTestFiles
COMMAND ${CMAKE_COMMAND} -E touch LyxTestFiles
- COMMAND env LYX_USERDIR_VER=${LYX_USERDIR_VER}
-${LYX_PYTHON_EXECUTABLE} "${runtestsScript}" $ "${scriptDir}" "${CMAKE_CURRENT_SOURCE_DIR}"
+ COMMAND ${CMAKE_COMMAND}
+-DENV1="${LYX_USERDIR_VER}=${LYX_TESTS_USERDIR}"
+-DCOMMAND="${LYX_PYTHON_EXECUTABLE}"
+-DPARAM1="${runtestsScript}"
+-DPARAM2=$
+-DPARAM3="${scriptDir}"
+-DPARAM4="${CMAKE_CURRENT_SOURCE_DIR}"
+-P ${TOP_SRC_DIR}/src/tex2lyx/test/env_custom_command.cmake
+ #COMMAND env LYX_USERDIR_VER=${LYX_USERDIR_VER}
+ # ${LYX_PYTHON_EXECUTABLE} "${runtestsScript}" $ "${scriptDir}" "${CMAKE_CURRENT_SOURCE_DIR}"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
DEPENDS ${_tex2lyx} ${_lyx}
)
signature.asc
Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am Dienstag, 27. Oktober 2015 um 11:48:20, schrieb Vincent van Ravesteijn
> >>
> >> Well yes, there is no "env" command on Windows.
> >
> > Sorry, I did not knew about this.
> >
> >> This is used in both "src/tests/CMakeLists.txt" and
> >> "src/tex2lyx/CMakeLists.txt".
> >
> > I will look into this. Unfortunately the 'add_test()' command does hot have
> > a parameter to set the environment.
> > Probably I have to create a cmake script for this :(
>
> From CMake version 3.0.something and higher you can use:
>
> ${CMAKE_COMMAND} -E env NAME=VALUE COMMAND (with COMMAND=python
> runtests... etc.)
This is valid for add_custom_command(), but not for add_test()
> For lower versions of CMake (we say we support from 2.6.8 IIRC), I
> didn't find a solution.
Unfortunately we cannot demand this version now.
In the meantime I found one working for add_test(). (I have cmake 3.4, not
sure, if this is also valid for cmake 2.6)
set_tests_properties(... PROPERTIES ENVIRONMENT
LYX_USERDIR_22x=${LYX_TESTS_USERDIR})
used _after_ the command add_test(NAME ...)
> >
> >> Besides that, running is a nightmare:
> >> - Somehow tex2lyx often deadlocks when outputting errors,
> >
> > It never happened on linux. 'tex2lyx' tests are run sequentially, so I
> > didn't see possible deadlocks ...
>
> I don't know. It just gets stuck in SystemCall::stderr ...
Sorry
> >
> >> - I need to have python in the path,
> >
> > It should be the same 'python' as found in the configuration phase. Why the
> > need for PATH, I don't understand
> > (I mean in the tex2lyx tests, have I lost track?)
> >
> >> - It complains about not finding git to run "git ls-files"
> >
> > Where? This command-parameter is under linux not valid. I did not see it.
>
> During the execution of updatetex2lyxtests, it complains about git...
> not a big issue, but polluting the output and it seems a bit non-sense
> as I don't have a GUI. Maybe something to fix some time.
Yes
> Vincent
Kornel
signature.asc
Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
>>
>> Well yes, there is no "env" command on Windows.
>
> Sorry, I did not knew about this.
>
>> This is used in both "src/tests/CMakeLists.txt" and
>> "src/tex2lyx/CMakeLists.txt".
>
> I will look into this. Unfortunately the 'add_test()' command does hot have a
> parameter to set the environment.
> Probably I have to create a cmake script for this :(
>From CMake version 3.0.something and higher you can use:
${CMAKE_COMMAND} -E env NAME=VALUE COMMAND (with COMMAND=python
runtests... etc.)
For lower versions of CMake (we say we support from 2.6.8 IIRC), I
didn't find a solution.
>
>> Besides that, running is a nightmare:
>> - Somehow tex2lyx often deadlocks when outputting errors,
>
> It never happened on linux. 'tex2lyx' tests are run sequentially, so I didn't
> see possible deadlocks ...
I don't know. It just gets stuck in SystemCall::stderr ...
>
>> - I need to have python in the path,
>
> It should be the same 'python' as found in the configuration phase. Why the
> need for PATH, I don't understand
> (I mean in the tex2lyx tests, have I lost track?)
>
>> - It complains about not finding git to run "git ls-files"
>
> Where? This command-parameter is under linux not valid. I did not see it.
During the execution of updatetex2lyxtests, it complains about git...
not a big issue, but polluting the output and it seems a bit non-sense
as I don't have a GUI. Maybe something to fix some time.
>
>> - I need ssconvert for Gnumeric
These were errors during the execution of updatetex2lyxtests. It is a
bit strange I need 3rd party dependencies for testing tex2lyx.
>> - It complains that I cannot convert fig formats to pdftex
>> - and more.
Again, these were errors during the execution of updatetex2lyxtests.
Vincent
Re: Many tex2lyx failing now on master
Am Montag, 26. Oktober 2015 um 21:36:21, schrieb Vincent van Ravesteijn > > > When I try to compile this file via the console or directly in MSVC I > > get always this error: > > > > Building Custom Rule D:/LyXGit/Master/src/tex2lyx/test/CMakeLists.txt > > CMake does not need to re-run because > > D:\LyXGit\Master\compile-result\src\tex2lyx\test\CMakeFiles\generate.stamp > > is up-to-date. > > Generating LyxTestFiles > > The command "env" is either misspelled or could not be found > > C:\Program Files > > (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets(151,5): > > error MSB6006: "cmd.exe" was finished with code 9009 > > > > Vincent, do you perhaps have an idea? > > Well yes, there is no "env" command on Windows. Sorry, I did not knew about this. > This is used in both "src/tests/CMakeLists.txt" and > "src/tex2lyx/CMakeLists.txt". I will look into this. Unfortunately the 'add_test()' command does hot have a parameter to set the environment. Probably I have to create a cmake script for this :( > Besides that, running is a nightmare: > - Somehow tex2lyx often deadlocks when outputting errors, It never happened on linux. 'tex2lyx' tests are run sequentially, so I didn't see possible deadlocks ... > - I need to have python in the path, It should be the same 'python' as found in the configuration phase. Why the need for PATH, I don't understand (I mean in the tex2lyx tests, have I lost track?) > - It complains about not finding git to run "git ls-files" Where? This command-parameter is under linux not valid. I did not see it. > - I need ssconvert for Gnumeric Again, where? I could not find it in our cmake relevant sources. > - It complains that I cannot convert fig formats to pdftex > - and more. > > Vincent Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
When I try to compile this file via the console or directly in MSVC I get always this error: Building Custom Rule D:/LyXGit/Master/src/tex2lyx/test/CMakeLists.txt CMake does not need to re-run because D:\LyXGit\Master\compile-result\src\tex2lyx\test\CMakeFiles\generate.stamp is up-to-date. Generating LyxTestFiles The command "env" is either misspelled or could not be found C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets(151,5): error MSB6006: "cmd.exe" was finished with code 9009 Vincent, do you perhaps have an idea? Well yes, there is no "env" command on Windows. This is used in both "src/tests/CMakeLists.txt" and "src/tex2lyx/CMakeLists.txt". Besides that, running is a nightmare: - Somehow tex2lyx often deadlocks when outputting errors, - I need to have python in the path, - It complains about not finding git to run "git ls-files" - I need ssconvert for Gnumeric - It complains that I cannot convert fig formats to pdftex - and more. Vincent
Re: Many tex2lyx failing now on master
Richard Heck wrote: > Yes, I was just talking about the mention of lyx2lyx, so maybe we are at > cross purposes. But one certainly could have two format N .lyx files and > two format N+1 .lyx files---one each for forward and one for backward > conversion---and test whether lyx2lyx gets you the right results both > times. Or we could just focus on the forward conversion, which is a lot > more important to most people. Having tests for the forward lyx2lyx conversion would already be a big improvement. This would also allow changes in the lyx2lyx infrastructure. Those are impossible to do now, because we do not have tests to ensure that older conversions do still work. Georg
Re: Many tex2lyx failing now on master
On 10/23/2015 02:17 PM, Guenter Milde wrote: On 2015-10-22, Richard Heck wrote: On 10/22/2015 04:23 PM, Georg Baum wrote: Guenter Milde wrote: Maybe a suggestion to provide a minimal example of the new feature requiring the file format change in form of a *.lyx or *.tex file that can either be simply added to the test base (if tex2lyx works fine) or put on in a set of "TODO examples". Whether we provide a directory for such incoming test examples (so that test-experts can review them and bring them in line with the other tests) or ask to send them to the devel list or whatever should be agreed on by people caring for the test suite. That would be very useful indeed (also for producing a lyx2lyx testsuite). This is more difficult, since we in no way guarantee round trip equivalence. Or am I missing something? (I'm not well informed about automated testing, etc.) My suggestion was not (necessarily) about round-trip but in first place about import (tex2lyx). Yes, I was just talking about the mention of lyx2lyx, so maybe we are at cross purposes. But one certainly could have two format N .lyx files and two format N+1 .lyx files---one each for forward and one for backward conversion---and test whether lyx2lyx gets you the right results both times. Or we could just focus on the forward conversion, which is a lot more important to most people. Richard
Re: Many tex2lyx failing now on master
On 2015-10-22, Richard Heck wrote: > On 10/22/2015 04:23 PM, Georg Baum wrote: >> Guenter Milde wrote: >>> Maybe a suggestion to provide a minimal example of the new feature >>> requiring the file format change in form of a *.lyx or *.tex file that >>> can either be simply added to the test base (if tex2lyx works fine) or >>> put on in a set of "TODO examples". >>> Whether we provide a directory for such incoming test examples (so that >>> test-experts can review them and bring them in line with the other tests) >>> or ask to send them to the devel list or whatever should be agreed on by >>> people caring for the test suite. >> That would be very useful indeed (also for producing a lyx2lyx testsuite). > This is more difficult, since we in no way guarantee round trip > equivalence. Or am I missing something? (I'm not well informed about > automated testing, etc.) My suggestion was not (necessarily) about round-trip but in first place about import (tex2lyx). We don't ensure round-trips without change for arbitrary input indeed. However in "well behaved" use cases there should be no change after the first round, i.e. it should be possible to provide example files (both *.tex and *.lyx) that must not to change during a round trip. Just like "tex - lyx" pairs of files, these will need adaption for intended changes (like the file version number or improved export/import). Günter
Re: Many tex2lyx failing now on master
Am 20.10.2015 um 10:45 schrieb Kornel Benko: > You should use the correct target for your platform. > This 'make updatetex2lyxtests' is meant for linux. > For Windows it may be something with appropriate solution, e.g. > 'msbuild updatetex2lyxtests.sln' (I am guessing here, Vincent will know) Thanks Kornel, I invested now some time but still cannot get it to work: I found a file updatetex2lyxtests.vcxproj in the folder D:\LyXGit\Master\compile-result\src\tex2lyx\test (D:\LyXGit\Master\compile-result is my build directory where the compiled lyx.exe will be stored.) When I try to compile this file via the console or directly in MSVC I get always this error: Building Custom Rule D:/LyXGit/Master/src/tex2lyx/test/CMakeLists.txt CMake does not need to re-run because D:\LyXGit\Master\compile-result\src\tex2lyx\test\CMakeFiles\generate.stamp is up-to-date. Generating LyxTestFiles The command "env" is either misspelled or could not be found C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets(151,5): error MSB6006: "cmd.exe" was finished with code 9009 Vincent, do you perhaps have an idea? thanks and regards Uwe
Re: Many tex2lyx failing now on master
Am 20.10.2015 um 17:12 schrieb Guenter Milde: I updated the lyxformat number in the src/tex2lyx/test/*.lyx.lyx files and now all tex2lyx tests pass here. Commited at a0e2d48a569510dce6f3f288ee7a14e239cd4de2/lyxgit many thanks Günther regards Uwe
Re: Many tex2lyx failing now on master
On 10/22/2015 04:23 PM, Georg Baum wrote: Guenter Milde wrote: Maybe a suggestion to provide a minimal example of the new feature requiring the file format change in form of a *.lyx or *.tex file that can either be simply added to the test base (if tex2lyx works fine) or put on in a set of "TODO examples". Whether we provide a directory for such incoming test examples (so that test-experts can review them and bring them in line with the other tests) or ask to send them to the devel list or whatever should be agreed on by people caring for the test suite. That would be very useful indeed (also for producing a lyx2lyx testsuite). This is more difficult, since we in no way guarantee round trip equivalence. Or am I missing something? (I'm not well informed about automated testing, etc.) Richard
Re: Many tex2lyx failing now on master
Guenter Milde wrote: > Maybe a suggestion to provide a minimal example of the new feature > requiring the file format change in form of a *.lyx or *.tex file that > can either be simply added to the test base (if tex2lyx works fine) or > put on in a set of "TODO examples". > > Whether we provide a directory for such incoming test examples (so that > test-experts can review them and bring them in line with the other tests) > or ask to send them to the devel list or whatever should be agreed on by > people caring for the test suite. That would be very useful indeed (also for producing a lyx2lyx testsuite). I added a section to Development.lyx, but kept it a bit unspecific, since I do not have the time currently to work on the needed infrastructure. Georg
Re: Many tex2lyx failing now on master
Guenter Milde wrote: > On 2015-10-20, Georg Baum wrote: > >> Just to avoid future errors: I assume that you did check before that a >> simple update of the format number is OK, and that no tex2lyx source code >> changes are needed? How to do such checks in described in section 2.3 of >> Development.lyx, item 5. > > Actually, no. (My aim was just to ensure the tests work again, not to make > a full review of the fileformat change.) Well, if the tests pass, then this means that the parts of tex2lyx that are tested by them work correctly. However, if the references are updated without a check whether the file format change needs an adjustment in tex2lyx, then the tests could potentially lie. Fortunately we do not have many examples where this happened, because the tests are now mostly updated correctly. However, one case where the test update was not done immediately is http://www.lyx.org/trac/changeset/c246076e/lyxgit/. If for this change only the file format number of the test references would have been updated, then wrong output would have been hidden. As you can see in http://www.lyx.org/trac/changeset/9c33bddf/lyxgit/, the update did also require a change in tex2lyx itself. For this reason it is better to not update the tests, if one does not want to do the necessary checks for correctness as well. > From the posts in this thread, I realized that the version number was > changed without updating the reference files. > > I run the tests and found that only the *.lyx.lyx reference files > produced errors, and the only difference was the version number. > > Then I changed just the last digit of the number via a grep-replace in my > text editor. This is the method to fix false positives I am used to from > my work on test driven development of "Docutils". It ensures that no > change can go unnoticed. > I did not touch any files with LyX but also not use "make updatetests". Unfortunately even this manual procedure does not ensure that the file format update does not hide a bug. In the example I gave above, it would have lead to passing tests, although the output would have been wrong. > @Uwe: could you please check, if the improvements that led to the file > format change require changes in tex2lyx and either add a note to > "src/tex2lyx/TODO.txt" if yes? In order to stop unneeded work I did the check now. Fortunately it is very simple to do in this case: Section 2.3 of Development.lyx says that simply updating the file format number is safe if the lyx2lyx conversion from the previous format to the new one is empty. Since this is the case for the conversion from 497 to 498, all is fine now. > In any case, it would be nice to have a minimal *.tex file using > the new features so that it may be put in the test suite (or a > related test case amended). Definitely. Georg
Re: Many tex2lyx failing now on master
On 2015-10-20, Georg Baum wrote: > Guenter Milde wrote: > If you think that anything else is missing in the documentation of file > format changes or concerning the tex2lyx tests, then please tell what you'd > expect, and we can improve the docs. Maybe a suggestion to provide a minimal example of the new feature requiring the file format change in form of a *.lyx or *.tex file that can either be simply added to the test base (if tex2lyx works fine) or put on in a set of "TODO examples". Whether we provide a directory for such incoming test examples (so that test-experts can review them and bring them in line with the other tests) or ask to send them to the devel list or whatever should be agreed on by people caring for the test suite. Günter
Re: Many tex2lyx failing now on master
On 2015-10-20, Georg Baum wrote: > Guenter Milde wrote: >> I updated the lyxformat number in the src/tex2lyx/test/*.lyx.lyx files and >> now all tex2lyx tests pass here. > Very good. > Just to avoid future errors: I assume that you did check before that a > simple update of the format number is OK, and that no tex2lyx source code > changes are needed? How to do such checks in described in section 2.3 of > Development.lyx, item 5. Actually, no. (My aim was just to ensure the tests work again, not to make a full review of the fileformat change.) >From the posts in this thread, I realized that the version number was changed without updating the reference files. I run the tests and found that only the *.lyx.lyx reference files produced errors, and the only difference was the version number. Then I changed just the last digit of the number via a grep-replace in my text editor. This is the method to fix false positives I am used to from my work on test driven development of "Docutils". It ensures that no change can go unnoticed. I did not touch any files with LyX but also not use "make updatetests". Finally I run "make alltests" in lyxbuild/src/tex2lyx and all tests passed. @Uwe: could you please check, if the improvements that led to the file format change require changes in tex2lyx and either add a note to "src/tex2lyx/TODO.txt" if yes? In any case, it would be nice to have a minimal *.tex file using the new features so that it may be put in the test suite (or a related test case amended). Günter
Re: Many tex2lyx failing now on master
Am Dienstag, 20. Oktober 2015 um 20:18:33, schrieb Georg Baum > Guenter Milde wrote: > > > On 2015-10-19, Uwe Stöhr wrote: > > > >> I don't understand what I made wrong. I only changed something in a > >> module and added a fileformat change for lyx2lyx. I did not change any > >> tex2lxy test file nor did I change tex2lyx code. > > It is all documented in Development.lyx. > > >> I also followed lib/doc/Development.lyx sec. 2.3. This references sec. > >> 3.2.2 and I tried that too, but this is not possible on my PC: > > If a procedure that is documented to be required for certain changes does > not work, then I would expect every developer to ask on the list for help > _before submitting_ instead of ignoring the issue. > > > If the hint that a fileformat change requires adapting the test examples > > is missing there, this is a documentation bug. > > The hint is there, in section 2.3. > > > In addition, adding the "change the fileformat numer for tests" > > requirement to Development.lyx would help prevent others to fall in this > > trap and frustration by others frentically trying to get the tests in > > shape before a release. > > It is already in. The only thing which is indeed missing is a description > how running the tests and updating the test references works with MSVC. > Unfortunately I don't know that, otherwise I would have documented it. > Vincent, maybe you can find that out? This is easy, because it is not platform specific. The command 'ctest' comes together with cmake, so everyone using cmake has it too. > If you think that anything else is missing in the documentation of file > format changes or concerning the tex2lyx tests, then please tell what you'd > expect, and we can improve the docs. > > > Georg Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Guenter Milde wrote: > On 2015-10-19, Uwe Stöhr wrote: > >> I don't understand what I made wrong. I only changed something in a >> module and added a fileformat change for lyx2lyx. I did not change any >> tex2lxy test file nor did I change tex2lyx code. It is all documented in Development.lyx. >> I also followed lib/doc/Development.lyx sec. 2.3. This references sec. >> 3.2.2 and I tried that too, but this is not possible on my PC: If a procedure that is documented to be required for certain changes does not work, then I would expect every developer to ask on the list for help _before submitting_ instead of ignoring the issue. > If the hint that a fileformat change requires adapting the test examples > is missing there, this is a documentation bug. The hint is there, in section 2.3. > In addition, adding the "change the fileformat numer for tests" > requirement to Development.lyx would help prevent others to fall in this > trap and frustration by others frentically trying to get the tests in > shape before a release. It is already in. The only thing which is indeed missing is a description how running the tests and updating the test references works with MSVC. Unfortunately I don't know that, otherwise I would have documented it. Vincent, maybe you can find that out? If you think that anything else is missing in the documentation of file format changes or concerning the tex2lyx tests, then please tell what you'd expect, and we can improve the docs. Georg
Re: Many tex2lyx failing now on master
Guenter Milde wrote: > I updated the lyxformat number in the src/tex2lyx/test/*.lyx.lyx files and > now all tex2lyx tests pass here. Very good. Just to avoid future errors: I assume that you did check before that a simple update of the format number is OK, and that no tex2lyx source code changes are needed? How to do such checks in described in section 2.3 of Development.lyx, item 5. Georg
Re: Many tex2lyx failing now on master
On 2015-10-19, Scott Kostyshak wrote: > On Mon, Oct 19, 2015 at 07:38:12PM +0100, Guillaume Munch wrote: >> Le 19/10/2015 19:12, Scott Kostyshak a écrit : >> >Many tex2lyx tests have started to fail (before it was just 4 and we >> >knew why and I think Günter's recent commit might have fixed them?). Now >> >14 are failing. ... >> I see lots of diffs of the form >> #LyX file created by tex2lyx 2.2 >> -\lyxformat 497 >> +\lyxformat 498 >> \begin_document >> \begin_header >> \origin roundtrip >> and the commit ce933b1e14 has incremented the format to 498. Could that >> alone make the tests fail? > It could be. Please do not spend any time on this right now though. > Let's first see if Uwe has an idea of how to fix them. I updated the lyxformat number in the src/tex2lyx/test/*.lyx.lyx files and now all tex2lyx tests pass here. Commited at a0e2d48a569510dce6f3f288ee7a14e239cd4de2/lyxgit Hope this helps, Günter
Re: Many tex2lyx failing now on master
On 2015-10-19, Uwe Stöhr wrote: > Am 19.10.2015 um 20:55 schrieb Scott Kostyshak: >> There is some at lib/doc/Development.lyx >>> I see lots of diffs of the form >>> #LyX file created by tex2lyx 2.2 >>> -\lyxformat 497 >>> +\lyxformat 498 >>> and the commit ce933b1e14 has incremented the format to 498. Could that >>> alone make the tests fail? > I don't understand what I made wrong. I only changed something in a > module and added a fileformat change for lyx2lyx. I did not change any > tex2lxy test file nor did I change tex2lyx code. A fileformat change is IMO a "big" change that should be agreed with the release manage so short before an alpha release. As the tex2lyx tests compare the result of converting a *.tex file with a stored example of the expected LyX file, they need updating with every change to the default settings and bits in these LyX files, including the fileformat number. > I also followed lib/doc/Development.lyx sec. 2.3. This references sec. > 3.2.2 and I tried that too, but this is not possible on my PC: If the hint that a fileformat change requires adapting the test examples is missing there, this is a documentation bug. > So if anybody could please tell me what I should do I will do this and > also update lib/doc/Development.lyx accordingly. If there is agreement to the fileformat change, replacing the fileformat number in the *.lyx and *.lyx.lyx files in the tex2lyx tests would (hopefully) solve the issue. I can confirm that before these changes but applying my patches for "combining lines below", tests passed. In addition, adding the "change the fileformat numer for tests" requirement to Development.lyx would help prevent others to fall in this trap and frustration by others frentically trying to get the tests in shape before a release. Günter
Re: Many tex2lyx failing now on master
Am Dienstag, 20. Oktober 2015 um 01:04:36, schrieb Uwe Stöhr > Am 19.10.2015 um 20:55 schrieb Scott Kostyshak: > > > There is some at lib/doc/Development.lyx > >> I see lots of diffs of the form > >> > >> #LyX file created by tex2lyx 2.2 > >> -\lyxformat 497 > >> +\lyxformat 498 > >> and the commit ce933b1e14 has incremented the format to 498. Could that > >> alone make the tests fail? > > I don't understand what I made wrong. I only changed something in a > module and added a fileformat change for lyx2lyx. It is this 'fileformat change'. The files (*.lyx.lyx) created now by tex2lyx are now different (the first 2 lines of a created file reads #LyX file created by tex2lyx 2.2 \lyxformat 497 ) > I did not change any > tex2lxy test file nor did I change tex2lyx code. > > I also followed lib/doc/Development.lyx sec. 2.3. This references sec. > 3.2.2 and I tried that too, but this is not possible on my PC: > > I use CMake. So I opened a MSVC console with all build commands loaded. > The result is: > > D:\LyXGit\Master\src\tex2lyx\test>make updatetex2lyxtests > MAKE Version 5.2 Copyright (c) 1987, 1998 Inprise Corp. > Fatal: Unable to open makefile You should use the correct target for your platform. This 'make updatetex2lyxtests' is meant for linux. For Windows it may be something with appropriate solution, e.g. 'msbuild updatetex2lyxtests.sln' (I am guessing here, Vincent will know) > So if anybody could please tell me what I should do I will do this and > also update lib/doc/Development.lyx accordingly. > > thanks and regards > Uwe Kornel signature.asc Description: This is a digitally signed message part.
Re: Many tex2lyx failing now on master
Am 19.10.2015 um 20:55 schrieb Scott Kostyshak: There is some at lib/doc/Development.lyx I see lots of diffs of the form #LyX file created by tex2lyx 2.2 -\lyxformat 497 +\lyxformat 498 and the commit ce933b1e14 has incremented the format to 498. Could that alone make the tests fail? I don't understand what I made wrong. I only changed something in a module and added a fileformat change for lyx2lyx. I did not change any tex2lxy test file nor did I change tex2lyx code. I also followed lib/doc/Development.lyx sec. 2.3. This references sec. 3.2.2 and I tried that too, but this is not possible on my PC: I use CMake. So I opened a MSVC console with all build commands loaded. The result is: D:\LyXGit\Master\src\tex2lyx\test>make updatetex2lyxtests MAKE Version 5.2 Copyright (c) 1987, 1998 Inprise Corp. Fatal: Unable to open makefile So if anybody could please tell me what I should do I will do this and also update lib/doc/Development.lyx accordingly. thanks and regards Uwe
Re: Many tex2lyx failing now on master
Scott Kostyshak wrote: > Many tex2lyx tests have started to fail (before it was just 4 and we > knew why and I think Günter's recent commit might have fixed them?). Now > 14 are failing. If you look at the differences between the test references and the actual results you see only changed format numbers 497 -> 498. This makes it very clear that the failures are caused by ce933b1e. I am not going to find out whether the references can simply be updated, or whether tex2lyx does now produce a broken file format. If a certain developer continues to treat LyX development as a one way push to git without any feedback path and only comes asking if he has compilation problems, I really have to hold my horses not to make git spit back his changes (i.e. reverting them). Georg
Re: Many tex2lyx failing now on master
On Mon, Oct 19, 2015 at 07:38:12PM +0100, Guillaume Munch wrote: > Le 19/10/2015 19:12, Scott Kostyshak a écrit : > >Many tex2lyx tests have started to fail (before it was just 4 and we > >knew why and I think Günter's recent commit might have fixed them?). Now > >14 are failing. > > > >Scott > > > > > I would have liked to help with tests, but when I tried to run "make > alltests" before committing and tried to read various log files I could > find, none contained useful informations. It was also before Günter's > commits, so I assumed that it was normal that some failed. There really > ought to be some documentation about tests. Yes we need better documentation. > Did I miss it? There is some at lib/doc/Development.lyx The tex2lyx tests are the most important and are also very fast to run. We're still working on getting the export tests in shape and we do need to document how to run them and how to interpret them. > Anyway now I tried it again and before: > > “Converting the following files failed: test.ltx, algo2e.tex, > box-color-size-space-align.tex, CJK.tex, CJKutf8.tex, test-insets.tex, > test-insets-basic.tex, test-memoir.tex, test-modules.tex, > test-refstyle-theorems.tex, test-scr.tex, test-structure.tex, verbatim.tex, > XeTeX-polyglossia.tex” > > I see lots of diffs of the form > > #LyX file created by tex2lyx 2.2 > -\lyxformat 497 > +\lyxformat 498 > \begin_document > \begin_header > \origin roundtrip > > and the commit ce933b1e14 has incremented the format to 498. Could that > alone make the tests fail? It could be. Please do not spend any time on this right now though. Let's first see if Uwe has an idea of how to fix them. Scott
Re: Many tex2lyx failing now on master
Le 19/10/2015 19:55, Scott Kostyshak a écrit : It could be. Please do not spend any time on this right now though. Let's first see if Uwe has an idea of how to fix them. Just making sure I'm not responsible for breaking things again :)
Re: Many tex2lyx failing now on master
On Mon, Oct 19, 2015 at 02:12:16PM -0400, Scott Kostyshak wrote: > Many tex2lyx tests have started to fail (before it was just 4 and we > knew why and I think Günter's recent commit might have fixed them?). Now > 14 are failing. They start failing for me because of ce933b1e. Uwe are you able to run the tests? By the way, do you use CMake? Scott
Re: Many tex2lyx failing now on master
Le 19/10/2015 19:12, Scott Kostyshak a écrit : Many tex2lyx tests have started to fail (before it was just 4 and we knew why and I think Günter's recent commit might have fixed them?). Now 14 are failing. Scott I would have liked to help with tests, but when I tried to run "make alltests" before committing and tried to read various log files I could find, none contained useful informations. It was also before Günter's commits, so I assumed that it was normal that some failed. There really ought to be some documentation about tests. Did I miss it? Anyway now I tried it again and before: “Converting the following files failed: test.ltx, algo2e.tex, box-color-size-space-align.tex, CJK.tex, CJKutf8.tex, test-insets.tex, test-insets-basic.tex, test-memoir.tex, test-modules.tex, test-refstyle-theorems.tex, test-scr.tex, test-structure.tex, verbatim.tex, XeTeX-polyglossia.tex” I see lots of diffs of the form #LyX file created by tex2lyx 2.2 -\lyxformat 497 +\lyxformat 498 \begin_document \begin_header \origin roundtrip and the commit ce933b1e14 has incremented the format to 498. Could that alone make the tests fail? Guillaume
