On Tue, Jan 8, 2013 at 4:10 PM, Georg Baum <[email protected]> wrote: > Scott Kostyshak wrote: > >> Dear Developers, >> >> I'm a newbie trying to figure out different aspects of a large >> programming project. One thing that I'm curious about is why there are >> so few tests for LyX. >> >> In particular, I'm interested in the following two questions: >> >> 1) Why don't you write tests? > > I write tests if I know the tested area and the test infrastructure very > well, so that the needed amount of time is very small. Otherwise I don't, > since it is not fun. > >> 2) Why do you think others don't write tests? >> >> My own answers are: >> 1) I didn't see anyone else writing them and I didn't know that there >> is already a framework in LyX for writing them (autotests). >> >> 2) I wonder if the following responses cover some of the reasons: >> >> a. I do. They're called assertions. >> >> b. The time it would take me to write a test would be better spent on >> some other aspect of LyX development. >> >> c. No one runs the tests. >> >> d. It's not fun. >> >> e. Writing tests by emulating keystrokes (and sometimes having to put >> in manual pauses) is not elegant and is fragile. > > I don't know any method of GUI testing which is not fragile. If anybody > knows one, please tell. > >> f. The initial setup (dependencies, and cmake vs. autotools) of the >> getting the tests to work is too annoying. >> >> >> Some tests seem difficult to write. For example, I find Tommmaso's >> advanced find tests to be very creative but I imagine they were >> time-consuming to write. Thus, my next question is: >> >> 3) What are the types of tests that are the most useful in the context >> of LyX and is there anything I can do to make writing those tests >> easier? >> >> My own attempt at an answer: >> The tests that are the most useful are those that are the easiest to >> write (because then we will actually write them), which in LyX are >> tests that >> (a) can be expressed by a command sequence, and >> (b) trigger an assertion or crash (so there's no need to redirect with >> a debug flag to a log and use pcregrep; and because a crash is often a >> bad type of bug from the user perspective). >> >> Currently, writing this type of tests is pretty easy, but perhaps it >> could be made even easier by just asking the developer to add one line >> to a file, say LFUNcrashes.txt: >> >> command-sequence lfun1; lfun2; lfun3 # see [trac bug number] >> >> Any ideas? > > Regarding the "easy to write" I agree 100%. I did not look at the autotests > in detail yet, so I can't comment further on this part. There are however > two other areas that could be tested quite easily: > > a) Export of .lyx documents (to .tex, .html, whatever) > b) lyx2lyx > > In both cases you would have a set of input files, and compare the generated > output with a reference. I bet that introducing those tests would find bugs > (e.g. by first producing the references with 2.0 and then compare with the > current output). Such tests would probably require a bit more infrastructure > than the tex2lyx tests (which are very simioar otherwise).
OK this is good to know and does seem easy and useful. I might look into this in a few months but since I haven't worked with exporting much I might not get to it for a while longer. Thanks for the comments. Scott
